2023-08-25T08:29:59.680 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-25T08:29:59.681 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270 --name rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/orig.config.yaml 2023-08-25T08:29:59.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-25T08:29:59.824 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270 branch: wip-rishabh-2023aug3-b5 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7379270' kernel: client: branch: testing flavor: default kdb: true sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023aug3-b5 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: test-nfs-pr-52556 sha1: 4b369cf18ed1391a426ab4ae86da834e9c074f81 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 sleep_before_teardown: 0 suite: fs suite_branch: test-nfs-pr-52556 suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 4b369cf18ed1391a426ab4ae86da834e9c074f81 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi099.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi157.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 449a1bc2027504e7b3c3d7b30fa4178906581da7 timestamp: 2023-08-25_06:38:25 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-25T08:29:59.825 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa; will attempt to use it 2023-08-25T08:29:59.825 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa/tasks 2023-08-25T08:29:59.826 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-25T08:29:59.828 INFO:teuthology.task.internal:Checking packages... 2023-08-25T08:29:59.856 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'ba6b39e4f4f77fb95c747099b53db0467d435f56' 2023-08-25T08:29:59.857 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-25T08:29:59.857 INFO:teuthology.packaging:ref: None 2023-08-25T08:29:59.857 INFO:teuthology.packaging:tag: None 2023-08-25T08:29:59.858 INFO:teuthology.packaging:branch: wip-rishabh-2023aug3-b5 2023-08-25T08:29:59.858 INFO:teuthology.packaging:sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:29:59.858 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023aug3-b5 2023-08-25T08:30:00.071 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5703.gba6b39e4 2023-08-25T08:30:00.072 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-25T08:30:00.078 INFO:teuthology.task.internal:no buildpackages task found 2023-08-25T08:30:00.078 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-25T08:30:00.130 INFO:teuthology.task.internal:Saving configuration 2023-08-25T08:30:00.176 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-25T08:30:00.183 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T08:30:00.209 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 08:24:41.311634', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T08:30:00.231 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 08:24:41.312701', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T08:30:00.255 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 08:24:41.309983', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T08:30:00.256 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-25T08:30:00.281 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-08-25T08:30:00.281 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-08-25T08:30:00.281 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-08-25T08:30:00.281 INFO:teuthology.run_tasks:Running task console_log... 2023-08-25T08:30:00.470 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f93339c2f70>, signals=[15]) 2023-08-25T08:30:00.474 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-25T08:30:00.480 INFO:teuthology.task.internal:Opening connections... 2023-08-25T08:30:00.481 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T08:30:00.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:30:00.554 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2023-08-25T08:30:00.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:30:00.624 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2023-08-25T08:30:00.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:30:00.694 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-25T08:30:00.701 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2023-08-25T08:30:00.723 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2023-08-25T08:30:00.723 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-08-25T08:30:00.782 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Red Hat Enterprise Linux" 2023-08-25T08:30:00.783 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8.6 (Ootpa)" 2023-08-25T08:30:00.783 INFO:teuthology.orchestra.run.smithi099.stdout:ID="rhel" 2023-08-25T08:30:00.783 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="fedora" 2023-08-25T08:30:00.784 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8.6" 2023-08-25T08:30:00.784 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2023-08-25T08:30:00.784 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-25T08:30:00.784 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-08-25T08:30:00.784 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-25T08:30:00.785 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.redhat.com/" 2023-08-25T08:30:00.785 INFO:teuthology.orchestra.run.smithi099.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-25T08:30:00.785 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T08:30:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:30:00.786 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-25T08:30:00.786 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-25T08:30:00.786 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-25T08:30:00.786 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-25T08:30:00.788 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-08-25T08:30:00.811 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-08-25T08:30:00.833 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-08-25T08:30:00.834 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-08-25T08:30:00.893 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2023-08-25T08:30:00.894 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.6 (Ootpa)" 2023-08-25T08:30:00.894 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2023-08-25T08:30:00.894 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2023-08-25T08:30:00.894 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.6" 2023-08-25T08:30:00.894 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-08-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-08-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2023-08-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T08:30:00.896 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:30:00.896 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-25T08:30:00.896 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-25T08:30:00.896 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-25T08:30:00.896 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-25T08:30:00.897 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-08-25T08:30:00.919 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-08-25T08:30:00.940 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-08-25T08:30:00.940 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-08-25T08:30:00.999 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2023-08-25T08:30:01.000 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.6 (Ootpa)" 2023-08-25T08:30:01.000 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2023-08-25T08:30:01.000 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2023-08-25T08:30:01.000 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.6" 2023-08-25T08:30:01.000 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-08-25T08:30:01.000 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-25T08:30:01.001 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-08-25T08:30:01.001 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-25T08:30:01.001 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2023-08-25T08:30:01.001 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-25T08:30:01.001 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T08:30:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:30:01.001 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-25T08:30:01.002 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-25T08:30:01.002 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-25T08:30:01.002 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-25T08:30:01.002 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-08-25T08:30:01.027 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-25T08:30:01.037 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-25T08:30:01.077 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-25T08:30:01.078 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2023-08-25T08:30:01.081 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2023-08-25T08:30:01.084 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2023-08-25T08:30:01.102 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-25T08:30:01.253 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-25T08:30:01.253 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2023-08-25T08:30:01.256 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2023-08-25T08:30:01.258 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2023-08-25T08:30:01.289 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-25T08:30:01.365 INFO:teuthology.run_tasks:Running task kernel... 2023-08-25T08:30:01.387 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'} 2023-08-25T08:30:01.388 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-08-25T08:30:01.389 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-08-25T08:30:01.389 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T08:30:01.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T08:30:01.409 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-08-25T08:30:01.714 DEBUG:teuthology.task.kernel:sha1 for client.0 is a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T08:30:01.714 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a6a69670761025e039dc5ae86619b0cf04c46dc2"... 2023-08-25T08:30:01.714 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-08-25T08:30:01.734 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-25T08:30:01.734 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T08:30:01.735 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-25T08:30:01.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum upgrade -y linux-firmware 2023-08-25T08:30:02.977 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-08-25T08:30:02.978 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2023-08-25T08:30:04.519 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-08-25T08:30:05.245 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-08-25T08:30:08.753 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 434 kB/s | 24 kB 00:00 2023-08-25T08:30:09.485 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-08-25T08:30:09.487 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-08-25T08:30:09.487 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-08-25T08:30:09.617 INFO:teuthology.task.kernel:Downloading kernel a6a69670761025e039dc5ae86619b0cf04c46dc2 on client.0... 2023-08-25T08:30:09.618 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T08:30:09.766 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/8/flavors/default/x86_64/ 2023-08-25T08:30:09.767 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_ga6a696707610-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/8/flavors/default/x86_64/ --input-file=- 2023-08-25T08:30:09.769 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi099.front.sepia.ceph.com to complete... 2023-08-25T08:30:22.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25 08:30:22 URL:https://3.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_ga6a696707610-1.x86_64.rpm [380759896/380759896] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-25T08:30:22.614 INFO:teuthology.orchestra.run.smithi099.stderr:FINISHED --2023-08-25 08:30:22-- 2023-08-25T08:30:22.614 INFO:teuthology.orchestra.run.smithi099.stderr:Total wall clock time: 13s 2023-08-25T08:30:22.614 INFO:teuthology.orchestra.run.smithi099.stderr:Downloaded: 1 files, 363M in 13s (28.9 MB/s) 2023-08-25T08:30:22.615 INFO:teuthology.task.kernel:Installing kernel a6a69670761025e039dc5ae86619b0cf04c46dc2 on client.0... 2023-08-25T08:30:22.616 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-25T08:30:24.217 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying... ######################################## 2023-08-25T08:30:24.272 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing... ######################################## 2023-08-25T08:30:24.363 INFO:teuthology.orchestra.run.smithi099.stdout:Updating / installing... 2023-08-25T08:31:02.456 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-6.5.0_rc7_ga6a696707610-1 ######################################## 2023-08-25T08:31:26.486 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-25T08:31:26.487 DEBUG:teuthology.orchestra.run.smithi099:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-25T08:31:26.534 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/System.map-6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.534 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/config-6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.534 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/vmlinuz-6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.535 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.535 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel 2023-08-25T08:31:26.535 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch 2023-08-25T08:31:26.535 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86 2023-08-25T08:31:26.535 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto 2023-08-25T08:31:26.535 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-25T08:31:26.535 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-25T08:31:26.536 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-25T08:31:26.536 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-25T08:31:26.536 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-25T08:31:26.536 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-25T08:31:26.536 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-25T08:31:26.536 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-25T08:31:26.536 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-25T08:31:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-25T08:31:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-25T08:31:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-25T08:31:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-25T08:31:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-25T08:31:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-25T08:31:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-25T08:31:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-25T08:31:26.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-25T08:31:26.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-25T08:31:26.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-25T08:31:26.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel 2023-08-25T08:31:26.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu 2023-08-25T08:31:26.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu/mce 2023-08-25T08:31:26.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-25T08:31:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm 2023-08-25T08:31:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-25T08:31:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-25T08:31:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm.ko 2023-08-25T08:31:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto 2023-08-25T08:31:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/ansi_cprng.ko 2023-08-25T08:31:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/anubis.ko 2023-08-25T08:31:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/arc4.ko 2023-08-25T08:31:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx 2023-08-25T08:31:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_memcpy.ko 2023-08-25T08:31:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_pq.ko 2023-08-25T08:31:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-25T08:31:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_tx.ko 2023-08-25T08:31:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_xor.ko 2023-08-25T08:31:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/raid6test.ko 2023-08-25T08:31:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/authenc.ko 2023-08-25T08:31:26.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/authencesn.ko 2023-08-25T08:31:26.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blake2b_generic.ko 2023-08-25T08:31:26.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blowfish_common.ko 2023-08-25T08:31:26.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blowfish_generic.ko 2023-08-25T08:31:26.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/camellia_generic.ko 2023-08-25T08:31:26.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast5_generic.ko 2023-08-25T08:31:26.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast6_generic.ko 2023-08-25T08:31:26.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast_common.ko 2023-08-25T08:31:26.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/ccm.ko 2023-08-25T08:31:26.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cmac.ko 2023-08-25T08:31:26.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crc32_generic.ko 2023-08-25T08:31:26.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crypto_engine.ko 2023-08-25T08:31:26.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crypto_user.ko 2023-08-25T08:31:26.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cts.ko 2023-08-25T08:31:26.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/des_generic.ko 2023-08-25T08:31:26.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/dh_generic.ko 2023-08-25T08:31:26.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/echainiv.ko 2023-08-25T08:31:26.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/essiv.ko 2023-08-25T08:31:26.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/fcrypt.ko 2023-08-25T08:31:26.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/khazad.ko 2023-08-25T08:31:26.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/lz4.ko 2023-08-25T08:31:26.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/lz4hc.ko 2023-08-25T08:31:26.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/md4.ko 2023-08-25T08:31:26.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/michael_mic.ko 2023-08-25T08:31:26.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/pcbc.ko 2023-08-25T08:31:26.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/pcrypt.ko 2023-08-25T08:31:26.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/rmd160.ko 2023-08-25T08:31:26.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/seed.ko 2023-08-25T08:31:26.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/serpent_generic.ko 2023-08-25T08:31:26.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/tcrypt.ko 2023-08-25T08:31:26.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/tea.ko 2023-08-25T08:31:26.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/twofish_common.ko 2023-08-25T08:31:26.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/twofish_generic.ko 2023-08-25T08:31:26.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/vmac.ko 2023-08-25T08:31:26.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/wp512.ko 2023-08-25T08:31:26.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xcbc.ko 2023-08-25T08:31:26.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xor.ko 2023-08-25T08:31:26.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xxhash_generic.ko 2023-08-25T08:31:26.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers 2023-08-25T08:31:26.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi 2023-08-25T08:31:26.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-25T08:31:26.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/acpi_pad.ko 2023-08-25T08:31:26.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/custom_method.ko 2023-08-25T08:31:26.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/ec_sys.ko 2023-08-25T08:31:26.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/platform_profile.ko 2023-08-25T08:31:26.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/sbs.ko 2023-08-25T08:31:26.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/sbshc.ko 2023-08-25T08:31:26.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/video.ko 2023-08-25T08:31:26.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata 2023-08-25T08:31:26.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/acard-ahci.ko 2023-08-25T08:31:26.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/ahci_platform.ko 2023-08-25T08:31:26.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/ata_generic.ko 2023-08-25T08:31:26.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/libahci_platform.ko 2023-08-25T08:31:26.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_acpi.ko 2023-08-25T08:31:26.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ali.ko 2023-08-25T08:31:26.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_amd.ko 2023-08-25T08:31:26.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_artop.ko 2023-08-25T08:31:26.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_atiixp.ko 2023-08-25T08:31:26.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_atp867x.ko 2023-08-25T08:31:26.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cmd640.ko 2023-08-25T08:31:26.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cmd64x.ko 2023-08-25T08:31:26.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cypress.ko 2023-08-25T08:31:26.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_efar.ko 2023-08-25T08:31:26.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt366.ko 2023-08-25T08:31:26.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt37x.ko 2023-08-25T08:31:26.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-25T08:31:26.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-25T08:31:26.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_it8213.ko 2023-08-25T08:31:26.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_it821x.ko 2023-08-25T08:31:26.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_jmicron.ko 2023-08-25T08:31:26.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_marvell.ko 2023-08-25T08:31:26.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_mpiix.ko 2023-08-25T08:31:26.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_netcell.ko 2023-08-25T08:31:26.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ninja32.ko 2023-08-25T08:31:26.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ns87410.ko 2023-08-25T08:31:26.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ns87415.ko 2023-08-25T08:31:26.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_oldpiix.ko 2023-08-25T08:31:26.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_opti.ko 2023-08-25T08:31:26.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_optidma.ko 2023-08-25T08:31:26.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pcmcia.ko 2023-08-25T08:31:26.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-25T08:31:26.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-25T08:31:26.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_piccolo.ko 2023-08-25T08:31:26.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_rdc.ko 2023-08-25T08:31:26.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sch.ko 2023-08-25T08:31:26.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_serverworks.ko 2023-08-25T08:31:26.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sil680.ko 2023-08-25T08:31:26.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sis.ko 2023-08-25T08:31:26.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sl82c105.ko 2023-08-25T08:31:26.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_triflex.ko 2023-08-25T08:31:26.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_via.ko 2023-08-25T08:31:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pdc_adma.ko 2023-08-25T08:31:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_inic162x.ko 2023-08-25T08:31:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_mv.ko 2023-08-25T08:31:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_nv.ko 2023-08-25T08:31:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_promise.ko 2023-08-25T08:31:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_qstor.ko 2023-08-25T08:31:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sil.ko 2023-08-25T08:31:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sil24.ko 2023-08-25T08:31:26.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sis.ko 2023-08-25T08:31:26.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_svw.ko 2023-08-25T08:31:26.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sx4.ko 2023-08-25T08:31:26.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_uli.ko 2023-08-25T08:31:26.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_via.ko 2023-08-25T08:31:26.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_vsc.ko 2023-08-25T08:31:26.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/auxdisplay 2023-08-25T08:31:26.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/auxdisplay/ks0108.ko 2023-08-25T08:31:26.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base 2023-08-25T08:31:26.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap 2023-08-25T08:31:26.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-25T08:31:26.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-25T08:31:26.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/bcma 2023-08-25T08:31:26.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/bcma/bcma.ko 2023-08-25T08:31:26.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block 2023-08-25T08:31:26.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/aoe 2023-08-25T08:31:26.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/aoe/aoe.ko 2023-08-25T08:31:26.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/brd.ko 2023-08-25T08:31:26.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/drbd 2023-08-25T08:31:26.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/drbd/drbd.ko 2023-08-25T08:31:26.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/floppy.ko 2023-08-25T08:31:26.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/loop.ko 2023-08-25T08:31:26.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/mtip32xx 2023-08-25T08:31:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-25T08:31:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/nbd.ko 2023-08-25T08:31:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/null_blk 2023-08-25T08:31:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/null_blk/null_blk.ko 2023-08-25T08:31:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/pktcdvd.ko 2023-08-25T08:31:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/rbd.ko 2023-08-25T08:31:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/virtio_blk.ko 2023-08-25T08:31:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkback 2023-08-25T08:31:26.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-25T08:31:26.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkfront.ko 2023-08-25T08:31:26.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/zram 2023-08-25T08:31:26.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/zram/zram.ko 2023-08-25T08:31:26.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char 2023-08-25T08:31:26.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hangcheck-timer.ko 2023-08-25T08:31:26.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random 2023-08-25T08:31:26.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-25T08:31:26.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-25T08:31:26.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-25T08:31:26.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/via-rng.ko 2023-08-25T08:31:26.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-25T08:31:26.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi 2023-08-25T08:31:26.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-25T08:31:26.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-25T08:31:26.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-25T08:31:26.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-25T08:31:26.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-25T08:31:26.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-25T08:31:26.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/lp.ko 2023-08-25T08:31:26.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/mwave 2023-08-25T08:31:26.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/mwave/mwave.ko 2023-08-25T08:31:26.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ppdev.ko 2023-08-25T08:31:26.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tlclk.ko 2023-08-25T08:31:26.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm 2023-08-25T08:31:26.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm.ko 2023-08-25T08:31:26.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-25T08:31:26.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-25T08:31:26.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-25T08:31:26.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-25T08:31:26.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-25T08:31:26.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/uv_mmtimer.ko 2023-08-25T08:31:26.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/virtio_console.ko 2023-08-25T08:31:26.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq 2023-08-25T08:31:26.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-25T08:31:26.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-25T08:31:26.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-25T08:31:26.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-25T08:31:26.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-25T08:31:26.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-25T08:31:26.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto 2023-08-25T08:31:26.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp 2023-08-25T08:31:26.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-25T08:31:26.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp/ccp.ko 2023-08-25T08:31:26.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel 2023-08-25T08:31:26.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat 2023-08-25T08:31:26.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_common 2023-08-25T08:31:26.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-25T08:31:26.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-25T08:31:26.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-25T08:31:26.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/padlock-aes.ko 2023-08-25T08:31:26.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/padlock-sha.ko 2023-08-25T08:31:26.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/virtio 2023-08-25T08:31:26.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-25T08:31:26.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dca 2023-08-25T08:31:26.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dca/dca.ko 2023-08-25T08:31:26.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/devfreq 2023-08-25T08:31:26.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-25T08:31:26.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma 2023-08-25T08:31:26.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/dw 2023-08-25T08:31:26.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-25T08:31:26.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/ioat 2023-08-25T08:31:26.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-25T08:31:26.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac 2023-08-25T08:31:26.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/amd64_edac.ko 2023-08-25T08:31:26.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/e752x_edac.ko 2023-08-25T08:31:26.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/edac_mce_amd.ko 2023-08-25T08:31:26.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i3000_edac.ko 2023-08-25T08:31:26.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i3200_edac.ko 2023-08-25T08:31:26.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i5100_edac.ko 2023-08-25T08:31:26.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i5400_edac.ko 2023-08-25T08:31:26.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i7300_edac.ko 2023-08-25T08:31:26.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i7core_edac.ko 2023-08-25T08:31:26.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i82975x_edac.ko 2023-08-25T08:31:26.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/ie31200_edac.ko 2023-08-25T08:31:26.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/sb_edac.ko 2023-08-25T08:31:26.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/x38_edac.ko 2023-08-25T08:31:26.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware 2023-08-25T08:31:26.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware/edd.ko 2023-08-25T08:31:26.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-25T08:31:26.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio 2023-08-25T08:31:26.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio/gpio-ich.ko 2023-08-25T08:31:26.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-25T08:31:26.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid 2023-08-25T08:31:26.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-appleir.ko 2023-08-25T08:31:26.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-aureal.ko 2023-08-25T08:31:26.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-axff.ko 2023-08-25T08:31:26.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-dr.ko 2023-08-25T08:31:26.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-elecom.ko 2023-08-25T08:31:26.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-elo.ko 2023-08-25T08:31:26.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-emsff.ko 2023-08-25T08:31:26.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gaff.ko 2023-08-25T08:31:26.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gt683r.ko 2023-08-25T08:31:26.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gyration.ko 2023-08-25T08:31:26.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-25T08:31:26.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-25T08:31:26.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtekff.ko 2023-08-25T08:31:26.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-hyperv.ko 2023-08-25T08:31:26.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-icade.ko 2023-08-25T08:31:26.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-keytouch.ko 2023-08-25T08:31:26.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-kye.ko 2023-08-25T08:31:26.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-lcpower.ko 2023-08-25T08:31:26.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-led.ko 2023-08-25T08:31:26.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-lenovo.ko 2023-08-25T08:31:26.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-25T08:31:26.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-25T08:31:26.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-multitouch.ko 2023-08-25T08:31:26.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-ortek.ko 2023-08-25T08:31:26.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-penmount.ko 2023-08-25T08:31:26.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-petalynx.ko 2023-08-25T08:31:26.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-picolcd.ko 2023-08-25T08:31:26.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-pl.ko 2023-08-25T08:31:26.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-plantronics.ko 2023-08-25T08:31:26.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-primax.ko 2023-08-25T08:31:26.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-rmi.ko 2023-08-25T08:31:26.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-25T08:31:26.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-common.ko 2023-08-25T08:31:26.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-25T08:31:26.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-25T08:31:26.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-25T08:31:26.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-25T08:31:26.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-25T08:31:26.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-25T08:31:26.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-25T08:31:26.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-25T08:31:26.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-25T08:31:26.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat.ko 2023-08-25T08:31:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-saitek.ko 2023-08-25T08:31:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-samsung.ko 2023-08-25T08:31:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-25T08:31:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sjoy.ko 2023-08-25T08:31:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sony.ko 2023-08-25T08:31:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-speedlink.ko 2023-08-25T08:31:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-steelseries.ko 2023-08-25T08:31:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sunplus.ko 2023-08-25T08:31:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-25T08:31:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-tivo.ko 2023-08-25T08:31:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-tmff.ko 2023-08-25T08:31:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-topseed.ko 2023-08-25T08:31:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-twinhan.ko 2023-08-25T08:31:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-uclogic.ko 2023-08-25T08:31:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-waltop.ko 2023-08-25T08:31:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-wiimote.ko 2023-08-25T08:31:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-xinmo.ko 2023-08-25T08:31:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-zpff.ko 2023-08-25T08:31:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-zydacron.ko 2023-08-25T08:31:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/uhid.ko 2023-08-25T08:31:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/wacom.ko 2023-08-25T08:31:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv 2023-08-25T08:31:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_balloon.ko 2023-08-25T08:31:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_utils.ko 2023-08-25T08:31:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_vmbus.ko 2023-08-25T08:31:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon 2023-08-25T08:31:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/abituguru.ko 2023-08-25T08:31:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/abituguru3.ko 2023-08-25T08:31:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-25T08:31:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ad7414.ko 2023-08-25T08:31:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ad7418.ko 2023-08-25T08:31:26.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adc128d818.ko 2023-08-25T08:31:26.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1025.ko 2023-08-25T08:31:26.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1026.ko 2023-08-25T08:31:26.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1029.ko 2023-08-25T08:31:26.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1031.ko 2023-08-25T08:31:26.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm9240.ko 2023-08-25T08:31:26.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ads7828.ko 2023-08-25T08:31:26.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7410.ko 2023-08-25T08:31:26.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7411.ko 2023-08-25T08:31:26.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7462.ko 2023-08-25T08:31:26.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7470.ko 2023-08-25T08:31:26.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7475.ko 2023-08-25T08:31:26.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7x10.ko 2023-08-25T08:31:26.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/amc6821.ko 2023-08-25T08:31:26.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/applesmc.ko 2023-08-25T08:31:26.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asb100.ko 2023-08-25T08:31:26.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asc7621.ko 2023-08-25T08:31:26.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-25T08:31:26.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/atxp1.ko 2023-08-25T08:31:26.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/coretemp.ko 2023-08-25T08:31:26.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-25T08:31:26.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/dme1737.ko 2023-08-25T08:31:26.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ds1621.ko 2023-08-25T08:31:26.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ds620.ko 2023-08-25T08:31:26.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/emc1403.ko 2023-08-25T08:31:26.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/emc6w201.ko 2023-08-25T08:31:26.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f71805f.ko 2023-08-25T08:31:26.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f71882fg.ko 2023-08-25T08:31:26.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f75375s.ko 2023-08-25T08:31:26.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/fam15h_power.ko 2023-08-25T08:31:26.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/fschmd.ko 2023-08-25T08:31:26.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/g760a.ko 2023-08-25T08:31:26.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/g762.ko 2023-08-25T08:31:26.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/gl518sm.ko 2023-08-25T08:31:26.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/gl520sm.ko 2023-08-25T08:31:26.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-25T08:31:26.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/i5500_temp.ko 2023-08-25T08:31:26.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/i5k_amb.ko 2023-08-25T08:31:26.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ibmaem.ko 2023-08-25T08:31:26.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ibmpex.ko 2023-08-25T08:31:26.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ina209.ko 2023-08-25T08:31:26.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ina2xx.ko 2023-08-25T08:31:26.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/it87.ko 2023-08-25T08:31:26.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/k10temp.ko 2023-08-25T08:31:26.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/k8temp.ko 2023-08-25T08:31:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lineage-pem.ko 2023-08-25T08:31:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm63.ko 2023-08-25T08:31:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm73.ko 2023-08-25T08:31:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm75.ko 2023-08-25T08:31:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm77.ko 2023-08-25T08:31:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm78.ko 2023-08-25T08:31:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm80.ko 2023-08-25T08:31:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm83.ko 2023-08-25T08:31:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm85.ko 2023-08-25T08:31:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm87.ko 2023-08-25T08:31:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm90.ko 2023-08-25T08:31:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm92.ko 2023-08-25T08:31:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm93.ko 2023-08-25T08:31:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95234.ko 2023-08-25T08:31:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95241.ko 2023-08-25T08:31:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95245.ko 2023-08-25T08:31:26.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc2945.ko 2023-08-25T08:31:26.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4151.ko 2023-08-25T08:31:26.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4215.ko 2023-08-25T08:31:26.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4222.ko 2023-08-25T08:31:26.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4245.ko 2023-08-25T08:31:26.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4260.ko 2023-08-25T08:31:26.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4261.ko 2023-08-25T08:31:26.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max16065.ko 2023-08-25T08:31:26.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max1619.ko 2023-08-25T08:31:26.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max1668.ko 2023-08-25T08:31:26.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max197.ko 2023-08-25T08:31:26.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6639.ko 2023-08-25T08:31:26.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6650.ko 2023-08-25T08:31:26.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6697.ko 2023-08-25T08:31:26.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/mcp3021.ko 2023-08-25T08:31:26.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6683.ko 2023-08-25T08:31:26.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6775-core.ko 2023-08-25T08:31:26.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6775.ko 2023-08-25T08:31:26.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct7802.ko 2023-08-25T08:31:26.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pc87360.ko 2023-08-25T08:31:26.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pc87427.ko 2023-08-25T08:31:26.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pcf8591.ko 2023-08-25T08:31:26.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus 2023-08-25T08:31:26.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-25T08:31:26.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-25T08:31:26.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-25T08:31:26.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-25T08:31:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-25T08:31:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-25T08:31:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-25T08:31:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-25T08:31:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-25T08:31:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-25T08:31:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-25T08:31:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-25T08:31:26.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/powr1220.ko 2023-08-25T08:31:26.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch5627.ko 2023-08-25T08:31:26.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch5636.ko 2023-08-25T08:31:26.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-25T08:31:26.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sht15.ko 2023-08-25T08:31:26.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sht21.ko 2023-08-25T08:31:26.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/shtc1.ko 2023-08-25T08:31:26.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sis5595.ko 2023-08-25T08:31:26.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47b397.ko 2023-08-25T08:31:26.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47m1.ko 2023-08-25T08:31:26.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47m192.ko 2023-08-25T08:31:26.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/thmc50.ko 2023-08-25T08:31:26.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp102.ko 2023-08-25T08:31:26.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp103.ko 2023-08-25T08:31:26.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp401.ko 2023-08-25T08:31:26.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp421.ko 2023-08-25T08:31:26.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/via-cputemp.ko 2023-08-25T08:31:26.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/via686a.ko 2023-08-25T08:31:26.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/vt1211.ko 2023-08-25T08:31:26.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/vt8231.ko 2023-08-25T08:31:26.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83627ehf.ko 2023-08-25T08:31:26.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83627hf.ko 2023-08-25T08:31:26.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83781d.ko 2023-08-25T08:31:26.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83791d.ko 2023-08-25T08:31:26.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83792d.ko 2023-08-25T08:31:26.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83793.ko 2023-08-25T08:31:26.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83795.ko 2023-08-25T08:31:26.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83l785ts.ko 2023-08-25T08:31:26.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83l786ng.ko 2023-08-25T08:31:26.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c 2023-08-25T08:31:26.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos 2023-08-25T08:31:26.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-25T08:31:26.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-25T08:31:26.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses 2023-08-25T08:31:26.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-25T08:31:26.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-25T08:31:26.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-25T08:31:26.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-25T08:31:26.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-25T08:31:26.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-25T08:31:26.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-25T08:31:26.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-25T08:31:26.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-25T08:31:26.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-25T08:31:26.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-25T08:31:26.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-25T08:31:26.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-25T08:31:26.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-25T08:31:26.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-25T08:31:26.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-25T08:31:26.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-25T08:31:26.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-25T08:31:26.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-25T08:31:26.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-25T08:31:26.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-25T08:31:26.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-25T08:31:26.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-25T08:31:26.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-dev.ko 2023-08-25T08:31:26.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-mux.ko 2023-08-25T08:31:26.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-25T08:31:26.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-smbus.ko 2023-08-25T08:31:26.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-stub.ko 2023-08-25T08:31:26.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband 2023-08-25T08:31:26.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core 2023-08-25T08:31:26.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-25T08:31:26.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_core.ko 2023-08-25T08:31:26.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-25T08:31:26.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-25T08:31:26.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-25T08:31:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-25T08:31:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-25T08:31:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw 2023-08-25T08:31:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/cxgb4 2023-08-25T08:31:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-25T08:31:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx4 2023-08-25T08:31:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-25T08:31:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx5 2023-08-25T08:31:26.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-25T08:31:26.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mthca 2023-08-25T08:31:26.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-25T08:31:26.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp 2023-08-25T08:31:26.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/ipoib 2023-08-25T08:31:26.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-25T08:31:26.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/iser 2023-08-25T08:31:26.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-25T08:31:26.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/isert 2023-08-25T08:31:26.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-25T08:31:26.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srp 2023-08-25T08:31:26.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-25T08:31:26.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srpt 2023-08-25T08:31:26.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-25T08:31:26.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input 2023-08-25T08:31:26.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport 2023-08-25T08:31:26.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-25T08:31:26.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-25T08:31:26.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/gameport.ko 2023-08-25T08:31:26.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/lightning.ko 2023-08-25T08:31:26.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/ns558.ko 2023-08-25T08:31:26.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/joydev.ko 2023-08-25T08:31:26.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/keyboard 2023-08-25T08:31:26.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-25T08:31:26.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse 2023-08-25T08:31:26.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/appletouch.ko 2023-08-25T08:31:26.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/bcm5974.ko 2023-08-25T08:31:26.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/cyapatp.ko 2023-08-25T08:31:26.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-25T08:31:26.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/sermouse.ko 2023-08-25T08:31:26.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-25T08:31:26.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-25T08:31:26.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-25T08:31:26.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/rmi4 2023-08-25T08:31:26.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-25T08:31:26.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio 2023-08-25T08:31:26.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/altera_ps2.ko 2023-08-25T08:31:26.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/arc_ps2.ko 2023-08-25T08:31:26.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-25T08:31:26.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/serio_raw.ko 2023-08-25T08:31:26.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/sparse-keymap.ko 2023-08-25T08:31:26.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu 2023-08-25T08:31:26.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu/amd 2023-08-25T08:31:26.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-25T08:31:26.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds 2023-08-25T08:31:26.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-blinkm.ko 2023-08-25T08:31:26.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-lm3530.ko 2023-08-25T08:31:26.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-lp3944.ko 2023-08-25T08:31:26.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-ss4200.ko 2023-08-25T08:31:26.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger 2023-08-25T08:31:26.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-25T08:31:26.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-25T08:31:26.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-25T08:31:26.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-25T08:31:26.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-25T08:31:26.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-25T08:31:26.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-25T08:31:26.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-25T08:31:26.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md 2023-08-25T08:31:26.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/bcache 2023-08-25T08:31:26.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/bcache/bcache.ko 2023-08-25T08:31:26.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-bio-prison.ko 2023-08-25T08:31:26.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-cache-smq.ko 2023-08-25T08:31:26.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-cache.ko 2023-08-25T08:31:26.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-crypt.ko 2023-08-25T08:31:26.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-delay.ko 2023-08-25T08:31:26.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-flakey.ko 2023-08-25T08:31:26.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-log-userspace.ko 2023-08-25T08:31:26.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-multipath.ko 2023-08-25T08:31:26.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-queue-length.ko 2023-08-25T08:31:26.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-raid.ko 2023-08-25T08:31:26.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-round-robin.ko 2023-08-25T08:31:26.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-service-time.ko 2023-08-25T08:31:26.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-switch.ko 2023-08-25T08:31:26.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-thin-pool.ko 2023-08-25T08:31:26.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-verity.ko 2023-08-25T08:31:26.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/faulty.ko 2023-08-25T08:31:26.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/linear.ko 2023-08-25T08:31:26.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/multipath.ko 2023-08-25T08:31:26.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/persistent-data 2023-08-25T08:31:26.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-25T08:31:26.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid0.ko 2023-08-25T08:31:26.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid1.ko 2023-08-25T08:31:26.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid10.ko 2023-08-25T08:31:26.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid456.ko 2023-08-25T08:31:26.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message 2023-08-25T08:31:26.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion 2023-08-25T08:31:26.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptbase.ko 2023-08-25T08:31:26.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptctl.ko 2023-08-25T08:31:26.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptfc.ko 2023-08-25T08:31:26.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptlan.ko 2023-08-25T08:31:26.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptsas.ko 2023-08-25T08:31:26.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptscsih.ko 2023-08-25T08:31:26.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptspi.ko 2023-08-25T08:31:26.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd 2023-08-25T08:31:26.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/lpc_ich.ko 2023-08-25T08:31:26.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/lpc_sch.ko 2023-08-25T08:31:26.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/mfd-core.ko 2023-08-25T08:31:26.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/sm501.ko 2023-08-25T08:31:26.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/viperboard.ko 2023-08-25T08:31:26.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/vx855.ko 2023-08-25T08:31:26.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/wl1273-core.ko 2023-08-25T08:31:26.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc 2023-08-25T08:31:26.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/altera-stapl 2023-08-25T08:31:26.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-25T08:31:26.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/apds9802als.ko 2023-08-25T08:31:26.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/apds990x.ko 2023-08-25T08:31:26.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/bh1770glc.ko 2023-08-25T08:31:26.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/cb710 2023-08-25T08:31:26.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/cb710/cb710.ko 2023-08-25T08:31:26.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom 2023-08-25T08:31:26.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/at24.ko 2023-08-25T08:31:26.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-25T08:31:26.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-25T08:31:26.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/max6875.ko 2023-08-25T08:31:26.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/enclosure.ko 2023-08-25T08:31:26.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/hpilo.ko 2023-08-25T08:31:26.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/isl29003.ko 2023-08-25T08:31:26.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/isl29020.ko 2023-08-25T08:31:26.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d 2023-08-25T08:31:26.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-25T08:31:26.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-25T08:31:26.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei 2023-08-25T08:31:26.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei-me.ko 2023-08-25T08:31:26.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei-txe.ko 2023-08-25T08:31:26.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei.ko 2023-08-25T08:31:26.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-gru 2023-08-25T08:31:26.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-25T08:31:26.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp 2023-08-25T08:31:26.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-25T08:31:26.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-25T08:31:26.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-25T08:31:26.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tifm_7xx1.ko 2023-08-25T08:31:26.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tifm_core.ko 2023-08-25T08:31:26.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tsl2550.ko 2023-08-25T08:31:26.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_balloon.ko 2023-08-25T08:31:26.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_vmci 2023-08-25T08:31:26.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-25T08:31:26.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net 2023-08-25T08:31:26.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/bonding 2023-08-25T08:31:26.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/bonding/bonding.ko 2023-08-25T08:31:26.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa 2023-08-25T08:31:26.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/b53 2023-08-25T08:31:26.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-25T08:31:26.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-25T08:31:26.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-25T08:31:26.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6xxx 2023-08-25T08:31:26.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-25T08:31:26.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dummy.ko 2023-08-25T08:31:26.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/eql.ko 2023-08-25T08:31:26.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet 2023-08-25T08:31:26.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com 2023-08-25T08:31:26.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-25T08:31:26.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-25T08:31:26.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-25T08:31:26.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-25T08:31:26.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390 2023-08-25T08:31:26.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-25T08:31:26.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-25T08:31:26.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-25T08:31:26.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-25T08:31:26.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/adaptec 2023-08-25T08:31:26.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-25T08:31:26.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/agere 2023-08-25T08:31:26.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-25T08:31:26.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/alteon 2023-08-25T08:31:26.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-25T08:31:26.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/altera 2023-08-25T08:31:26.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-25T08:31:26.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd 2023-08-25T08:31:26.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-25T08:31:26.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-25T08:31:26.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-25T08:31:26.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros 2023-08-25T08:31:26.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/alx 2023-08-25T08:31:26.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-25T08:31:26.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-25T08:31:26.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-25T08:31:26.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-25T08:31:26.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-25T08:31:26.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx 2023-08-25T08:31:26.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-25T08:31:26.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-25T08:31:26.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom 2023-08-25T08:31:26.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-25T08:31:26.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-25T08:31:26.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-25T08:31:26.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-25T08:31:26.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-25T08:31:26.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/genet 2023-08-25T08:31:26.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-25T08:31:26.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-25T08:31:26.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade 2023-08-25T08:31:26.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade/bna 2023-08-25T08:31:26.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-25T08:31:26.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio 2023-08-25T08:31:26.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-25T08:31:26.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-25T08:31:26.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-25T08:31:26.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-25T08:31:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-25T08:31:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-25T08:31:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-25T08:31:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-25T08:31:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-25T08:31:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-25T08:31:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco 2023-08-25T08:31:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco/enic 2023-08-25T08:31:26.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-25T08:31:26.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec 2023-08-25T08:31:26.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip 2023-08-25T08:31:26.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-25T08:31:26.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-25T08:31:26.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-25T08:31:26.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-25T08:31:26.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-25T08:31:26.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-25T08:31:26.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink 2023-08-25T08:31:26.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-25T08:31:26.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-25T08:31:26.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dnet.ko 2023-08-25T08:31:26.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex 2023-08-25T08:31:26.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex/benet 2023-08-25T08:31:26.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-25T08:31:26.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ethoc.ko 2023-08-25T08:31:26.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/fealnx.ko 2023-08-25T08:31:26.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel 2023-08-25T08:31:26.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-25T08:31:26.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000 2023-08-25T08:31:26.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-25T08:31:26.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000e 2023-08-25T08:31:26.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-25T08:31:26.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/fm10k 2023-08-25T08:31:26.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-25T08:31:26.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/i40e 2023-08-25T08:31:26.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-25T08:31:26.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/iavf 2023-08-25T08:31:26.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-25T08:31:26.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igb 2023-08-25T08:31:26.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-25T08:31:26.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igbvf 2023-08-25T08:31:26.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-25T08:31:26.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-25T08:31:26.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-25T08:31:26.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-25T08:31:26.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-25T08:31:26.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/jme.ko 2023-08-25T08:31:26.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell 2023-08-25T08:31:26.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-25T08:31:26.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-25T08:31:26.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-25T08:31:26.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox 2023-08-25T08:31:26.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-25T08:31:26.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-25T08:31:26.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-25T08:31:26.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-25T08:31:26.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-25T08:31:26.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-25T08:31:26.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/micrel 2023-08-25T08:31:26.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-25T08:31:26.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom 2023-08-25T08:31:26.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-25T08:31:26.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-25T08:31:26.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi 2023-08-25T08:31:26.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-25T08:31:26.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-25T08:31:26.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/neterion 2023-08-25T08:31:26.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-25T08:31:26.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/nvidia 2023-08-25T08:31:26.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-25T08:31:26.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines 2023-08-25T08:31:26.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-25T08:31:26.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-25T08:31:26.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic 2023-08-25T08:31:26.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-25T08:31:26.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-25T08:31:26.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-25T08:31:26.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-25T08:31:26.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-25T08:31:26.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rdc 2023-08-25T08:31:26.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-25T08:31:26.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek 2023-08-25T08:31:26.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-25T08:31:26.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-25T08:31:26.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-25T08:31:26.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-25T08:31:26.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rocker 2023-08-25T08:31:26.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-25T08:31:26.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sfc 2023-08-25T08:31:26.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-25T08:31:26.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/silan 2023-08-25T08:31:26.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-25T08:31:26.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis 2023-08-25T08:31:26.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-25T08:31:26.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-25T08:31:26.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc 2023-08-25T08:31:26.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-25T08:31:26.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-25T08:31:26.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-25T08:31:26.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-25T08:31:26.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro 2023-08-25T08:31:26.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-25T08:31:26.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-25T08:31:26.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-25T08:31:26.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun 2023-08-25T08:31:26.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-25T08:31:26.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-25T08:31:26.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-25T08:31:26.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-25T08:31:26.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/tehuti 2023-08-25T08:31:26.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-25T08:31:26.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ti 2023-08-25T08:31:26.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-25T08:31:26.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via 2023-08-25T08:31:26.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-25T08:31:26.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-25T08:31:26.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet 2023-08-25T08:31:26.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-25T08:31:26.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-25T08:31:26.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/xircom 2023-08-25T08:31:26.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-25T08:31:26.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/geneve.ko 2023-08-25T08:31:26.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/hyperv 2023-08-25T08:31:26.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-25T08:31:26.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ieee802154 2023-08-25T08:31:26.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-25T08:31:26.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ifb.ko 2023-08-25T08:31:26.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ipvlan 2023-08-25T08:31:26.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-25T08:31:26.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/macvlan.ko 2023-08-25T08:31:26.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/macvtap.ko 2023-08-25T08:31:26.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio 2023-08-25T08:31:26.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio.ko 2023-08-25T08:31:26.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-25T08:31:26.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-25T08:31:26.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-25T08:31:26.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mii.ko 2023-08-25T08:31:26.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/net_failover.ko 2023-08-25T08:31:26.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/netconsole.ko 2023-08-25T08:31:26.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/nlmon.ko 2023-08-25T08:31:26.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs 2023-08-25T08:31:26.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-25T08:31:26.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-25T08:31:26.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy 2023-08-25T08:31:26.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/amd.ko 2023-08-25T08:31:26.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-25T08:31:26.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-25T08:31:26.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-25T08:31:26.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm87xx.ko 2023-08-25T08:31:26.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/broadcom.ko 2023-08-25T08:31:26.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/cicada.ko 2023-08-25T08:31:26.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/davicom.ko 2023-08-25T08:31:26.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/dp83640.ko 2023-08-25T08:31:26.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/et1011c.ko 2023-08-25T08:31:26.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/icplus.ko 2023-08-25T08:31:26.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/lxt.ko 2023-08-25T08:31:26.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/marvell.ko 2023-08-25T08:31:26.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/micrel.ko 2023-08-25T08:31:26.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/national.ko 2023-08-25T08:31:26.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/phylink.ko 2023-08-25T08:31:26.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/qsemi.ko 2023-08-25T08:31:26.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/realtek.ko 2023-08-25T08:31:26.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/smsc.ko 2023-08-25T08:31:26.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/ste10Xp.ko 2023-08-25T08:31:26.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/vitesse.ko 2023-08-25T08:31:26.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp 2023-08-25T08:31:26.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-25T08:31:26.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_async.ko 2023-08-25T08:31:26.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-25T08:31:26.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-25T08:31:26.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-25T08:31:26.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-25T08:31:26.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pppoe.ko 2023-08-25T08:31:26.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pppox.ko 2023-08-25T08:31:26.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pptp.ko 2023-08-25T08:31:26.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip 2023-08-25T08:31:26.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip/slhc.ko 2023-08-25T08:31:26.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip/slip.ko 2023-08-25T08:31:26.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/sungem_phy.ko 2023-08-25T08:31:26.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/tap.ko 2023-08-25T08:31:26.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team 2023-08-25T08:31:26.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team.ko 2023-08-25T08:31:26.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-25T08:31:26.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-25T08:31:26.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-25T08:31:26.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_random.ko 2023-08-25T08:31:26.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-25T08:31:26.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/tun.ko 2023-08-25T08:31:26.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/veth.ko 2023-08-25T08:31:26.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/virtio_net.ko 2023-08-25T08:31:26.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vmxnet3 2023-08-25T08:31:26.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-25T08:31:26.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vxlan 2023-08-25T08:31:26.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vxlan/vxlan.ko 2023-08-25T08:31:26.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netback 2023-08-25T08:31:26.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-25T08:31:26.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netfront.ko 2023-08-25T08:31:26.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ntb 2023-08-25T08:31:26.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ntb/ntb.ko 2023-08-25T08:31:26.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme 2023-08-25T08:31:26.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host 2023-08-25T08:31:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host/nvme-core.ko 2023-08-25T08:31:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host/nvme.ko 2023-08-25T08:31:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport 2023-08-25T08:31:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport.ko 2023-08-25T08:31:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_cs.ko 2023-08-25T08:31:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_pc.ko 2023-08-25T08:31:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_serial.ko 2023-08-25T08:31:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci 2023-08-25T08:31:26.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/hotplug 2023-08-25T08:31:26.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-25T08:31:26.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/pcie 2023-08-25T08:31:26.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-25T08:31:26.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/xen-pcifront.ko 2023-08-25T08:31:26.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia 2023-08-25T08:31:26.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/i82092.ko 2023-08-25T08:31:26.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/pd6729.ko 2023-08-25T08:31:26.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-25T08:31:26.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl 2023-08-25T08:31:26.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl/intel 2023-08-25T08:31:26.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-25T08:31:26.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform 2023-08-25T08:31:26.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86 2023-08-25T08:31:26.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-25T08:31:26.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/acerhdf.ko 2023-08-25T08:31:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-25T08:31:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-25T08:31:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-25T08:31:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-25T08:31:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-25T08:31:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-25T08:31:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-25T08:31:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-25T08:31:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-25T08:31:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/hdaps.ko 2023-08-25T08:31:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel 2023-08-25T08:31:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-25T08:31:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel_ips.ko 2023-08-25T08:31:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-25T08:31:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-25T08:31:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-25T08:31:26.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-25T08:31:26.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-25T08:31:26.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-25T08:31:26.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-25T08:31:26.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-25T08:31:26.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-25T08:31:26.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/wmi.ko 2023-08-25T08:31:26.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap 2023-08-25T08:31:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-25T08:31:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-25T08:31:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps 2023-08-25T08:31:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients 2023-08-25T08:31:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-25T08:31:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-25T08:31:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps_parport.ko 2023-08-25T08:31:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/pps_core.ko 2023-08-25T08:31:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp 2023-08-25T08:31:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp/ptp.ko 2023-08-25T08:31:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp/ptp_kvm.ko 2023-08-25T08:31:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc 2023-08-25T08:31:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-25T08:31:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-25T08:31:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-25T08:31:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-25T08:31:26.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-25T08:31:26.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-25T08:31:26.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-25T08:31:26.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-25T08:31:26.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-25T08:31:26.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-25T08:31:26.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-25T08:31:26.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-em3027.ko 2023-08-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-max6900.ko 2023-08-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-25T08:31:26.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-25T08:31:26.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-25T08:31:26.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-25T08:31:26.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-25T08:31:26.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-25T08:31:26.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-25T08:31:26.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-x1205.ko 2023-08-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi 2023-08-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-9xxx.ko 2023-08-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-sas.ko 2023-08-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-xxxx.ko 2023-08-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/BusLogic.ko 2023-08-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/a100u2w.ko 2023-08-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aacraid 2023-08-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/advansys.ko 2023-08-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx 2023-08-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-25T08:31:26.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-25T08:31:26.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/am53c974.ko 2023-08-25T08:31:26.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/arcmsr 2023-08-25T08:31:26.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-25T08:31:26.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/atp870u.ko 2023-08-25T08:31:26.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/be2iscsi 2023-08-25T08:31:26.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-25T08:31:26.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bfa 2023-08-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bfa/bfa.ko 2023-08-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2fc 2023-08-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2i 2023-08-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ch.ko 2023-08-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/csiostor 2023-08-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi 2023-08-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/dc395x.ko 2023-08-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler 2023-08-25T08:31:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-25T08:31:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-25T08:31:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-25T08:31:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-25T08:31:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/dmx3191d.ko 2023-08-25T08:31:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esas2r 2023-08-25T08:31:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-25T08:31:26.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esp_scsi.ko 2023-08-25T08:31:26.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe 2023-08-25T08:31:26.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-25T08:31:26.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-25T08:31:26.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fnic 2023-08-25T08:31:26.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fnic/fnic.ko 2023-08-25T08:31:26.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hpsa.ko 2023-08-25T08:31:26.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hptiop.ko 2023-08-25T08:31:26.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hv_storvsc.ko 2023-08-25T08:31:26.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/initio.ko 2023-08-25T08:31:26.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ipr.ko 2023-08-25T08:31:26.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ips.ko 2023-08-25T08:31:26.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/isci 2023-08-25T08:31:26.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/isci/isci.ko 2023-08-25T08:31:26.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libfc 2023-08-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libfc/libfc.ko 2023-08-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libiscsi.ko 2023-08-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libsas 2023-08-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libsas/libsas.ko 2023-08-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/lpfc 2023-08-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid 2023-08-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid.ko 2023-08-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mpt3sas 2023-08-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvsas 2023-08-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvumi.ko 2023-08-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pm8001 2023-08-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pmcraid.ko 2023-08-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla1280.ko 2023-08-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx 2023-08-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla4xxx 2023-08-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/raid_class.ko 2023-08-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_debug.ko 2023-08-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ses.ko 2023-08-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/st.ko 2023-08-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/stex.ko 2023-08-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/sym53c8xx_2 2023-08-25T08:31:26.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-25T08:31:26.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/virtio_scsi.ko 2023-08-25T08:31:26.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-25T08:31:26.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/wd719x.ko 2023-08-25T08:31:26.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ssb 2023-08-25T08:31:26.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ssb/ssb.ko 2023-08-25T08:31:26.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target 2023-08-25T08:31:26.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/iscsi 2023-08-25T08:31:26.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-25T08:31:26.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/loopback 2023-08-25T08:31:26.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-25T08:31:26.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_file.ko 2023-08-25T08:31:26.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_iblock.ko 2023-08-25T08:31:26.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_mod.ko 2023-08-25T08:31:26.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_pscsi.ko 2023-08-25T08:31:26.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_user.ko 2023-08-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/tcm_fc 2023-08-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal 2023-08-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel 2023-08-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal 2023-08-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty 2023-08-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/ipwireless 2023-08-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/n_gsm.ko 2023-08-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/n_hdlc.ko 2023-08-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/nozomi.ko 2023-08-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial 2023-08-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/8250 2023-08-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/arc_uart.ko 2023-08-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/jsm 2023-08-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/synclink_gt.ko 2023-08-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs 2023-08-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/core 2023-08-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/host 2023-08-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio 2023-08-25T08:31:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio.ko 2023-08-25T08:31:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_aec.ko 2023-08-25T08:31:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_cif.ko 2023-08-25T08:31:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_pci_generic.ko 2023-08-25T08:31:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_sercos3.ko 2023-08-25T08:31:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb 2023-08-25T08:31:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class 2023-08-25T08:31:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/cdc-acm.ko 2023-08-25T08:31:26.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-25T08:31:26.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/usblp.ko 2023-08-25T08:31:26.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/usbtmc.ko 2023-08-25T08:31:26.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/host 2023-08-25T08:31:26.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-25T08:31:26.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image 2023-08-25T08:31:26.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image/mdc800.ko 2023-08-25T08:31:26.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image/microtek.ko 2023-08-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc 2023-08-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/adutux.ko 2023-08-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/appledisplay.ko 2023-08-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/emi26.ko 2023-08-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/emi62.ko 2023-08-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/ezusb.ko 2023-08-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/idmouse.ko 2023-08-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/iowarrior.ko 2023-08-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/ldusb.ko 2023-08-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/legousbtower.ko 2023-08-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/sisusbvga 2023-08-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usb3503.ko 2023-08-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usblcd.ko 2023-08-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/uss720.ko 2023-08-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/yurex.ko 2023-08-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/phy 2023-08-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/phy/phy-generic.ko 2023-08-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial 2023-08-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/aircable.ko 2023-08-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ark3116.ko 2023-08-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ch341.ko 2023-08-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cp210x.ko 2023-08-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cyberjack.ko 2023-08-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/empeg.ko 2023-08-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/io_ti.ko 2023-08-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ipaq.ko 2023-08-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ipw.ko 2023-08-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ir-usb.ko 2023-08-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/keyspan.ko 2023-08-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mct_u232.ko 2023-08-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mos7720.ko 2023-08-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mos7840.ko 2023-08-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/navman.ko 2023-08-25T08:31:26.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/omninet.ko 2023-08-25T08:31:26.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/opticon.ko 2023-08-25T08:31:26.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/option.ko 2023-08-25T08:31:26.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/oti6858.ko 2023-08-25T08:31:26.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/pl2303.ko 2023-08-25T08:31:26.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/qcaux.ko 2023-08-25T08:31:26.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/qcserial.ko 2023-08-25T08:31:26.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/quatech2.ko 2023-08-25T08:31:26.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/safe_serial.ko 2023-08-25T08:31:26.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/sierra.ko 2023-08-25T08:31:26.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-25T08:31:26.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ssu100.ko 2023-08-25T08:31:26.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/symbolserial.ko 2023-08-25T08:31:26.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-25T08:31:26.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-25T08:31:26.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb_debug.ko 2023-08-25T08:31:26.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-25T08:31:26.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/visor.ko 2023-08-25T08:31:26.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/whiteheat.ko 2023-08-25T08:31:26.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-25T08:31:26.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage 2023-08-25T08:31:26.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/uas.ko 2023-08-25T08:31:26.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-25T08:31:26.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-25T08:31:26.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-25T08:31:26.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-25T08:31:26.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-25T08:31:26.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-25T08:31:26.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-25T08:31:26.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-karma.ko 2023-08-25T08:31:26.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-25T08:31:26.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-25T08:31:26.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-25T08:31:26.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-25T08:31:26.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-25T08:31:26.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/usb-storage.ko 2023-08-25T08:31:26.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip 2023-08-25T08:31:26.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-25T08:31:26.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-25T08:31:26.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-25T08:31:26.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio 2023-08-25T08:31:26.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci 2023-08-25T08:31:26.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-25T08:31:26.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-25T08:31:26.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/vfio.ko 2023-08-25T08:31:26.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-25T08:31:26.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost 2023-08-25T08:31:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost.ko 2023-08-25T08:31:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-25T08:31:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_net.ko 2023-08-25T08:31:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_scsi.ko 2023-08-25T08:31:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video 2023-08-25T08:31:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight 2023-08-25T08:31:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/apple_bl.ko 2023-08-25T08:31:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/lcd.ko 2023-08-25T08:31:26.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-25T08:31:26.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio 2023-08-25T08:31:26.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio.ko 2023-08-25T08:31:26.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_balloon.ko 2023-08-25T08:31:26.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_mmio.ko 2023-08-25T08:31:26.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci.ko 2023-08-25T08:31:26.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-25T08:31:26.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-25T08:31:26.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_ring.ko 2023-08-25T08:31:26.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1 2023-08-25T08:31:26.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters 2023-08-25T08:31:26.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters/ds2482.ko 2023-08-25T08:31:26.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters/ds2490.ko 2023-08-25T08:31:26.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves 2023-08-25T08:31:26.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-25T08:31:26.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-25T08:31:26.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-25T08:31:26.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-25T08:31:26.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-25T08:31:26.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-25T08:31:26.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-25T08:31:26.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-25T08:31:26.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-25T08:31:26.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-25T08:31:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-25T08:31:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/wire.ko 2023-08-25T08:31:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog 2023-08-25T08:31:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-25T08:31:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-25T08:31:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-25T08:31:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/hpwdt.ko 2023-08-25T08:31:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/i6300esb.ko 2023-08-25T08:31:26.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-25T08:31:26.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-25T08:31:26.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ib700wdt.ko 2023-08-25T08:31:26.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ibmasr.ko 2023-08-25T08:31:26.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-25T08:31:26.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-25T08:31:26.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/it87_wdt.ko 2023-08-25T08:31:26.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/machzwd.ko 2023-08-25T08:31:26.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/nv_tco.ko 2023-08-25T08:31:26.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-25T08:31:26.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-25T08:31:26.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-25T08:31:26.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-25T08:31:26.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/softdog.ko 2023-08-25T08:31:26.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-25T08:31:26.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/via_wdt.ko 2023-08-25T08:31:26.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-25T08:31:26.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-25T08:31:26.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-25T08:31:26.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/wdt_pci.ko 2023-08-25T08:31:26.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/xen_wdt.ko 2023-08-25T08:31:26.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen 2023-08-25T08:31:26.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-25T08:31:26.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-evtchn.ko 2023-08-25T08:31:26.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-gntalloc.ko 2023-08-25T08:31:26.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-gntdev.ko 2023-08-25T08:31:26.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-pciback 2023-08-25T08:31:26.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-25T08:31:26.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-privcmd.ko 2023-08-25T08:31:26.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xenfs 2023-08-25T08:31:26.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-25T08:31:26.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs 2023-08-25T08:31:26.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/9p 2023-08-25T08:31:26.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/9p/9p.ko 2023-08-25T08:31:26.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/affs 2023-08-25T08:31:26.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/affs/affs.ko 2023-08-25T08:31:26.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/befs 2023-08-25T08:31:26.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/befs/befs.ko 2023-08-25T08:31:26.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/binfmt_misc.ko 2023-08-25T08:31:26.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/btrfs 2023-08-25T08:31:26.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/btrfs/btrfs.ko 2023-08-25T08:31:26.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cachefiles 2023-08-25T08:31:26.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cachefiles/cachefiles.ko 2023-08-25T08:31:26.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ceph 2023-08-25T08:31:26.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ceph/ceph.ko 2023-08-25T08:31:26.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/coda 2023-08-25T08:31:26.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/coda/coda.ko 2023-08-25T08:31:26.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cramfs 2023-08-25T08:31:26.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cramfs/cramfs.ko 2023-08-25T08:31:26.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/dlm 2023-08-25T08:31:26.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/dlm/dlm.ko 2023-08-25T08:31:26.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ecryptfs 2023-08-25T08:31:26.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-25T08:31:26.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/f2fs 2023-08-25T08:31:26.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/f2fs/f2fs.ko 2023-08-25T08:31:26.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat 2023-08-25T08:31:26.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/fat.ko 2023-08-25T08:31:26.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/msdos.ko 2023-08-25T08:31:26.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/vfat.ko 2023-08-25T08:31:26.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fscache 2023-08-25T08:31:26.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fscache/fscache.ko 2023-08-25T08:31:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse 2023-08-25T08:31:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse/cuse.ko 2023-08-25T08:31:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse/fuse.ko 2023-08-25T08:31:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/gfs2 2023-08-25T08:31:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/gfs2/gfs2.ko 2023-08-25T08:31:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfs 2023-08-25T08:31:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfs/hfs.ko 2023-08-25T08:31:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfsplus 2023-08-25T08:31:26.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfsplus/hfsplus.ko 2023-08-25T08:31:26.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/isofs 2023-08-25T08:31:26.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/isofs/isofs.ko 2023-08-25T08:31:26.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/jfs 2023-08-25T08:31:26.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/jfs/jfs.ko 2023-08-25T08:31:26.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/lockd 2023-08-25T08:31:26.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/lockd/lockd.ko 2023-08-25T08:31:26.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/minix 2023-08-25T08:31:26.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/minix/minix.ko 2023-08-25T08:31:26.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/netfs 2023-08-25T08:31:26.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/netfs/netfs.ko 2023-08-25T08:31:26.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs 2023-08-25T08:31:26.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/blocklayout 2023-08-25T08:31:26.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-25T08:31:26.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/filelayout 2023-08-25T08:31:26.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-25T08:31:26.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/flexfilelayout 2023-08-25T08:31:26.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-25T08:31:26.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfs.ko 2023-08-25T08:31:26.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfsv3.ko 2023-08-25T08:31:26.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfsv4.ko 2023-08-25T08:31:26.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common 2023-08-25T08:31:26.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common/grace.ko 2023-08-25T08:31:26.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common/nfs_acl.ko 2023-08-25T08:31:26.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfsd 2023-08-25T08:31:26.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfsd/nfsd.ko 2023-08-25T08:31:26.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nilfs2 2023-08-25T08:31:26.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nilfs2/nilfs2.ko 2023-08-25T08:31:26.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls 2023-08-25T08:31:26.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-celtic.ko 2023-08-25T08:31:26.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-centeuro.ko 2023-08-25T08:31:26.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-croatian.ko 2023-08-25T08:31:26.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-cyrillic.ko 2023-08-25T08:31:26.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-gaelic.ko 2023-08-25T08:31:26.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-greek.ko 2023-08-25T08:31:26.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-iceland.ko 2023-08-25T08:31:26.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-inuit.ko 2023-08-25T08:31:26.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-roman.ko 2023-08-25T08:31:26.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-romanian.ko 2023-08-25T08:31:26.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-turkish.ko 2023-08-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1250.ko 2023-08-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1251.ko 2023-08-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1255.ko 2023-08-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp737.ko 2023-08-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp775.ko 2023-08-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp850.ko 2023-08-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp852.ko 2023-08-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp855.ko 2023-08-25T08:31:26.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp857.ko 2023-08-25T08:31:26.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp860.ko 2023-08-25T08:31:26.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp861.ko 2023-08-25T08:31:26.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp862.ko 2023-08-25T08:31:26.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp863.ko 2023-08-25T08:31:26.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp864.ko 2023-08-25T08:31:26.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp865.ko 2023-08-25T08:31:26.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp866.ko 2023-08-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp869.ko 2023-08-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp874.ko 2023-08-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp932.ko 2023-08-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp936.ko 2023-08-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp949.ko 2023-08-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp950.ko 2023-08-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_euc-jp.ko 2023-08-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-1.ko 2023-08-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-13.ko 2023-08-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-14.ko 2023-08-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-15.ko 2023-08-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-2.ko 2023-08-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-3.ko 2023-08-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-4.ko 2023-08-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-5.ko 2023-08-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-6.ko 2023-08-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-7.ko 2023-08-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-9.ko 2023-08-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-r.ko 2023-08-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-ru.ko 2023-08-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-u.ko 2023-08-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_utf8.ko 2023-08-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2 2023-08-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/cluster 2023-08-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlm 2023-08-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlmfs 2023-08-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2.ko 2023-08-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/overlayfs 2023-08-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/overlayfs/overlay.ko 2023-08-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/pstore 2023-08-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/pstore/ramoops.ko 2023-08-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/reiserfs 2023-08-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/reiserfs/reiserfs.ko 2023-08-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/romfs 2023-08-25T08:31:26.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/romfs/romfs.ko 2023-08-25T08:31:26.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb 2023-08-25T08:31:26.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/client 2023-08-25T08:31:26.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/client/cifs.ko 2023-08-25T08:31:26.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common 2023-08-25T08:31:26.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common/cifs_arc4.ko 2023-08-25T08:31:26.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common/cifs_md4.ko 2023-08-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/squashfs 2023-08-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/squashfs/squashfs.ko 2023-08-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/sysv 2023-08-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/sysv/sysv.ko 2023-08-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/udf 2023-08-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/udf/udf.ko 2023-08-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ufs 2023-08-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ufs/ufs.ko 2023-08-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/xfs 2023-08-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/xfs/xfs.ko 2023-08-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel 2023-08-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/rcu 2023-08-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/rcu/rcutorture.ko 2023-08-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/torture.ko 2023-08-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/trace 2023-08-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib 2023-08-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/asn1_encoder.ko 2023-08-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crc-itu-t.ko 2023-08-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crc8.ko 2023-08-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto 2023-08-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto/libarc4.ko 2023-08-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto/libdes.ko 2023-08-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/libcrc32c.ko 2023-08-25T08:31:26.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lru_cache.ko 2023-08-25T08:31:26.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4 2023-08-25T08:31:26.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4/lz4_compress.ko 2023-08-25T08:31:26.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4/lz4hc_compress.ko 2023-08-25T08:31:26.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/math 2023-08-25T08:31:26.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/math/cordic.ko 2023-08-25T08:31:26.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/raid6 2023-08-25T08:31:26.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/raid6/raid6_pq.ko 2023-08-25T08:31:26.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/reed_solomon 2023-08-25T08:31:26.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-25T08:31:26.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_bm.ko 2023-08-25T08:31:26.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_fsm.ko 2023-08-25T08:31:26.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_kmp.ko 2023-08-25T08:31:26.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/zstd 2023-08-25T08:31:26.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/zstd/zstd_compress.ko 2023-08-25T08:31:26.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/mm 2023-08-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/mm/hwpoison-inject.ko 2023-08-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net 2023-08-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan 2023-08-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/6lowpan.ko 2023-08-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_dest.ko 2023-08-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_fragment.ko 2023-08-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_hop.ko 2023-08-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_mobility.ko 2023-08-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_routing.ko 2023-08-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_udp.ko 2023-08-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802 2023-08-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/garp.ko 2023-08-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/mrp.ko 2023-08-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/p8022.ko 2023-08-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/psnap.ko 2023-08-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/stp.ko 2023-08-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/8021q 2023-08-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/8021q/8021q.ko 2023-08-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p 2023-08-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet.ko 2023-08-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_fd.ko 2023-08-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_rdma.ko 2023-08-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_virtio.ko 2023-08-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/batman-adv 2023-08-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/batman-adv/batman-adv.ko 2023-08-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge 2023-08-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/br_netfilter.ko 2023-08-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/bridge.ko 2023-08-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter 2023-08-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-25T08:31:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-25T08:31:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-25T08:31:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-25T08:31:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-25T08:31:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-25T08:31:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-25T08:31:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-25T08:31:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-25T08:31:26.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-25T08:31:26.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-25T08:31:26.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-25T08:31:26.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-25T08:31:26.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-25T08:31:26.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-25T08:31:26.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-25T08:31:26.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-25T08:31:26.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-25T08:31:26.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-25T08:31:26.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtables.ko 2023-08-25T08:31:26.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-25T08:31:26.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-25T08:31:26.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-25T08:31:26.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ceph 2023-08-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ceph/libceph.ko 2023-08-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core 2023-08-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core/failover.ko 2023-08-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core/pktgen.ko 2023-08-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp 2023-08-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp.ko 2023-08-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_diag.ko 2023-08-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_ipv4.ko 2023-08-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_ipv6.ko 2023-08-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dns_resolver 2023-08-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dns_resolver/dns_resolver.ko 2023-08-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa 2023-08-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/dsa_core.ko 2023-08-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_brcm.ko 2023-08-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_dsa.ko 2023-08-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_none.ko 2023-08-25T08:31:26.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_trailer.ko 2023-08-25T08:31:26.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154 2023-08-25T08:31:26.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/6lowpan 2023-08-25T08:31:26.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-25T08:31:26.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/ieee802154.ko 2023-08-25T08:31:26.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-25T08:31:26.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4 2023-08-25T08:31:26.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ah4.ko 2023-08-25T08:31:26.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/esp4.ko 2023-08-25T08:31:26.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/fou.ko 2023-08-25T08:31:26.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/gre.ko 2023-08-25T08:31:26.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/inet_diag.ko 2023-08-25T08:31:26.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_gre.ko 2023-08-25T08:31:26.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_tunnel.ko 2023-08-25T08:31:26.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_vti.ko 2023-08-25T08:31:26.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ipcomp.ko 2023-08-25T08:31:26.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ipip.ko 2023-08-25T08:31:26.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter 2023-08-25T08:31:26.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-25T08:31:26.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-25T08:31:26.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-25T08:31:26.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-25T08:31:26.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-25T08:31:26.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-25T08:31:26.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-25T08:31:26.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-25T08:31:26.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-25T08:31:26.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-25T08:31:26.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-25T08:31:26.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-25T08:31:26.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-25T08:31:26.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-25T08:31:26.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-25T08:31:26.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-25T08:31:26.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-25T08:31:26.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-25T08:31:26.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-25T08:31:26.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-25T08:31:26.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-25T08:31:26.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_bic.ko 2023-08-25T08:31:26.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_dctcp.ko 2023-08-25T08:31:26.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_diag.ko 2023-08-25T08:31:26.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_highspeed.ko 2023-08-25T08:31:26.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_htcp.ko 2023-08-25T08:31:26.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_hybla.ko 2023-08-25T08:31:26.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_illinois.ko 2023-08-25T08:31:26.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_lp.ko 2023-08-25T08:31:26.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_scalable.ko 2023-08-25T08:31:26.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_vegas.ko 2023-08-25T08:31:26.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_veno.ko 2023-08-25T08:31:26.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_westwood.ko 2023-08-25T08:31:26.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_yeah.ko 2023-08-25T08:31:26.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tunnel4.ko 2023-08-25T08:31:26.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/udp_diag.ko 2023-08-25T08:31:26.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/udp_tunnel.ko 2023-08-25T08:31:26.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-25T08:31:26.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6 2023-08-25T08:31:26.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ah6.ko 2023-08-25T08:31:26.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/esp6.ko 2023-08-25T08:31:26.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/fou6.ko 2023-08-25T08:31:26.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_tunnel.ko 2023-08-25T08:31:26.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-25T08:31:26.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_vti.ko 2023-08-25T08:31:26.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ipcomp6.ko 2023-08-25T08:31:26.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter 2023-08-25T08:31:26.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-25T08:31:26.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-25T08:31:26.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-25T08:31:26.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-25T08:31:26.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-25T08:31:26.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-25T08:31:26.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-25T08:31:26.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-25T08:31:26.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-25T08:31:26.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-25T08:31:26.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-25T08:31:26.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-25T08:31:26.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-25T08:31:26.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-25T08:31:26.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-25T08:31:26.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-25T08:31:26.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-25T08:31:26.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-25T08:31:26.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-25T08:31:26.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-25T08:31:26.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-25T08:31:26.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-25T08:31:26.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-25T08:31:26.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-25T08:31:26.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/sit.ko 2023-08-25T08:31:26.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/tunnel6.ko 2023-08-25T08:31:26.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-25T08:31:26.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/key 2023-08-25T08:31:26.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/key/af_key.ko 2023-08-25T08:31:26.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp 2023-08-25T08:31:26.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_core.ko 2023-08-25T08:31:26.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-25T08:31:26.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_eth.ko 2023-08-25T08:31:26.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ip.ko 2023-08-25T08:31:26.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ip6.ko 2023-08-25T08:31:26.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_netlink.ko 2023-08-25T08:31:26.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ppp.ko 2023-08-25T08:31:26.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/llc 2023-08-25T08:31:26.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/llc/llc.ko 2023-08-25T08:31:26.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mac802154 2023-08-25T08:31:26.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mac802154/mac802154.ko 2023-08-25T08:31:26.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mpls 2023-08-25T08:31:26.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mpls/mpls_gso.ko 2023-08-25T08:31:26.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter 2023-08-25T08:31:26.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset 2023-08-25T08:31:26.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set.ko 2023-08-25T08:31:26.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-25T08:31:26.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-25T08:31:26.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-25T08:31:26.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-25T08:31:26.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-25T08:31:26.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-25T08:31:26.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-25T08:31:26.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-25T08:31:26.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-25T08:31:26.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-25T08:31:26.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-25T08:31:26.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-25T08:31:26.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-25T08:31:26.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-25T08:31:26.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-25T08:31:26.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs 2023-08-25T08:31:26.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-25T08:31:26.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-25T08:31:26.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-25T08:31:26.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-25T08:31:26.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-25T08:31:26.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-25T08:31:26.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-25T08:31:26.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-25T08:31:26.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-25T08:31:26.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-25T08:31:26.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-25T08:31:26.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-25T08:31:26.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-25T08:31:26.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-25T08:31:26.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conncount.ko 2023-08-25T08:31:26.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack.ko 2023-08-25T08:31:26.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-25T08:31:26.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-25T08:31:26.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-25T08:31:26.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-25T08:31:26.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-25T08:31:26.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-25T08:31:26.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-25T08:31:26.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-25T08:31:26.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-25T08:31:26.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-25T08:31:26.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-25T08:31:26.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-25T08:31:26.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-25T08:31:26.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_flow_table.ko 2023-08-25T08:31:26.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-25T08:31:26.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_log_syslog.ko 2023-08-25T08:31:26.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat.ko 2023-08-25T08:31:26.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-25T08:31:26.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-25T08:31:26.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_irc.ko 2023-08-25T08:31:26.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_sip.ko 2023-08-25T08:31:26.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-25T08:31:26.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-25T08:31:26.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_tables.ko 2023-08-25T08:31:26.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink.ko 2023-08-25T08:31:26.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-25T08:31:26.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_log.ko 2023-08-25T08:31:26.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-25T08:31:26.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-25T08:31:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_chain_nat.ko 2023-08-25T08:31:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_compat.ko 2023-08-25T08:31:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_ct.ko 2023-08-25T08:31:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-25T08:31:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_fib.ko 2023-08-25T08:31:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-25T08:31:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_hash.ko 2023-08-25T08:31:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_limit.ko 2023-08-25T08:31:26.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_log.ko 2023-08-25T08:31:26.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_masq.ko 2023-08-25T08:31:26.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_nat.ko 2023-08-25T08:31:26.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_numgen.ko 2023-08-25T08:31:26.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_queue.ko 2023-08-25T08:31:26.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_quota.ko 2023-08-25T08:31:26.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_redir.ko 2023-08-25T08:31:26.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_reject.ko 2023-08-25T08:31:26.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_reject_inet.ko 2023-08-25T08:31:26.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_socket.ko 2023-08-25T08:31:26.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_tunnel.ko 2023-08-25T08:31:26.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_AUDIT.ko 2023-08-25T08:31:26.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-25T08:31:26.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-25T08:31:26.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-25T08:31:26.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CT.ko 2023-08-25T08:31:26.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_DSCP.ko 2023-08-25T08:31:26.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_HL.ko 2023-08-25T08:31:26.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_HMARK.ko 2023-08-25T08:31:26.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-25T08:31:26.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_LED.ko 2023-08-25T08:31:26.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_LOG.ko 2023-08-25T08:31:26.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-25T08:31:26.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NETMAP.ko 2023-08-25T08:31:26.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NFLOG.ko 2023-08-25T08:31:26.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-25T08:31:26.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_RATEEST.ko 2023-08-25T08:31:26.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-25T08:31:26.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_SECMARK.ko 2023-08-25T08:31:26.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-25T08:31:26.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-25T08:31:26.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TEE.ko 2023-08-25T08:31:26.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TPROXY.ko 2023-08-25T08:31:26.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TRACE.ko 2023-08-25T08:31:26.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_addrtype.ko 2023-08-25T08:31:26.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_bpf.ko 2023-08-25T08:31:26.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cgroup.ko 2023-08-25T08:31:26.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cluster.ko 2023-08-25T08:31:26.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_comment.ko 2023-08-25T08:31:26.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connbytes.ko 2023-08-25T08:31:26.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connlabel.ko 2023-08-25T08:31:26.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connlimit.ko 2023-08-25T08:31:26.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connmark.ko 2023-08-25T08:31:26.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_conntrack.ko 2023-08-25T08:31:26.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cpu.ko 2023-08-25T08:31:26.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_dccp.ko 2023-08-25T08:31:26.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_devgroup.ko 2023-08-25T08:31:26.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_dscp.ko 2023-08-25T08:31:26.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ecn.ko 2023-08-25T08:31:26.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_esp.ko 2023-08-25T08:31:26.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_hashlimit.ko 2023-08-25T08:31:26.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_helper.ko 2023-08-25T08:31:26.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_hl.ko 2023-08-25T08:31:26.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ipcomp.ko 2023-08-25T08:31:26.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_iprange.ko 2023-08-25T08:31:26.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ipvs.ko 2023-08-25T08:31:26.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_l2tp.ko 2023-08-25T08:31:26.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_length.ko 2023-08-25T08:31:26.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_limit.ko 2023-08-25T08:31:26.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_mac.ko 2023-08-25T08:31:26.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_mark.ko 2023-08-25T08:31:26.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_multiport.ko 2023-08-25T08:31:26.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_nat.ko 2023-08-25T08:31:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_nfacct.ko 2023-08-25T08:31:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_osf.ko 2023-08-25T08:31:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_owner.ko 2023-08-25T08:31:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_physdev.ko 2023-08-25T08:31:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_pkttype.ko 2023-08-25T08:31:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_policy.ko 2023-08-25T08:31:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_quota.ko 2023-08-25T08:31:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_rateest.ko 2023-08-25T08:31:26.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_realm.ko 2023-08-25T08:31:26.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_recent.ko 2023-08-25T08:31:26.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_sctp.ko 2023-08-25T08:31:26.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_set.ko 2023-08-25T08:31:26.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_socket.ko 2023-08-25T08:31:26.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_state.ko 2023-08-25T08:31:26.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_statistic.ko 2023-08-25T08:31:26.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_string.ko 2023-08-25T08:31:26.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_tcpmss.ko 2023-08-25T08:31:26.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_time.ko 2023-08-25T08:31:26.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_u32.ko 2023-08-25T08:31:26.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netlink 2023-08-25T08:31:26.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netlink/netlink_diag.ko 2023-08-25T08:31:26.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/nsh 2023-08-25T08:31:26.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/nsh/nsh.ko 2023-08-25T08:31:26.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch 2023-08-25T08:31:26.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/openvswitch.ko 2023-08-25T08:31:26.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-geneve.ko 2023-08-25T08:31:26.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-gre.ko 2023-08-25T08:31:26.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-vxlan.ko 2023-08-25T08:31:26.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/packet 2023-08-25T08:31:26.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/packet/af_packet_diag.ko 2023-08-25T08:31:26.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds 2023-08-25T08:31:26.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds.ko 2023-08-25T08:31:26.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds_rdma.ko 2023-08-25T08:31:26.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds_tcp.ko 2023-08-25T08:31:26.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched 2023-08-25T08:31:26.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_csum.ko 2023-08-25T08:31:26.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_gact.ko 2023-08-25T08:31:26.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_ipt.ko 2023-08-25T08:31:26.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_mirred.ko 2023-08-25T08:31:26.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_nat.ko 2023-08-25T08:31:26.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_pedit.ko 2023-08-25T08:31:26.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_police.ko 2023-08-25T08:31:26.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_simple.ko 2023-08-25T08:31:26.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_skbedit.ko 2023-08-25T08:31:26.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_vlan.ko 2023-08-25T08:31:26.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_basic.ko 2023-08-25T08:31:26.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_bpf.ko 2023-08-25T08:31:26.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_flow.ko 2023-08-25T08:31:26.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_fw.ko 2023-08-25T08:31:26.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_route.ko 2023-08-25T08:31:26.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_u32.ko 2023-08-25T08:31:26.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_cmp.ko 2023-08-25T08:31:26.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_ipset.ko 2023-08-25T08:31:26.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_meta.ko 2023-08-25T08:31:26.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_nbyte.ko 2023-08-25T08:31:26.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_text.ko 2023-08-25T08:31:26.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_u32.ko 2023-08-25T08:31:26.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_choke.ko 2023-08-25T08:31:26.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_codel.ko 2023-08-25T08:31:26.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_drr.ko 2023-08-25T08:31:26.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_fq.ko 2023-08-25T08:31:26.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_gred.ko 2023-08-25T08:31:26.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_hfsc.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_hhf.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_htb.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_ingress.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_mqprio.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_mqprio_lib.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_multiq.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_netem.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_pie.ko 2023-08-25T08:31:26.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_plug.ko 2023-08-25T08:31:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_prio.ko 2023-08-25T08:31:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_qfq.ko 2023-08-25T08:31:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_red.ko 2023-08-25T08:31:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_sfb.ko 2023-08-25T08:31:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_sfq.ko 2023-08-25T08:31:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_tbf.ko 2023-08-25T08:31:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_teql.ko 2023-08-25T08:31:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp 2023-08-25T08:31:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp/sctp.ko 2023-08-25T08:31:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp/sctp_diag.ko 2023-08-25T08:31:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc 2023-08-25T08:31:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss 2023-08-25T08:31:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-25T08:31:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-25T08:31:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/sunrpc.ko 2023-08-25T08:31:26.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/xprtrdma 2023-08-25T08:31:26.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-25T08:31:26.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc 2023-08-25T08:31:26.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc/diag.ko 2023-08-25T08:31:26.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc/tipc.ko 2023-08-25T08:31:26.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/unix 2023-08-25T08:31:26.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/unix/unix_diag.ko 2023-08-25T08:31:26.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock 2023-08-25T08:31:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-25T08:31:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-25T08:31:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock.ko 2023-08-25T08:31:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-25T08:31:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-25T08:31:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/xfrm 2023-08-25T08:31:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-25T08:31:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security 2023-08-25T08:31:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys 2023-08-25T08:31:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/encrypted-keys 2023-08-25T08:31:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-25T08:31:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/trusted-keys 2023-08-25T08:31:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/trusted-keys/trusted.ko 2023-08-25T08:31:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt 2023-08-25T08:31:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt/lib 2023-08-25T08:31:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt/lib/irqbypass.ko 2023-08-25T08:31:26.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.alias 2023-08-25T08:31:26.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.alias.bin 2023-08-25T08:31:26.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin 2023-08-25T08:31:26.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.bin 2023-08-25T08:31:26.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.modinfo 2023-08-25T08:31:26.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.dep 2023-08-25T08:31:26.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.dep.bin 2023-08-25T08:31:26.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.devname 2023-08-25T08:31:26.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.order 2023-08-25T08:31:26.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.softdep 2023-08-25T08:31:26.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.symbols 2023-08-25T08:31:26.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.symbols.bin 2023-08-25T08:31:26.787 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.787 DEBUG:teuthology.orchestra.run.smithi099:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-25T08:31:26.813 INFO:teuthology.orchestra.run.smithi099.stdout:postinstall scriptlet (using /bin/sh): 2023-08-25T08:31:26.813 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-ga6a696707610 -a -r /boot/System.map-6.5.0-rc7-ga6a696707610 ]; then 2023-08-25T08:31:26.814 INFO:teuthology.orchestra.run.smithi099.stdout:cp /boot/vmlinuz-6.5.0-rc7-ga6a696707610 /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T08:31:26.814 INFO:teuthology.orchestra.run.smithi099.stdout:cp /boot/System.map-6.5.0-rc7-ga6a696707610 /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T08:31:26.814 INFO:teuthology.orchestra.run.smithi099.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-ga6a696707610 /boot/System.map-6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.814 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/installkernel 6.5.0-rc7-ga6a696707610 /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T08:31:26.814 INFO:teuthology.orchestra.run.smithi099.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T08:31:26.814 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2023-08-25T08:31:26.814 INFO:teuthology.orchestra.run.smithi099.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-25T08:31:26.815 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-25T08:31:26.815 INFO:teuthology.orchestra.run.smithi099.stdout:new-kernel-pkg --remove 6.5.0-rc7-ga6a696707610 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-ga6a696707610.img 2023-08-25T08:31:26.815 INFO:teuthology.orchestra.run.smithi099.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-25T08:31:26.815 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-install remove 6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.815 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2023-08-25T08:31:26.815 INFO:teuthology.orchestra.run.smithi099.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-25T08:31:26.815 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-25T08:31:26.815 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.816 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2023-08-25T08:31:26.816 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2023-08-25T08:31:26.893 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2023-08-25T08:31:26.893 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2023-08-25T08:31:26.893 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.02 2023-08-25T08:31:26.893 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 123.el8 2023-08-25T08:31:26.893 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2023-08-25T08:31:26.893 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-08-25T08:31:26.894 INFO:teuthology.orchestra.run.smithi099.stdout:Group : System Environment/Base 2023-08-25T08:31:26.894 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 9265450 2023-08-25T08:31:26.894 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2023-08-25T08:31:26.894 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-08-25T08:31:26.894 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-08-25T08:31:26.894 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-08-25T08:31:26.894 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-08-25T08:31:26.894 INFO:teuthology.orchestra.run.smithi099.stdout:Relocations : (not relocatable) 2023-08-25T08:31:26.895 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : Red Hat, Inc. 2023-08-25T08:31:26.895 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : Red Hat, Inc. 2023-08-25T08:31:26.895 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-25T08:31:26.895 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2023-08-25T08:31:26.895 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2023-08-25T08:31:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:31:26.895 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-25T08:31:26.896 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-25T08:31:26.896 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-25T08:31:26.896 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2023-08-25T08:31:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:31:26.896 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2023-08-25T08:31:26.897 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-25T08:31:26.897 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 6.5.0-rc7-ga6a696707610 2023-08-25T08:31:26.897 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-25T08:31:30.815 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2023-08-25T08:31:33.060 INFO:teuthology.orchestra.run.smithi099.stderr:done 2023-08-25T08:31:33.062 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-08-25T08:31:33.120 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.EqR3i5SQdD 2023-08-25T08:31:33.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EqR3i5SQdD 2023-08-25T08:31:33.193 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.EqR3i5SQdD 2023-08-25T08:31:33.338 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.EqR3i5SQdD is 6KB 2023-08-25T08:31:33.386 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.EqR3i5SQdD 2023-08-25T08:31:33.402 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2023-08-25T08:31:33.465 INFO:teuthology.orchestra.run.smithi099.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-08-25T08:31:33.465 INFO:teuthology.orchestra.run.smithi099.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-08-25T08:31:33.466 INFO:teuthology.orchestra.run.smithi099.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-ga6a696707610.conf 2023-08-25T08:31:33.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-ga6a696707610 2023-08-25T08:31:33.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2023-08-25T08:32:03.625 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-08-25T08:32:03.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-08-25T08:32:03.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:32:22.128 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-08-25T08:32:31.129 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-08-25T08:32:31.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:32:34.192 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-08-25T08:32:46.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-08-25T08:32:46.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:33:18.071 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-08-25T08:33:18.437 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-08-25T08:33:18.437 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-ga6a696707610"... 2023-08-25T08:33:18.438 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-08-25T08:33:18.479 INFO:teuthology.orchestra.run.smithi099.stdout:6.5.0-rc7-ga6a696707610 2023-08-25T08:33:18.479 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-ga6a696707610 vs 6.5.0-rc7-ga6a696707610 2023-08-25T08:33:18.479 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-25T08:33:18.479 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-08-25T08:33:19.480 DEBUG:teuthology.parallel:result is None 2023-08-25T08:33:19.481 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-25T08:33:19.491 INFO:teuthology.task.internal:Creating test directory... 2023-08-25T08:33:19.492 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T08:33:19.497 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T08:33:19.500 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T08:33:19.535 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-25T08:33:19.544 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-25T08:33:19.555 INFO:teuthology.task.internal:Creating archive directory... 2023-08-25T08:33:19.556 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T08:33:19.579 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T08:33:19.582 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T08:33:19.634 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-25T08:33:19.643 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-25T08:33:19.643 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T08:33:19.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T08:33:19.663 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T08:33:19.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T08:33:19.682 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T08:33:19.686 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T08:33:19.706 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T08:33:19.758 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T08:33:19.778 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T08:33:19.790 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T08:33:19.813 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T08:33:19.824 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T08:33:19.855 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T08:33:19.859 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-25T08:33:19.871 INFO:teuthology.task.internal:Configuring sudo... 2023-08-25T08:33:19.871 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T08:33:19.876 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T08:33:19.879 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T08:33:19.941 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-25T08:33:19.964 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-25T08:33:19.965 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T08:33:19.969 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T08:33:19.971 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T08:33:20.007 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T08:33:20.118 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T08:33:20.187 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T08:33:20.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T08:33:20.339 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:33:20.340 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T08:33:20.408 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T08:33:20.455 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T08:33:20.521 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T08:33:20.565 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T08:33:20.610 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:33:20.611 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T08:33:20.679 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T08:33:20.724 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T08:33:20.779 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T08:33:20.864 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T08:33:20.912 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:33:20.912 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T08:33:20.982 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2023-08-25T08:33:20.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2023-08-25T08:33:20.989 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2023-08-25T08:33:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T08:33:21.094 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T08:33:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T08:33:21.575 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-25T08:33:21.584 INFO:teuthology.task.internal:Starting timer... 2023-08-25T08:33:21.584 INFO:teuthology.run_tasks:Running task pcp... 2023-08-25T08:33:21.623 INFO:teuthology.run_tasks:Running task selinux... 2023-08-25T08:33:21.668 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-08-25T08:33:21.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2023-08-25T08:33:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2023-08-25T08:33:21.789 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2023-08-25T08:33:21.855 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2023-08-25T08:33:21.856 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2023-08-25T08:33:21.915 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2023-08-25T08:33:21.917 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-25T08:33:21.917 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2023-08-25T08:33:21.965 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2023-08-25T08:33:21.965 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2023-08-25T08:33:21.986 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2023-08-25T08:33:21.987 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2023-08-25T08:33:22.017 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2023-08-25T08:33:22.018 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2023-08-25T08:33:22.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T08:33:22.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T08:33:22.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T08:33:22.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T08:33:22.115 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T08:33:22.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T08:33:22.143 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-25T08:33:22.143 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2023-08-25T08:33:22.211 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2023-08-25T08:33:22.259 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2023-08-25T08:33:22.285 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-25T08:33:22.295 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-08-25T08:33:22.296 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-25T08:33:22.316 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-25T08:33:22.318 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi157.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-25T08:46:02.038 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2023-08-25T08:46:02.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-08-25T08:46:02.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:46:02.210 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-08-25T08:46:02.322 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-08-25T08:46:02.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-08-25T08:46:02.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:46:02.403 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-08-25T08:46:02.467 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-08-25T08:46:02.468 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-08-25T08:46:02.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T08:46:02.540 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-08-25T08:46:02.613 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-08-25T08:46:02.614 INFO:teuthology.run_tasks:Running task clock... 2023-08-25T08:46:02.630 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-25T08:46:02.631 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T08:46:02.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T08:46:02.641 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T08:46:02.641 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T08:46:02.645 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T08:46:02.646 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T08:46:02.678 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T08:46:02.699 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T08:46:02.699 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T08:46:02.709 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T08:46:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T08:46:02.742 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T08:46:02.763 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2023-08-25T08:46:02.774 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2023-08-25T08:46:02.785 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2023-08-25T08:46:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T08:46:02.798 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2023-08-25T08:46:02.804 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T08:46:02.814 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T08:46:02.829 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T08:46:02.835 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2023-08-25T08:46:02.864 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2023-08-25T08:46:02.870 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-08-25T08:46:02.874 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T08:46:02.874 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-08-25T08:46:02.874 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:02.874 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:02.875 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:02.875 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:02.898 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T08:46:02.910 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-08-25T08:46:02.913 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T08:46:02.914 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-08-25T08:46:02.914 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:02.914 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:02.914 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:02.914 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:02.935 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T08:46:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-08-25T08:46:03.026 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T08:46:03.026 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-08-25T08:46:03.027 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:03.027 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:03.027 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:03.027 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T08:46:03.027 INFO:teuthology.run_tasks:Running task pexec... 2023-08-25T08:46:03.054 INFO:teuthology.task.pexec:Executing custom commands... 2023-08-25T08:46:03.056 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T08:46:03.056 DEBUG:teuthology.orchestra.run.smithi099:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-25T08:46:03.057 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi157.front.sepia.ceph.com 2023-08-25T08:46:03.057 DEBUG:teuthology.orchestra.run.smithi157:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-25T08:46:03.059 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi181.front.sepia.ceph.com 2023-08-25T08:46:03.060 DEBUG:teuthology.orchestra.run.smithi181:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-25T08:46:03.434 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:03.437 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:03.540 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:05:01 ago on Fri 25 Aug 2023 08:41:02 AM UTC. 2023-08-25T08:46:04.271 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:05:08 ago on Fri 25 Aug 2023 08:40:55 AM UTC. 2023-08-25T08:46:04.488 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:54 ago on Fri 25 Aug 2023 08:44:09 AM UTC. 2023-08-25T08:46:06.000 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-08-25T08:46:06.001 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-08-25T08:46:06.001 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2023-08-25T08:46:06.001 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-08-25T08:46:06.001 INFO:teuthology.orchestra.run.smithi181.stdout:Resetting modules: 2023-08-25T08:46:06.002 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools 2023-08-25T08:46:06.002 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:06.002 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-08-25T08:46:06.002 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-08-25T08:46:06.002 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:06.104 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-08-25T08:46:06.105 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-08-25T08:46:06.105 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2023-08-25T08:46:06.105 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-08-25T08:46:06.105 INFO:teuthology.orchestra.run.smithi157.stdout:Resetting modules: 2023-08-25T08:46:06.105 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools 2023-08-25T08:46:06.105 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:06.106 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-08-25T08:46:06.106 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-08-25T08:46:06.106 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:06.279 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-08-25T08:46:06.280 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-08-25T08:46:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2023-08-25T08:46:06.281 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-08-25T08:46:06.281 INFO:teuthology.orchestra.run.smithi099.stdout:Resetting modules: 2023-08-25T08:46:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools 2023-08-25T08:46:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:06.282 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-08-25T08:46:06.282 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-08-25T08:46:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:07.001 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:05:11 ago on Fri 25 Aug 2023 08:40:55 AM UTC. 2023-08-25T08:46:07.067 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:05:04 ago on Fri 25 Aug 2023 08:41:02 AM UTC. 2023-08-25T08:46:07.502 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:57 ago on Fri 25 Aug 2023 08:44:09 AM UTC. 2023-08-25T08:46:09.542 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-08-25T08:46:09.900 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-08-25T08:46:10.014 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:10.294 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-08-25T08:46:10.382 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:10.833 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:05:15 ago on Fri 25 Aug 2023 08:40:55 AM UTC. 2023-08-25T08:46:10.884 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:11.257 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:05:08 ago on Fri 25 Aug 2023 08:41:02 AM UTC. 2023-08-25T08:46:11.950 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:02 ago on Fri 25 Aug 2023 08:44:09 AM UTC. 2023-08-25T08:46:12.697 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-08-25T08:46:12.786 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-08-25T08:46:12.787 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-08-25T08:46:12.787 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-08-25T08:46:12.787 INFO:teuthology.orchestra.run.smithi181.stdout:Installing group/module packages: 2023-08-25T08:46:12.787 INFO:teuthology.orchestra.run.smithi181.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-25T08:46:12.787 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-25T08:46:12.787 INFO:teuthology.orchestra.run.smithi181.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-25T08:46:12.787 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-25T08:46:12.788 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-25T08:46:12.788 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-25T08:46:12.788 INFO:teuthology.orchestra.run.smithi181.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-25T08:46:12.788 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-08-25T08:46:12.788 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-25T08:46:12.788 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-25T08:46:12.788 INFO:teuthology.orchestra.run.smithi181.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-25T08:46:12.789 INFO:teuthology.orchestra.run.smithi181.stdout:Installing module profiles: 2023-08-25T08:46:12.789 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools/common 2023-08-25T08:46:12.789 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:12.789 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2023-08-25T08:46:12.789 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools rhel8 2023-08-25T08:46:12.789 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:12.789 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-08-25T08:46:12.789 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-08-25T08:46:12.790 INFO:teuthology.orchestra.run.smithi181.stdout:Install 10 Packages 2023-08-25T08:46:12.790 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:12.790 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 18 M 2023-08-25T08:46:12.791 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 65 M 2023-08-25T08:46:12.791 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-08-25T08:46:12.956 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-08-25T08:46:12.981 INFO:teuthology.orchestra.run.smithi181.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 214 kB/s | 41 kB 00:00 2023-08-25T08:46:12.997 INFO:teuthology.orchestra.run.smithi181.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 452 kB/s | 94 kB 00:00 2023-08-25T08:46:13.014 INFO:teuthology.orchestra.run.smithi181.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 110 kB/s | 25 kB 00:00 2023-08-25T08:46:13.051 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================================ 2023-08-25T08:46:13.051 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-08-25T08:46:13.052 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================================ 2023-08-25T08:46:13.052 INFO:teuthology.orchestra.run.smithi157.stdout:Installing group/module packages: 2023-08-25T08:46:13.052 INFO:teuthology.orchestra.run.smithi157.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-25T08:46:13.052 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-25T08:46:13.052 INFO:teuthology.orchestra.run.smithi157.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-25T08:46:13.052 INFO:teuthology.orchestra.run.smithi157.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-25T08:46:13.053 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-25T08:46:13.053 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-25T08:46:13.053 INFO:teuthology.orchestra.run.smithi157.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-25T08:46:13.053 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-08-25T08:46:13.053 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-25T08:46:13.053 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-25T08:46:13.053 INFO:teuthology.orchestra.run.smithi157.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-25T08:46:13.054 INFO:teuthology.orchestra.run.smithi157.stdout:Installing module profiles: 2023-08-25T08:46:13.054 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools/common 2023-08-25T08:46:13.054 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:13.054 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2023-08-25T08:46:13.054 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools rhel8 2023-08-25T08:46:13.054 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:13.054 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-08-25T08:46:13.054 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================================ 2023-08-25T08:46:13.055 INFO:teuthology.orchestra.run.smithi157.stdout:Install 10 Packages 2023-08-25T08:46:13.055 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:13.055 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 18 M 2023-08-25T08:46:13.056 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 65 M 2023-08-25T08:46:13.056 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-08-25T08:46:13.132 INFO:teuthology.orchestra.run.smithi181.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 328 kB/s | 49 kB 00:00 2023-08-25T08:46:13.164 INFO:teuthology.orchestra.run.smithi181.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.1 MB/s | 524 kB 00:00 2023-08-25T08:46:13.242 INFO:teuthology.orchestra.run.smithi157.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 217 kB/s | 41 kB 00:00 2023-08-25T08:46:13.248 INFO:teuthology.orchestra.run.smithi181.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.2 MB/s | 149 kB 00:00 2023-08-25T08:46:13.259 INFO:teuthology.orchestra.run.smithi157.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 121 kB/s | 25 kB 00:00 2023-08-25T08:46:13.281 INFO:teuthology.orchestra.run.smithi181.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.8 MB/s | 213 kB 00:00 2023-08-25T08:46:13.292 INFO:teuthology.orchestra.run.smithi157.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 394 kB/s | 94 kB 00:00 2023-08-25T08:46:13.351 INFO:teuthology.orchestra.run.smithi157.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 454 kB/s | 49 kB 00:00 2023-08-25T08:46:13.385 INFO:teuthology.orchestra.run.smithi157.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.1 MB/s | 524 kB 00:00 2023-08-25T08:46:13.433 INFO:teuthology.orchestra.run.smithi181.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-08-25T08:46:13.476 INFO:teuthology.orchestra.run.smithi157.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.2 MB/s | 149 kB 00:00 2023-08-25T08:46:13.500 INFO:teuthology.orchestra.run.smithi181.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 10 MB/s | 2.2 MB 00:00 2023-08-25T08:46:13.502 INFO:teuthology.orchestra.run.smithi157.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.8 MB/s | 213 kB 00:00 2023-08-25T08:46:13.600 INFO:teuthology.orchestra.run.smithi181.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-08-25T08:46:13.600 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-08-25T08:46:13.600 INFO:teuthology.orchestra.run.smithi181.stdout:Total 22 MB/s | 18 MB 00:00 2023-08-25T08:46:13.686 INFO:teuthology.orchestra.run.smithi157.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 20 MB/s | 8.0 MB 00:00 2023-08-25T08:46:13.722 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-08-25T08:46:13.744 INFO:teuthology.orchestra.run.smithi157.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.1 MB/s | 2.2 MB 00:00 2023-08-25T08:46:13.799 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-08-25T08:46:13.819 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-08-25T08:46:13.820 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-08-25T08:46:13.825 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-08-25T08:46:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-08-25T08:46:13.826 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-08-25T08:46:13.826 INFO:teuthology.orchestra.run.smithi099.stdout:Installing group/module packages: 2023-08-25T08:46:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-25T08:46:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-25T08:46:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-25T08:46:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-25T08:46:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-25T08:46:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-25T08:46:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-25T08:46:13.828 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-08-25T08:46:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-25T08:46:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-25T08:46:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-25T08:46:13.829 INFO:teuthology.orchestra.run.smithi099.stdout:Installing module profiles: 2023-08-25T08:46:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools/common 2023-08-25T08:46:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:13.830 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2023-08-25T08:46:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools rhel8 2023-08-25T08:46:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:13.830 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-08-25T08:46:13.831 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-08-25T08:46:13.831 INFO:teuthology.orchestra.run.smithi099.stdout:Install 10 Packages 2023-08-25T08:46:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:13.831 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 18 M 2023-08-25T08:46:13.831 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 65 M 2023-08-25T08:46:13.831 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-08-25T08:46:13.835 INFO:teuthology.orchestra.run.smithi157.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-08-25T08:46:13.836 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-08-25T08:46:13.837 INFO:teuthology.orchestra.run.smithi157.stdout:Total 23 MB/s | 18 MB 00:00 2023-08-25T08:46:14.001 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-08-25T08:46:14.006 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-08-25T08:46:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 218 kB/s | 41 kB 00:00 2023-08-25T08:46:14.035 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-08-25T08:46:14.037 INFO:teuthology.orchestra.run.smithi099.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 117 kB/s | 25 kB 00:00 2023-08-25T08:46:14.054 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-08-25T08:46:14.054 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-08-25T08:46:14.062 INFO:teuthology.orchestra.run.smithi099.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 394 kB/s | 94 kB 00:00 2023-08-25T08:46:14.121 INFO:teuthology.orchestra.run.smithi099.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 452 kB/s | 49 kB 00:00 2023-08-25T08:46:14.197 INFO:teuthology.orchestra.run.smithi099.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.2 MB/s | 524 kB 00:00 2023-08-25T08:46:14.235 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-08-25T08:46:14.239 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-08-25T08:46:14.280 INFO:teuthology.orchestra.run.smithi099.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 942 kB/s | 149 kB 00:00 2023-08-25T08:46:14.305 INFO:teuthology.orchestra.run.smithi099.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.9 MB/s | 213 kB 00:00 2023-08-25T08:46:14.455 INFO:teuthology.orchestra.run.smithi099.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 20 MB/s | 8.0 MB 00:00 2023-08-25T08:46:14.536 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-08-25T08:46:14.539 INFO:teuthology.orchestra.run.smithi099.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.4 MB/s | 2.2 MB 00:00 2023-08-25T08:46:14.673 INFO:teuthology.orchestra.run.smithi099.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-08-25T08:46:14.674 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-08-25T08:46:14.674 INFO:teuthology.orchestra.run.smithi099.stdout:Total 21 MB/s | 18 MB 00:00 2023-08-25T08:46:14.725 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-25T08:46:15.300 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T08:46:15.301 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-25T08:46:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-08-25T08:46:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-08-25T08:46:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-08-25T08:46:15.302 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-08-25T08:46:15.303 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-25T08:46:15.303 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T08:46:15.303 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-25T08:46:15.321 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-25T08:46:15.480 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-25T08:46:15.559 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-08-25T08:46:15.563 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-08-25T08:46:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-25T08:46:15.837 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-25T08:46:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-08-25T08:46:16.356 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-25T08:46:16.457 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-25T08:46:16.532 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T08:46:16.590 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-25T08:46:16.783 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-25T08:46:16.995 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-25T08:46:17.347 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-25T08:46:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-25T08:46:17.464 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-25T08:46:17.567 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-25T08:46:17.683 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-25T08:46:17.740 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-25T08:46:17.857 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-25T08:46:17.951 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T08:46:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T08:46:18.204 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-25T08:46:18.220 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T08:46:18.220 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-25T08:46:18.220 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T08:46:18.221 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-25T08:46:18.221 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-25T08:46:18.221 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-25T08:46:18.221 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-25T08:46:18.222 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-25T08:46:18.222 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-25T08:46:18.222 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-25T08:46:18.287 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T08:46:18.288 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-25T08:46:18.288 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T08:46:18.288 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-25T08:46:18.288 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-25T08:46:18.288 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-25T08:46:18.289 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-25T08:46:18.289 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-25T08:46:18.289 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-25T08:46:18.289 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-25T08:46:19.119 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-25T08:46:19.388 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-25T08:46:19.426 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-25T08:46:19.427 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:05:23 ago on Fri 25 Aug 2023 08:40:55 AM UTC. 2023-08-25T08:46:19.471 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-25T08:46:19.471 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:05:16 ago on Fri 25 Aug 2023 08:41:02 AM UTC. 2023-08-25T08:46:19.571 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-25T08:46:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T08:46:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T08:46:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-25T08:46:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T08:46:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-25T08:46:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-25T08:46:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-25T08:46:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-25T08:46:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-25T08:46:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-25T08:46:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-25T08:46:21.596 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-25T08:46:21.597 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:12 ago on Fri 25 Aug 2023 08:44:09 AM UTC. 2023-08-25T08:46:22.599 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2023-08-25T08:46:22.786 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2023-08-25T08:46:22.805 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:22.805 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-08-25T08:46:22.805 INFO:teuthology.orchestra.run.smithi157.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:22.805 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T08:46:22.806 INFO:teuthology.orchestra.run.smithi157.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:22.806 INFO:teuthology.orchestra.run.smithi157.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-25T08:46:22.806 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-25T08:46:22.806 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-25T08:46:22.806 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:22.806 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:22.806 INFO:teuthology.orchestra.run.smithi157.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T08:46:22.807 INFO:teuthology.orchestra.run.smithi157.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-25T08:46:22.807 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:22.807 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-08-25T08:46:22.972 DEBUG:teuthology.parallel:result is None 2023-08-25T08:46:22.988 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:22.988 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-08-25T08:46:22.988 INFO:teuthology.orchestra.run.smithi181.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:22.989 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T08:46:22.989 INFO:teuthology.orchestra.run.smithi181.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:22.989 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-25T08:46:22.990 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-25T08:46:22.990 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-25T08:46:22.992 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:22.992 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:22.993 INFO:teuthology.orchestra.run.smithi181.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T08:46:22.993 INFO:teuthology.orchestra.run.smithi181.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-25T08:46:22.993 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:22.993 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-08-25T08:46:23.138 DEBUG:teuthology.parallel:result is None 2023-08-25T08:46:24.302 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2023-08-25T08:46:24.500 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:24.500 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-08-25T08:46:24.501 INFO:teuthology.orchestra.run.smithi099.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:24.501 INFO:teuthology.orchestra.run.smithi099.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T08:46:24.501 INFO:teuthology.orchestra.run.smithi099.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:24.501 INFO:teuthology.orchestra.run.smithi099.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-25T08:46:24.502 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-25T08:46:24.502 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-25T08:46:24.502 INFO:teuthology.orchestra.run.smithi099.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:24.503 INFO:teuthology.orchestra.run.smithi099.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T08:46:24.503 INFO:teuthology.orchestra.run.smithi099.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T08:46:24.503 INFO:teuthology.orchestra.run.smithi099.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-25T08:46:24.503 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:24.504 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-08-25T08:46:24.712 DEBUG:teuthology.parallel:result is None 2023-08-25T08:46:24.712 INFO:teuthology.run_tasks:Running task install... 2023-08-25T08:46:24.721 DEBUG:teuthology.task.install:project ceph 2023-08-25T08:46:24.722 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}} 2023-08-25T08:46:24.722 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'} 2023-08-25T08:46:24.723 INFO:teuthology.task.install:Using flavor: default 2023-08-25T08:46:24.728 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-25T08:46:24.729 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-25T08:46:24.729 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T08:46:24.730 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:46:24.732 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T08:46:24.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:46:24.733 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T08:46:24.734 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:46:24.879 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/ 2023-08-25T08:46:24.879 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T08:46:24.884 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/ 2023-08-25T08:46:24.884 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T08:46:24.986 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T08:46:24.986 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:46:24.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T08:46:25.044 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/ 2023-08-25T08:46:25.045 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T08:46:25.054 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T08:46:25.054 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T08:46:25.124 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T08:46:25.124 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:46:25.124 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T08:46:25.152 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T08:46:25.152 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T08:46:25.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T08:46:25.220 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T08:46:25.260 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T08:46:25.264 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T08:46:25.264 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:46:25.265 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T08:46:25.292 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T08:46:25.292 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T08:46:25.298 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T08:46:25.328 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2023-08-25T08:46:25.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2023-08-25T08:46:25.362 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T08:46:25.363 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2023-08-25T08:46:25.364 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2023-08-25T08:46:25.441 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T08:46:25.509 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2023-08-25T08:46:25.510 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2023-08-25T08:46:25.756 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:25.773 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:25.900 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:26.069 INFO:teuthology.orchestra.run.smithi181.stdout:44 files removed 2023-08-25T08:46:26.121 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T08:46:26.132 INFO:teuthology.orchestra.run.smithi099.stdout:44 files removed 2023-08-25T08:46:26.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T08:46:26.286 INFO:teuthology.orchestra.run.smithi157.stdout:44 files removed 2023-08-25T08:46:26.340 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T08:46:26.483 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:26.690 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-08-25T08:46:27.365 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 191 kB/s | 81 kB 00:00 2023-08-25T08:46:27.624 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2023-08-25T08:46:27.627 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 166 kB/s | 81 kB 00:00 2023-08-25T08:46:27.719 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 137 kB/s | 81 kB 00:00 2023-08-25T08:46:27.875 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-08-25T08:46:27.894 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2023-08-25T08:46:28.362 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 3.3 kB/s | 1.5 kB 00:00 2023-08-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 6.7 kB/s | 3.5 kB 00:00 2023-08-25T08:46:28.663 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-08-25T08:46:29.462 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 5.5 kB/s | 9.4 kB 00:01 2023-08-25T08:46:29.731 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-08-25T08:46:29.999 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-08-25T08:46:30.030 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-08-25T08:46:30.308 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-08-25T08:46:31.014 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-08-25T08:46:33.415 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 476 kB/s | 24 kB 00:00 2023-08-25T08:46:33.749 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 451 kB/s | 24 kB 00:00 2023-08-25T08:46:34.440 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 MB/s | 47 MB 00:00 2023-08-25T08:46:34.660 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-08-25T08:46:35.264 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 81 kB/s | 24 kB 00:00 2023-08-25T08:46:36.441 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-08-25T08:46:42.482 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 MB/s | 53 MB 00:00 2023-08-25T08:46:42.710 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 MB/s | 53 MB 00:00 2023-08-25T08:46:45.149 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 53 MB 00:01 2023-08-25T08:46:53.580 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T08:46:53.581 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T08:46:53.596 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-25T08:46:53.601 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-25T08:46:53.601 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-25T08:46:53.602 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-25T08:46:53.602 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-25T08:46:53.605 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-25T08:46:53.606 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-25T08:46:53.737 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-08-25T08:46:53.750 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-08-25T08:46:53.750 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-08-25T08:46:53.750 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-08-25T08:46:53.751 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-08-25T08:46:53.751 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-25T08:46:53.751 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-25T08:46:53.751 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.4 k 2023-08-25T08:46:53.751 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 5.4 M 2023-08-25T08:46:53.751 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 862 k 2023-08-25T08:46:53.751 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 167 k 2023-08-25T08:46:53.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 1.6 M 2023-08-25T08:46:53.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 132 k 2023-08-25T08:46:53.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 1.7 M 2023-08-25T08:46:53.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 7.4 M 2023-08-25T08:46:53.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 47 k 2023-08-25T08:46:53.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 14 M 2023-08-25T08:46:53.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 48 M 2023-08-25T08:46:53.753 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 261 k 2023-08-25T08:46:53.753 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 171 k 2023-08-25T08:46:53.753 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 260 k 2023-08-25T08:46:53.753 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 18 k 2023-08-25T08:46:53.753 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-25T08:46:53.753 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-25T08:46:53.753 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-25T08:46:53.754 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 31 k 2023-08-25T08:46:53.754 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 784 k 2023-08-25T08:46:53.754 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T08:46:53.754 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-25T08:46:53.754 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-25T08:46:53.754 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-25T08:46:53.754 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-25T08:46:53.755 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-25T08:46:53.755 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-25T08:46:53.755 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 197 k 2023-08-25T08:46:53.755 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 371 k 2023-08-25T08:46:53.755 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 364 k 2023-08-25T08:46:53.755 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 107 k 2023-08-25T08:46:53.756 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 94 k 2023-08-25T08:46:53.756 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.5 M 2023-08-25T08:46:53.756 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 179 k 2023-08-25T08:46:53.756 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-25T08:46:53.756 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-08-25T08:46:53.757 INFO:teuthology.orchestra.run.smithi181.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-25T08:46:53.757 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.7 M 2023-08-25T08:46:53.757 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.6 M 2023-08-25T08:46:53.757 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-08-25T08:46:53.758 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-25T08:46:53.758 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 22 M 2023-08-25T08:46:53.758 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 23 k 2023-08-25T08:46:53.758 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 2.3 M 2023-08-25T08:46:53.758 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 242 k 2023-08-25T08:46:53.758 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 4.6 M 2023-08-25T08:46:53.758 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 18 M 2023-08-25T08:46:53.759 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 15 k 2023-08-25T08:46:53.759 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 24 k 2023-08-25T08:46:53.759 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-25T08:46:53.759 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-25T08:46:53.760 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-25T08:46:53.760 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-25T08:46:53.760 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-25T08:46:53.760 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T08:46:53.760 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-25T08:46:53.760 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-25T08:46:53.760 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-25T08:46:53.761 INFO:teuthology.orchestra.run.smithi181.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-25T08:46:53.761 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-25T08:46:53.761 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-25T08:46:53.761 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 473 k 2023-08-25T08:46:53.761 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-25T08:46:53.761 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.8 M 2023-08-25T08:46:53.761 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-25T08:46:53.761 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-25T08:46:53.762 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-25T08:46:53.762 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-25T08:46:53.762 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-25T08:46:53.762 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-25T08:46:53.762 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-25T08:46:53.762 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-25T08:46:53.762 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-25T08:46:53.763 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-25T08:46:53.763 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-25T08:46:53.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-25T08:46:53.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-25T08:46:53.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-25T08:46:53.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-25T08:46:53.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-25T08:46:53.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 45 k 2023-08-25T08:46:53.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 126 k 2023-08-25T08:46:53.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-25T08:46:53.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-25T08:46:53.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-25T08:46:53.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-25T08:46:53.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-25T08:46:53.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-25T08:46:53.765 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-25T08:46:53.765 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-25T08:46:53.765 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-25T08:46:53.765 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-25T08:46:53.765 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-25T08:46:53.766 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T08:46:53.766 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-25T08:46:53.766 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-25T08:46:53.766 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-25T08:46:53.766 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-25T08:46:53.766 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-25T08:46:53.766 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-25T08:46:53.767 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-25T08:46:53.767 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-25T08:46:53.767 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-25T08:46:53.767 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-25T08:46:53.767 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-25T08:46:53.767 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-25T08:46:53.768 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-25T08:46:53.768 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-25T08:46:53.768 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-25T08:46:53.768 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-25T08:46:53.768 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-25T08:46:53.769 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-25T08:46:53.769 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T08:46:53.769 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-25T08:46:53.769 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-25T08:46:53.769 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-25T08:46:53.769 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-25T08:46:53.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-25T08:46:53.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-25T08:46:53.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-25T08:46:53.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-25T08:46:53.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-25T08:46:53.770 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-25T08:46:53.771 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-25T08:46:53.771 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-25T08:46:53.771 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-25T08:46:53.771 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-25T08:46:53.771 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-25T08:46:53.771 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-25T08:46:53.771 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-25T08:46:53.771 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-25T08:46:53.772 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2023-08-25T08:46:53.772 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql 10 2023-08-25T08:46:53.772 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:53.772 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-08-25T08:46:53.772 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-08-25T08:46:53.772 INFO:teuthology.orchestra.run.smithi181.stdout:Install 128 Packages 2023-08-25T08:46:53.772 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 3 Packages 2023-08-25T08:46:53.773 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:46:53.773 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 206 M 2023-08-25T08:46:53.773 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-08-25T08:46:54.018 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T08:46:54.018 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T08:46:54.033 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-25T08:46:54.034 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-25T08:46:54.035 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-25T08:46:54.037 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-25T08:46:54.039 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-25T08:46:54.042 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-25T08:46:54.044 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-25T08:46:54.178 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-08-25T08:46:54.195 INFO:teuthology.orchestra.run.smithi157.stdout:=========================================================================================================================== 2023-08-25T08:46:54.195 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-08-25T08:46:54.195 INFO:teuthology.orchestra.run.smithi157.stdout:=========================================================================================================================== 2023-08-25T08:46:54.196 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-08-25T08:46:54.196 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-25T08:46:54.196 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-25T08:46:54.196 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.4 k 2023-08-25T08:46:54.196 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 5.4 M 2023-08-25T08:46:54.197 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 862 k 2023-08-25T08:46:54.197 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 167 k 2023-08-25T08:46:54.197 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 1.6 M 2023-08-25T08:46:54.197 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 132 k 2023-08-25T08:46:54.198 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 1.7 M 2023-08-25T08:46:54.198 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 7.4 M 2023-08-25T08:46:54.199 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 47 k 2023-08-25T08:46:54.199 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 14 M 2023-08-25T08:46:54.199 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 48 M 2023-08-25T08:46:54.199 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 261 k 2023-08-25T08:46:54.200 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 171 k 2023-08-25T08:46:54.200 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 260 k 2023-08-25T08:46:54.200 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 18 k 2023-08-25T08:46:54.201 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-25T08:46:54.201 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-25T08:46:54.201 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-25T08:46:54.201 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 31 k 2023-08-25T08:46:54.201 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 784 k 2023-08-25T08:46:54.201 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T08:46:54.201 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-25T08:46:54.202 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-25T08:46:54.202 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-25T08:46:54.203 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-25T08:46:54.203 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-25T08:46:54.203 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-25T08:46:54.203 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 197 k 2023-08-25T08:46:54.203 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 371 k 2023-08-25T08:46:54.204 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 364 k 2023-08-25T08:46:54.204 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 107 k 2023-08-25T08:46:54.204 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 94 k 2023-08-25T08:46:54.204 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.5 M 2023-08-25T08:46:54.204 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 179 k 2023-08-25T08:46:54.204 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-25T08:46:54.205 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2023-08-25T08:46:54.205 INFO:teuthology.orchestra.run.smithi157.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-25T08:46:54.205 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.7 M 2023-08-25T08:46:54.205 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.6 M 2023-08-25T08:46:54.205 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-08-25T08:46:54.205 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-25T08:46:54.206 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 22 M 2023-08-25T08:46:54.206 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 23 k 2023-08-25T08:46:54.206 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 2.3 M 2023-08-25T08:46:54.206 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 242 k 2023-08-25T08:46:54.206 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 4.6 M 2023-08-25T08:46:54.207 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 18 M 2023-08-25T08:46:54.207 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 15 k 2023-08-25T08:46:54.207 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 24 k 2023-08-25T08:46:54.207 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-25T08:46:54.208 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-25T08:46:54.208 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-25T08:46:54.208 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-25T08:46:54.209 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-25T08:46:54.209 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T08:46:54.209 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-25T08:46:54.209 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-25T08:46:54.210 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-25T08:46:54.210 INFO:teuthology.orchestra.run.smithi157.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-25T08:46:54.210 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-25T08:46:54.210 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-25T08:46:54.210 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 473 k 2023-08-25T08:46:54.210 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-25T08:46:54.210 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.8 M 2023-08-25T08:46:54.211 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-25T08:46:54.211 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-25T08:46:54.211 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-25T08:46:54.211 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-25T08:46:54.211 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-25T08:46:54.211 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-25T08:46:54.211 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-25T08:46:54.212 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-25T08:46:54.212 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-25T08:46:54.212 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-25T08:46:54.212 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-25T08:46:54.212 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-25T08:46:54.213 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-25T08:46:54.213 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-25T08:46:54.213 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-25T08:46:54.213 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-25T08:46:54.213 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 45 k 2023-08-25T08:46:54.213 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 126 k 2023-08-25T08:46:54.213 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-25T08:46:54.213 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-25T08:46:54.214 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-25T08:46:54.214 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-25T08:46:54.214 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-25T08:46:54.214 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-25T08:46:54.214 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-25T08:46:54.215 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-25T08:46:54.215 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-25T08:46:54.215 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-25T08:46:54.215 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-25T08:46:54.215 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T08:46:54.215 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-25T08:46:54.215 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-25T08:46:54.215 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-25T08:46:54.216 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-25T08:46:54.216 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-25T08:46:54.216 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-25T08:46:54.216 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-25T08:46:54.216 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-25T08:46:54.216 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-25T08:46:54.216 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-25T08:46:54.217 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-25T08:46:54.217 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-25T08:46:54.217 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-25T08:46:54.217 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-25T08:46:54.217 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-25T08:46:54.217 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-25T08:46:54.218 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-25T08:46:54.218 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-25T08:46:54.218 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T08:46:54.218 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-25T08:46:54.218 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-25T08:46:54.218 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-25T08:46:54.218 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-25T08:46:54.218 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-25T08:46:54.219 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-25T08:46:54.219 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-25T08:46:54.219 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-25T08:46:54.219 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-25T08:46:54.219 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-25T08:46:54.219 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-25T08:46:54.219 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-25T08:46:54.219 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-25T08:46:54.220 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-25T08:46:54.220 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-25T08:46:54.220 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-25T08:46:54.220 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-25T08:46:54.220 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-25T08:46:54.220 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2023-08-25T08:46:54.220 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql 10 2023-08-25T08:46:54.221 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:54.221 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-08-25T08:46:54.221 INFO:teuthology.orchestra.run.smithi157.stdout:=========================================================================================================================== 2023-08-25T08:46:54.221 INFO:teuthology.orchestra.run.smithi157.stdout:Install 128 Packages 2023-08-25T08:46:54.221 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 3 Packages 2023-08-25T08:46:54.221 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:46:54.222 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 206 M 2023-08-25T08:46:54.223 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-08-25T08:46:54.282 INFO:teuthology.orchestra.run.smithi181.stdout:(1/131): ceph-18.0.0-5703.gba6b39e4.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-08-25T08:46:54.409 INFO:teuthology.orchestra.run.smithi181.stdout:(2/131): ceph-base-18.0.0-5703.gba6b39e4.el8.x8 12 MB/s | 5.4 MB 00:00 2023-08-25T08:46:54.458 INFO:teuthology.orchestra.run.smithi181.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 3.3 MB/s | 167 kB 00:00 2023-08-25T08:46:54.609 INFO:teuthology.orchestra.run.smithi181.stdout:(4/131): ceph-mds-18.0.0-5703.gba6b39e4.el8.x86 15 MB/s | 2.3 MB 00:00 2023-08-25T08:46:54.702 INFO:teuthology.orchestra.run.smithi157.stdout:(1/131): ceph-18.0.0-5703.gba6b39e4.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2023-08-25T08:46:54.736 INFO:teuthology.orchestra.run.smithi181.stdout:(5/131): ceph-mgr-18.0.0-5703.gba6b39e4.el8.x86 13 MB/s | 1.6 MB 00:00 2023-08-25T08:46:54.867 INFO:teuthology.orchestra.run.smithi181.stdout:(6/131): ceph-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.4 MB/s | 862 kB 00:00 2023-08-25T08:46:55.085 INFO:teuthology.orchestra.run.smithi181.stdout:(7/131): ceph-mon-18.0.0-5703.gba6b39e4.el8.x86 13 MB/s | 4.6 MB 00:00 2023-08-25T08:46:56.028 INFO:teuthology.orchestra.run.smithi157.stdout:(2/131): ceph-fuse-18.0.0-5703.gba6b39e4.el8.x8 650 kB/s | 862 kB 00:01 2023-08-25T08:46:56.076 INFO:teuthology.orchestra.run.smithi181.stdout:(8/131): ceph-radosgw-18.0.0-5703.gba6b39e4.el8 14 MB/s | 14 MB 00:00 2023-08-25T08:46:56.127 INFO:teuthology.orchestra.run.smithi181.stdout:(9/131): ceph-selinux-18.0.0-5703.gba6b39e4.el8 489 kB/s | 24 kB 00:00 2023-08-25T08:46:56.203 INFO:teuthology.orchestra.run.smithi157.stdout:(3/131): ceph-base-18.0.0-5703.gba6b39e4.el8.x8 2.9 MB/s | 5.4 MB 00:01 2023-08-25T08:46:56.220 INFO:teuthology.orchestra.run.smithi157.stdout:(4/131): ceph-immutable-object-cache-18.0.0-570 870 kB/s | 167 kB 00:00 2023-08-25T08:46:56.683 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T08:46:56.684 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T08:46:56.699 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-25T08:46:56.700 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-25T08:46:56.701 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-25T08:46:56.702 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-25T08:46:56.705 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-25T08:46:56.708 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-25T08:46:56.709 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-25T08:46:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:(5/131): ceph-mds-18.0.0-5703.gba6b39e4.el8.x86 4.1 MB/s | 2.3 MB 00:00 2023-08-25T08:46:56.847 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-08-25T08:46:56.860 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-08-25T08:46:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-08-25T08:46:56.860 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-08-25T08:46:56.860 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-08-25T08:46:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-25T08:46:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-25T08:46:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.4 k 2023-08-25T08:46:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 5.4 M 2023-08-25T08:46:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 862 k 2023-08-25T08:46:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 167 k 2023-08-25T08:46:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 1.6 M 2023-08-25T08:46:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 132 k 2023-08-25T08:46:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 1.7 M 2023-08-25T08:46:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 7.4 M 2023-08-25T08:46:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 47 k 2023-08-25T08:46:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 14 M 2023-08-25T08:46:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 48 M 2023-08-25T08:46:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 261 k 2023-08-25T08:46:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 171 k 2023-08-25T08:46:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 260 k 2023-08-25T08:46:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 18 k 2023-08-25T08:46:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-25T08:46:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-25T08:46:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-25T08:46:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 31 k 2023-08-25T08:46:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 784 k 2023-08-25T08:46:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T08:46:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-25T08:46:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-25T08:46:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-25T08:46:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-25T08:46:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-25T08:46:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-25T08:46:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 197 k 2023-08-25T08:46:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 371 k 2023-08-25T08:46:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 364 k 2023-08-25T08:46:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 107 k 2023-08-25T08:46:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 94 k 2023-08-25T08:46:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.5 M 2023-08-25T08:46:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 179 k 2023-08-25T08:46:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-25T08:46:56.865 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2023-08-25T08:46:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-25T08:46:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.7 M 2023-08-25T08:46:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.6 M 2023-08-25T08:46:56.866 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-08-25T08:46:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-25T08:46:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 22 M 2023-08-25T08:46:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 23 k 2023-08-25T08:46:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 2.3 M 2023-08-25T08:46:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 242 k 2023-08-25T08:46:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 4.6 M 2023-08-25T08:46:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 18 M 2023-08-25T08:46:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 15 k 2023-08-25T08:46:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 24 k 2023-08-25T08:46:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-25T08:46:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-25T08:46:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-25T08:46:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-25T08:46:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-25T08:46:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T08:46:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-25T08:46:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-25T08:46:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-25T08:46:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-25T08:46:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-25T08:46:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-25T08:46:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 473 k 2023-08-25T08:46:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-25T08:46:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.8 M 2023-08-25T08:46:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-25T08:46:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-25T08:46:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-25T08:46:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-25T08:46:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-25T08:46:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-25T08:46:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-25T08:46:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-25T08:46:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-25T08:46:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-25T08:46:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-25T08:46:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-25T08:46:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-25T08:46:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-25T08:46:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-25T08:46:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-25T08:46:56.872 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 45 k 2023-08-25T08:46:56.872 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 126 k 2023-08-25T08:46:56.872 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-25T08:46:56.872 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-25T08:46:56.872 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-25T08:46:56.872 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-25T08:46:56.872 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-25T08:46:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-25T08:46:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-25T08:46:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-25T08:46:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-25T08:46:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-25T08:46:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-25T08:46:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T08:46:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-25T08:46:56.875 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-25T08:46:56.875 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-25T08:46:56.875 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-25T08:46:56.875 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-25T08:46:56.875 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-25T08:46:56.875 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-25T08:46:56.875 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-25T08:46:56.875 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-25T08:46:56.876 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-25T08:46:56.876 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-25T08:46:56.876 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T08:46:56.876 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-25T08:46:56.876 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-25T08:46:56.876 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-25T08:46:56.876 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2023-08-25T08:46:56.879 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql 10 2023-08-25T08:46:56.879 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:56.879 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-08-25T08:46:56.879 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-08-25T08:46:56.879 INFO:teuthology.orchestra.run.smithi099.stdout:Install 128 Packages 2023-08-25T08:46:56.879 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 3 Packages 2023-08-25T08:46:56.879 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:46:56.880 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 206 M 2023-08-25T08:46:56.880 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-08-25T08:46:57.625 INFO:teuthology.orchestra.run.smithi099.stdout:(1/131): ceph-18.0.0-5703.gba6b39e4.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-08-25T08:46:57.629 INFO:teuthology.orchestra.run.smithi157.stdout:(6/131): ceph-mon-18.0.0-5703.gba6b39e4.el8.x86 5.3 MB/s | 4.6 MB 00:00 2023-08-25T08:46:58.401 INFO:teuthology.orchestra.run.smithi099.stdout:(2/131): ceph-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.1 MB/s | 862 kB 00:00 2023-08-25T08:46:58.575 INFO:teuthology.orchestra.run.smithi099.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 956 kB/s | 167 kB 00:00 2023-08-25T08:46:58.664 INFO:teuthology.orchestra.run.smithi157.stdout:(7/131): ceph-mgr-18.0.0-5703.gba6b39e4.el8.x86 655 kB/s | 1.6 MB 00:02 2023-08-25T08:46:59.551 INFO:teuthology.orchestra.run.smithi099.stdout:(4/131): ceph-mds-18.0.0-5703.gba6b39e4.el8.x86 2.4 MB/s | 2.3 MB 00:00 2023-08-25T08:46:59.825 INFO:teuthology.orchestra.run.smithi099.stdout:(5/131): ceph-mgr-18.0.0-5703.gba6b39e4.el8.x86 5.7 MB/s | 1.6 MB 00:00 2023-08-25T08:46:59.868 INFO:teuthology.orchestra.run.smithi181.stdout:(10/131): ceph-test-18.0.0-5703.gba6b39e4.el8.x 13 MB/s | 48 MB 00:03 2023-08-25T08:47:00.044 INFO:teuthology.orchestra.run.smithi181.stdout:(11/131): ceph-osd-18.0.0-5703.gba6b39e4.el8.x8 3.4 MB/s | 18 MB 00:05 2023-08-25T08:47:00.070 INFO:teuthology.orchestra.run.smithi181.stdout:(12/131): cephfs-mirror-18.0.0-5703.gba6b39e4.e 1.3 MB/s | 260 kB 00:00 2023-08-25T08:47:00.188 INFO:teuthology.orchestra.run.smithi181.stdout:(13/131): libcephfs-devel-18.0.0-5703.gba6b39e4 219 kB/s | 31 kB 00:00 2023-08-25T08:47:00.253 INFO:teuthology.orchestra.run.smithi181.stdout:(14/131): libcephfs2-18.0.0-5703.gba6b39e4.el8. 4.2 MB/s | 784 kB 00:00 2023-08-25T08:47:00.353 INFO:teuthology.orchestra.run.smithi181.stdout:(15/131): librados-devel-18.0.0-5703.gba6b39e4. 1.4 MB/s | 139 kB 00:00 2023-08-25T08:47:00.420 INFO:teuthology.orchestra.run.smithi181.stdout:(16/131): libcephsqlite-18.0.0-5703.gba6b39e4.e 594 kB/s | 139 kB 00:00 2023-08-25T08:47:00.453 INFO:teuthology.orchestra.run.smithi181.stdout:(17/131): libradosstriper1-18.0.0-5703.gba6b39e 4.6 MB/s | 473 kB 00:00 2023-08-25T08:47:00.556 INFO:teuthology.orchestra.run.smithi181.stdout:(18/131): python3-ceph-argparse-18.0.0-5703.gba 447 kB/s | 45 kB 00:00 2023-08-25T08:47:00.596 INFO:teuthology.orchestra.run.smithi181.stdout:(19/131): python3-ceph-common-18.0.0-5703.gba6b 2.9 MB/s | 126 kB 00:00 2023-08-25T08:47:00.609 INFO:teuthology.orchestra.run.smithi099.stdout:(6/131): ceph-mon-18.0.0-5703.gba6b39e4.el8.x86 5.9 MB/s | 4.6 MB 00:00 2023-08-25T08:47:00.654 INFO:teuthology.orchestra.run.smithi181.stdout:(20/131): python3-cephfs-18.0.0-5703.gba6b39e4. 3.3 MB/s | 197 kB 00:00 2023-08-25T08:47:00.738 INFO:teuthology.orchestra.run.smithi181.stdout:(21/131): python3-rados-18.0.0-5703.gba6b39e4.e 4.4 MB/s | 371 kB 00:00 2023-08-25T08:47:00.762 INFO:teuthology.orchestra.run.smithi157.stdout:(8/131): ceph-osd-18.0.0-5703.gba6b39e4.el8.x86 5.7 MB/s | 18 MB 00:03 2023-08-25T08:47:00.821 INFO:teuthology.orchestra.run.smithi181.stdout:(22/131): python3-rbd-18.0.0-5703.gba6b39e4.el8 4.3 MB/s | 364 kB 00:00 2023-08-25T08:47:00.846 INFO:teuthology.orchestra.run.smithi157.stdout:(9/131): ceph-selinux-18.0.0-5703.gba6b39e4.el8 293 kB/s | 24 kB 00:00 2023-08-25T08:47:00.906 INFO:teuthology.orchestra.run.smithi181.stdout:(23/131): python3-rgw-18.0.0-5703.gba6b39e4.el8 1.4 MB/s | 107 kB 00:00 2023-08-25T08:47:00.998 INFO:teuthology.orchestra.run.smithi181.stdout:(24/131): rbd-fuse-18.0.0-5703.gba6b39e4.el8.x8 937 kB/s | 94 kB 00:00 2023-08-25T08:47:01.497 INFO:teuthology.orchestra.run.smithi181.stdout:(25/131): rbd-mirror-18.0.0-5703.gba6b39e4.el8. 7.1 MB/s | 3.5 MB 00:00 2023-08-25T08:47:01.573 INFO:teuthology.orchestra.run.smithi181.stdout:(26/131): rbd-nbd-18.0.0-5703.gba6b39e4.el8.x86 2.3 MB/s | 179 kB 00:00 2023-08-25T08:47:01.623 INFO:teuthology.orchestra.run.smithi181.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5703.g 462 kB/s | 23 kB 00:00 2023-08-25T08:47:01.673 INFO:teuthology.orchestra.run.smithi181.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 2.6 MB/s | 132 kB 00:00 2023-08-25T08:47:01.805 INFO:teuthology.orchestra.run.smithi099.stdout:(7/131): ceph-common-18.0.0-5703.gba6b39e4.el8. 5.0 MB/s | 22 MB 00:04 2023-08-25T08:47:01.874 INFO:teuthology.orchestra.run.smithi181.stdout:(29/131): ceph-mgr-dashboard-18.0.0-5703.gba6b3 8.7 MB/s | 1.7 MB 00:00 2023-08-25T08:47:02.874 INFO:teuthology.orchestra.run.smithi181.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 7.4 MB/s | 7.4 MB 00:00 2023-08-25T08:47:02.957 INFO:teuthology.orchestra.run.smithi181.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5703.gba 2.8 MB/s | 242 kB 00:00 2023-08-25T08:47:03.002 INFO:teuthology.orchestra.run.smithi099.stdout:(8/131): ceph-base-18.0.0-5703.gba6b39e4.el8.x8 975 kB/s | 5.4 MB 00:05 2023-08-25T08:47:03.008 INFO:teuthology.orchestra.run.smithi181.stdout:(32/131): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 949 kB/s | 47 kB 00:00 2023-08-25T08:47:03.066 INFO:teuthology.orchestra.run.smithi181.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5703.gb 249 kB/s | 15 kB 00:00 2023-08-25T08:47:03.084 INFO:teuthology.orchestra.run.smithi099.stdout:(9/131): ceph-selinux-18.0.0-5703.gba6b39e4.el8 292 kB/s | 24 kB 00:00 2023-08-25T08:47:03.145 INFO:teuthology.orchestra.run.smithi181.stdout:(34/131): ceph-volume-18.0.0-5703.gba6b39e4.el8 3.4 MB/s | 261 kB 00:00 2023-08-25T08:47:03.217 INFO:teuthology.orchestra.run.smithi181.stdout:(35/131): cephadm-18.0.0-5703.gba6b39e4.el8.noa 2.2 MB/s | 171 kB 00:00 2023-08-25T08:47:03.311 INFO:teuthology.orchestra.run.smithi181.stdout:(36/131): cephfs-top-18.0.0-5703.gba6b39e4.el8. 196 kB/s | 18 kB 00:00 2023-08-25T08:47:03.631 INFO:teuthology.orchestra.run.smithi181.stdout:(37/131): librgw2-18.0.0-5703.gba6b39e4.el8.x86 2.1 MB/s | 6.8 MB 00:03 2023-08-25T08:47:03.810 INFO:teuthology.orchestra.run.smithi099.stdout:(10/131): ceph-radosgw-18.0.0-5703.gba6b39e4.el 6.8 MB/s | 14 MB 00:02 2023-08-25T08:47:03.894 INFO:teuthology.orchestra.run.smithi181.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 220 kB/s | 59 kB 00:00 2023-08-25T08:47:03.927 INFO:teuthology.orchestra.run.smithi181.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 496 kB/s | 306 kB 00:00 2023-08-25T08:47:04.035 INFO:teuthology.orchestra.run.smithi099.stdout:(11/131): ceph-osd-18.0.0-5703.gba6b39e4.el8.x8 5.2 MB/s | 18 MB 00:03 2023-08-25T08:47:04.060 INFO:teuthology.orchestra.run.smithi099.stdout:(12/131): cephfs-mirror-18.0.0-5703.gba6b39e4.e 1.0 MB/s | 260 kB 00:00 2023-08-25T08:47:04.088 INFO:teuthology.orchestra.run.smithi099.stdout:(13/131): libcephfs-devel-18.0.0-5703.gba6b39e4 625 kB/s | 31 kB 00:00 2023-08-25T08:47:04.136 INFO:teuthology.orchestra.run.smithi099.stdout:(14/131): libcephsqlite-18.0.0-5703.gba6b39e4.e 2.7 MB/s | 139 kB 00:00 2023-08-25T08:47:04.155 INFO:teuthology.orchestra.run.smithi181.stdout:(40/131): ceph-common-18.0.0-5703.gba6b39e4.el8 2.2 MB/s | 22 MB 00:10 2023-08-25T08:47:04.169 INFO:teuthology.orchestra.run.smithi099.stdout:(15/131): libcephfs2-18.0.0-5703.gba6b39e4.el8. 7.0 MB/s | 784 kB 00:00 2023-08-25T08:47:04.203 INFO:teuthology.orchestra.run.smithi099.stdout:(16/131): librados-devel-18.0.0-5703.gba6b39e4. 2.0 MB/s | 139 kB 00:00 2023-08-25T08:47:04.254 INFO:teuthology.orchestra.run.smithi099.stdout:(17/131): libradosstriper1-18.0.0-5703.gba6b39e 5.5 MB/s | 473 kB 00:00 2023-08-25T08:47:04.305 INFO:teuthology.orchestra.run.smithi099.stdout:(18/131): python3-ceph-argparse-18.0.0-5703.gba 895 kB/s | 45 kB 00:00 2023-08-25T08:47:04.371 INFO:teuthology.orchestra.run.smithi099.stdout:(19/131): python3-ceph-common-18.0.0-5703.gba6b 1.9 MB/s | 126 kB 00:00 2023-08-25T08:47:04.429 INFO:teuthology.orchestra.run.smithi099.stdout:(20/131): python3-cephfs-18.0.0-5703.gba6b39e4. 3.3 MB/s | 197 kB 00:00 2023-08-25T08:47:04.460 INFO:teuthology.orchestra.run.smithi181.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 142 kB/s | 44 kB 00:00 2023-08-25T08:47:04.538 INFO:teuthology.orchestra.run.smithi099.stdout:(21/131): python3-rados-18.0.0-5703.gba6b39e4.e 3.4 MB/s | 371 kB 00:00 2023-08-25T08:47:04.606 INFO:teuthology.orchestra.run.smithi099.stdout:(22/131): python3-rbd-18.0.0-5703.gba6b39e4.el8 5.4 MB/s | 364 kB 00:00 2023-08-25T08:47:04.655 INFO:teuthology.orchestra.run.smithi099.stdout:(23/131): python3-rgw-18.0.0-5703.gba6b39e4.el8 2.1 MB/s | 107 kB 00:00 2023-08-25T08:47:04.703 INFO:teuthology.orchestra.run.smithi181.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 613 kB/s | 476 kB 00:00 2023-08-25T08:47:04.714 INFO:teuthology.orchestra.run.smithi099.stdout:(24/131): rbd-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.6 MB/s | 94 kB 00:00 2023-08-25T08:47:04.749 INFO:teuthology.orchestra.run.smithi181.stdout:(43/131): libunwind-1.3.1-3.el8.x86_64.rpm 88 kB/s | 75 kB 00:00 2023-08-25T08:47:04.829 INFO:teuthology.orchestra.run.smithi181.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 504 kB/s | 185 kB 00:00 2023-08-25T08:47:05.054 INFO:teuthology.orchestra.run.smithi181.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 768 kB/s | 173 kB 00:00 2023-08-25T08:47:05.072 INFO:teuthology.orchestra.run.smithi099.stdout:(25/131): librgw2-18.0.0-5703.gba6b39e4.el8.x86 7.8 MB/s | 6.8 MB 00:00 2023-08-25T08:47:05.079 INFO:teuthology.orchestra.run.smithi181.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 49 kB/s | 16 kB 00:00 2023-08-25T08:47:05.148 INFO:teuthology.orchestra.run.smithi099.stdout:(26/131): rbd-mirror-18.0.0-5703.gba6b39e4.el8. 8.2 MB/s | 3.5 MB 00:00 2023-08-25T08:47:05.173 INFO:teuthology.orchestra.run.smithi099.stdout:(27/131): rbd-nbd-18.0.0-5703.gba6b39e4.el8.x86 1.7 MB/s | 179 kB 00:00 2023-08-25T08:47:05.198 INFO:teuthology.orchestra.run.smithi099.stdout:(28/131): ceph-grafana-dashboards-18.0.0-5703.g 462 kB/s | 23 kB 00:00 2023-08-25T08:47:05.223 INFO:teuthology.orchestra.run.smithi099.stdout:(29/131): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 2.6 MB/s | 132 kB 00:00 2023-08-25T08:47:05.238 INFO:teuthology.orchestra.run.smithi181.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 513 kB/s | 81 kB 00:00 2023-08-25T08:47:05.263 INFO:teuthology.orchestra.run.smithi181.stdout:(48/131): python3-cachetools-3.1.1-4.el8.noarch 59 kB/s | 33 kB 00:00 2023-08-25T08:47:05.333 INFO:teuthology.orchestra.run.smithi181.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-08-25T08:47:05.388 INFO:teuthology.orchestra.run.smithi181.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.1 MB/s | 384 kB 00:00 2023-08-25T08:47:05.416 INFO:teuthology.orchestra.run.smithi099.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5703.gba6b3 8.1 MB/s | 1.7 MB 00:00 2023-08-25T08:47:05.422 INFO:teuthology.orchestra.run.smithi181.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 111 kB/s | 18 kB 00:00 2023-08-25T08:47:05.502 INFO:teuthology.orchestra.run.smithi099.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5703.gba 2.7 MB/s | 242 kB 00:00 2023-08-25T08:47:05.556 INFO:teuthology.orchestra.run.smithi181.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 441 kB/s | 59 kB 00:00 2023-08-25T08:47:05.673 INFO:teuthology.orchestra.run.smithi181.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 3.4 MB/s | 1.2 MB 00:00 2023-08-25T08:47:05.810 INFO:teuthology.orchestra.run.smithi099.stdout:(32/131): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 155 kB/s | 47 kB 00:00 2023-08-25T08:47:05.859 INFO:teuthology.orchestra.run.smithi099.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5703.gb 292 kB/s | 15 kB 00:00 2023-08-25T08:47:05.881 INFO:teuthology.orchestra.run.smithi181.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 185 kB/s | 60 kB 00:00 2023-08-25T08:47:05.906 INFO:teuthology.orchestra.run.smithi181.stdout:(55/131): python3-logutils-0.3.5-11.el8.noarch. 94 kB/s | 49 kB 00:00 2023-08-25T08:47:05.948 INFO:teuthology.orchestra.run.smithi181.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2023-08-25T08:47:05.959 INFO:teuthology.orchestra.run.smithi099.stdout:(34/131): ceph-volume-18.0.0-5703.gba6b39e4.el8 2.5 MB/s | 261 kB 00:00 2023-08-25T08:47:05.990 INFO:teuthology.orchestra.run.smithi181.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 402 kB/s | 34 kB 00:00 2023-08-25T08:47:06.027 INFO:teuthology.orchestra.run.smithi099.stdout:(35/131): cephadm-18.0.0-5703.gba6b39e4.el8.noa 2.5 MB/s | 171 kB 00:00 2023-08-25T08:47:06.032 INFO:teuthology.orchestra.run.smithi181.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-08-25T08:47:06.050 INFO:teuthology.orchestra.run.smithi181.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-08-25T08:47:06.076 INFO:teuthology.orchestra.run.smithi099.stdout:(36/131): cephfs-top-18.0.0-5703.gba6b39e4.el8. 363 kB/s | 18 kB 00:00 2023-08-25T08:47:06.099 INFO:teuthology.orchestra.run.smithi181.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-08-25T08:47:06.149 INFO:teuthology.orchestra.run.smithi181.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 159 kB/s | 19 kB 00:00 2023-08-25T08:47:06.175 INFO:teuthology.orchestra.run.smithi181.stdout:(62/131): python3-portend-2.6-1.el8.noarch.rpm 53 kB/s | 16 kB 00:00 2023-08-25T08:47:06.200 INFO:teuthology.orchestra.run.smithi181.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-08-25T08:47:06.276 INFO:teuthology.orchestra.run.smithi099.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 7.0 MB/s | 7.4 MB 00:01 2023-08-25T08:47:06.359 INFO:teuthology.orchestra.run.smithi181.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.4 MB/s | 255 kB 00:00 2023-08-25T08:47:06.426 INFO:teuthology.orchestra.run.smithi099.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 877 kB/s | 306 kB 00:00 2023-08-25T08:47:06.434 INFO:teuthology.orchestra.run.smithi181.stdout:(65/131): python3-trustme-0.6.0-4.el8.noarch.rp 97 kB/s | 27 kB 00:00 2023-08-25T08:47:06.464 INFO:teuthology.orchestra.run.smithi181.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 610 kB/s | 61 kB 00:00 2023-08-25T08:47:06.502 INFO:teuthology.orchestra.run.smithi099.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 264 kB/s | 59 kB 00:00 2023-08-25T08:47:06.527 INFO:teuthology.orchestra.run.smithi099.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 748 kB/s | 75 kB 00:00 2023-08-25T08:47:06.560 INFO:teuthology.orchestra.run.smithi181.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-08-25T08:47:06.594 INFO:teuthology.orchestra.run.smithi099.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 656 kB/s | 44 kB 00:00 2023-08-25T08:47:06.601 INFO:teuthology.orchestra.run.smithi181.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 510 kB/s | 85 kB 00:00 2023-08-25T08:47:06.632 INFO:teuthology.orchestra.run.smithi181.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-08-25T08:47:06.771 INFO:teuthology.orchestra.run.smithi157.stdout:(10/131): ceph-test-18.0.0-5703.gba6b39e4.el8.x 8.2 MB/s | 48 MB 00:05 2023-08-25T08:47:06.774 INFO:teuthology.orchestra.run.smithi099.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-08-25T08:47:06.775 INFO:teuthology.orchestra.run.smithi099.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.3 MB/s | 476 kB 00:00 2023-08-25T08:47:06.775 INFO:teuthology.orchestra.run.smithi099.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 496 kB/s | 33 kB 00:00 2023-08-25T08:47:06.775 INFO:teuthology.orchestra.run.smithi099.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 243 kB/s | 16 kB 00:00 2023-08-25T08:47:06.775 INFO:teuthology.orchestra.run.smithi181.stdout:(70/131): python3-webob-1.8.5-1.el8.1.noarch.rp 556 kB/s | 251 kB 00:00 2023-08-25T08:47:06.776 INFO:teuthology.orchestra.run.smithi181.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-08-25T08:47:06.776 INFO:teuthology.orchestra.run.smithi181.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-08-25T08:47:06.777 INFO:teuthology.orchestra.run.smithi181.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 883 kB/s | 54 kB 00:00 2023-08-25T08:47:06.788 INFO:teuthology.orchestra.run.smithi157.stdout:(11/131): cephfs-mirror-18.0.0-5703.gba6b39e4.e 3.8 MB/s | 260 kB 00:00 2023-08-25T08:47:06.796 INFO:teuthology.orchestra.run.smithi181.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.0 MB/s | 259 kB 00:00 2023-08-25T08:47:06.821 INFO:teuthology.orchestra.run.smithi099.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-08-25T08:47:06.830 INFO:teuthology.orchestra.run.smithi157.stdout:(12/131): libcephfs-devel-18.0.0-5703.gba6b39e4 748 kB/s | 31 kB 00:00 2023-08-25T08:47:06.831 INFO:teuthology.orchestra.run.smithi181.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-08-25T08:47:06.846 INFO:teuthology.orchestra.run.smithi181.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 774 kB/s | 39 kB 00:00 2023-08-25T08:47:06.855 INFO:teuthology.orchestra.run.smithi099.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-08-25T08:47:06.884 INFO:teuthology.orchestra.run.smithi181.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-08-25T08:47:06.889 INFO:teuthology.orchestra.run.smithi099.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-08-25T08:47:06.913 INFO:teuthology.orchestra.run.smithi181.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.8 MB/s | 538 kB 00:00 2023-08-25T08:47:06.914 INFO:teuthology.orchestra.run.smithi099.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-08-25T08:47:06.922 INFO:teuthology.orchestra.run.smithi157.stdout:(13/131): libcephfs2-18.0.0-5703.gba6b39e4.el8. 8.4 MB/s | 784 kB 00:00 2023-08-25T08:47:06.929 INFO:teuthology.orchestra.run.smithi181.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-08-25T08:47:06.948 INFO:teuthology.orchestra.run.smithi099.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 299 kB/s | 18 kB 00:00 2023-08-25T08:47:06.955 INFO:teuthology.orchestra.run.smithi181.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-08-25T08:47:06.970 INFO:teuthology.orchestra.run.smithi157.stdout:(14/131): libcephsqlite-18.0.0-5703.gba6b39e4.e 3.3 MB/s | 139 kB 00:00 2023-08-25T08:47:07.002 INFO:teuthology.orchestra.run.smithi181.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-08-25T08:47:07.014 INFO:teuthology.orchestra.run.smithi157.stdout:(15/131): librados-devel-18.0.0-5703.gba6b39e4. 2.7 MB/s | 139 kB 00:00 2023-08-25T08:47:07.022 INFO:teuthology.orchestra.run.smithi099.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 651 kB/s | 49 kB 00:00 2023-08-25T08:47:07.041 INFO:teuthology.orchestra.run.smithi181.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 636 kB/s | 27 kB 00:00 2023-08-25T08:47:07.056 INFO:teuthology.orchestra.run.smithi099.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2023-08-25T08:47:07.081 INFO:teuthology.orchestra.run.smithi157.stdout:(16/131): libradosstriper1-18.0.0-5703.gba6b39e 6.9 MB/s | 473 kB 00:00 2023-08-25T08:47:07.081 INFO:teuthology.orchestra.run.smithi181.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-08-25T08:47:07.090 INFO:teuthology.orchestra.run.smithi099.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 881 kB/s | 59 kB 00:00 2023-08-25T08:47:07.123 INFO:teuthology.orchestra.run.smithi181.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-08-25T08:47:07.123 INFO:teuthology.orchestra.run.smithi099.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 904 kB/s | 60 kB 00:00 2023-08-25T08:47:07.182 INFO:teuthology.orchestra.run.smithi099.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2023-08-25T08:47:07.208 INFO:teuthology.orchestra.run.smithi099.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2023-08-25T08:47:07.231 INFO:teuthology.orchestra.run.smithi181.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-08-25T08:47:07.250 INFO:teuthology.orchestra.run.smithi099.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 505 kB/s | 34 kB 00:00 2023-08-25T08:47:07.283 INFO:teuthology.orchestra.run.smithi181.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-08-25T08:47:07.286 INFO:teuthology.orchestra.run.smithi099.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.6 MB/s | 196 kB 00:00 2023-08-25T08:47:07.309 INFO:teuthology.orchestra.run.smithi099.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-08-25T08:47:07.323 INFO:teuthology.orchestra.run.smithi181.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-08-25T08:47:07.342 INFO:teuthology.orchestra.run.smithi099.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2023-08-25T08:47:07.367 INFO:teuthology.orchestra.run.smithi099.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-08-25T08:47:07.391 INFO:teuthology.orchestra.run.smithi181.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-08-25T08:47:07.401 INFO:teuthology.orchestra.run.smithi099.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 499 kB/s | 29 kB 00:00 2023-08-25T08:47:07.416 INFO:teuthology.orchestra.run.smithi181.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-08-25T08:47:07.434 INFO:teuthology.orchestra.run.smithi099.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 412 kB/s | 27 kB 00:00 2023-08-25T08:47:07.444 INFO:teuthology.orchestra.run.smithi181.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-08-25T08:47:07.466 INFO:teuthology.orchestra.run.smithi181.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-08-25T08:47:07.491 INFO:teuthology.orchestra.run.smithi181.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-08-25T08:47:07.496 INFO:teuthology.orchestra.run.smithi099.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.6 MB/s | 255 kB 00:00 2023-08-25T08:47:07.525 INFO:teuthology.orchestra.run.smithi181.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-08-25T08:47:07.526 INFO:teuthology.orchestra.run.smithi099.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-08-25T08:47:07.556 INFO:teuthology.orchestra.run.smithi157.stdout:(17/131): librgw2-18.0.0-5703.gba6b39e4.el8.x86 14 MB/s | 6.8 MB 00:00 2023-08-25T08:47:07.562 INFO:teuthology.orchestra.run.smithi099.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 935 kB/s | 61 kB 00:00 2023-08-25T08:47:07.596 INFO:teuthology.orchestra.run.smithi099.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-08-25T08:47:07.598 INFO:teuthology.orchestra.run.smithi157.stdout:(18/131): python3-ceph-argparse-18.0.0-5703.gba 1.1 MB/s | 45 kB 00:00 2023-08-25T08:47:07.621 INFO:teuthology.orchestra.run.smithi099.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 390 kB/s | 23 kB 00:00 2023-08-25T08:47:07.640 INFO:teuthology.orchestra.run.smithi157.stdout:(19/131): python3-ceph-common-18.0.0-5703.gba6b 2.9 MB/s | 126 kB 00:00 2023-08-25T08:47:07.647 INFO:teuthology.orchestra.run.smithi099.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-08-25T08:47:07.660 INFO:teuthology.orchestra.run.smithi181.stdout:(94/131): python3-scipy-1.0.0-21.module+el8.5.0 13 MB/s | 14 MB 00:01 2023-08-25T08:47:07.684 INFO:teuthology.orchestra.run.smithi181.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-08-25T08:47:07.690 INFO:teuthology.orchestra.run.smithi157.stdout:(20/131): python3-cephfs-18.0.0-5703.gba6b39e4. 3.8 MB/s | 197 kB 00:00 2023-08-25T08:47:07.734 INFO:teuthology.orchestra.run.smithi181.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.9 MB/s | 202 kB 00:00 2023-08-25T08:47:07.740 INFO:teuthology.orchestra.run.smithi157.stdout:(21/131): python3-rados-18.0.0-5703.gba6b39e4.e 7.3 MB/s | 371 kB 00:00 2023-08-25T08:47:07.777 INFO:teuthology.orchestra.run.smithi181.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-08-25T08:47:07.780 INFO:teuthology.orchestra.run.smithi099.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 9.4 MB/s | 1.7 MB 00:00 2023-08-25T08:47:07.808 INFO:teuthology.orchestra.run.smithi157.stdout:(22/131): python3-rbd-18.0.0-5703.gba6b39e4.el8 5.3 MB/s | 364 kB 00:00 2023-08-25T08:47:07.839 INFO:teuthology.orchestra.run.smithi099.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-08-25T08:47:07.858 INFO:teuthology.orchestra.run.smithi157.stdout:(23/131): python3-rgw-18.0.0-5703.gba6b39e4.el8 2.1 MB/s | 107 kB 00:00 2023-08-25T08:47:07.872 INFO:teuthology.orchestra.run.smithi181.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 13 MB/s | 4.9 MB 00:00 2023-08-25T08:47:07.872 INFO:teuthology.orchestra.run.smithi099.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-08-25T08:47:07.900 INFO:teuthology.orchestra.run.smithi157.stdout:(24/131): rbd-fuse-18.0.0-5703.gba6b39e4.el8.x8 2.2 MB/s | 94 kB 00:00 2023-08-25T08:47:07.916 INFO:teuthology.orchestra.run.smithi099.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-08-25T08:47:07.956 INFO:teuthology.orchestra.run.smithi181.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-08-25T08:47:07.965 INFO:teuthology.orchestra.run.smithi099.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-08-25T08:47:08.002 INFO:teuthology.orchestra.run.smithi181.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-08-25T08:47:08.023 INFO:teuthology.orchestra.run.smithi099.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 670 kB/s | 39 kB 00:00 2023-08-25T08:47:08.036 INFO:teuthology.orchestra.run.smithi181.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-08-25T08:47:08.074 INFO:teuthology.orchestra.run.smithi099.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-08-25T08:47:08.078 INFO:teuthology.orchestra.run.smithi181.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-08-25T08:47:08.103 INFO:teuthology.orchestra.run.smithi181.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-08-25T08:47:08.125 INFO:teuthology.orchestra.run.smithi099.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-08-25T08:47:08.128 INFO:teuthology.orchestra.run.smithi181.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 852 kB/s | 43 kB 00:00 2023-08-25T08:47:08.153 INFO:teuthology.orchestra.run.smithi181.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-08-25T08:47:08.175 INFO:teuthology.orchestra.run.smithi157.stdout:(25/131): rbd-mirror-18.0.0-5703.gba6b39e4.el8. 13 MB/s | 3.5 MB 00:00 2023-08-25T08:47:08.179 INFO:teuthology.orchestra.run.smithi181.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 954 kB/s | 48 kB 00:00 2023-08-25T08:47:08.213 INFO:teuthology.orchestra.run.smithi181.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2023-08-25T08:47:08.238 INFO:teuthology.orchestra.run.smithi181.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 729 kB/s | 43 kB 00:00 2023-08-25T08:47:08.254 INFO:teuthology.orchestra.run.smithi181.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.6 MB/s | 109 kB 00:00 2023-08-25T08:47:08.274 INFO:teuthology.orchestra.run.smithi099.stdout:(78/131): python3-scipy-1.0.0-21.module+el8.5.0 22 MB/s | 14 MB 00:00 2023-08-25T08:47:08.288 INFO:teuthology.orchestra.run.smithi181.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-08-25T08:47:08.299 INFO:teuthology.orchestra.run.smithi099.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 896 kB/s | 157 kB 00:00 2023-08-25T08:47:08.317 INFO:teuthology.orchestra.run.smithi157.stdout:(26/131): ceph-radosgw-18.0.0-5703.gba6b39e4.el 1.4 MB/s | 14 MB 00:09 2023-08-25T08:47:08.325 INFO:teuthology.orchestra.run.smithi099.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-08-25T08:47:08.330 INFO:teuthology.orchestra.run.smithi181.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-08-25T08:47:08.343 INFO:teuthology.orchestra.run.smithi157.stdout:(27/131): rbd-nbd-18.0.0-5703.gba6b39e4.el8.x86 1.0 MB/s | 179 kB 00:00 2023-08-25T08:47:08.355 INFO:teuthology.orchestra.run.smithi181.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 280 kB/s | 19 kB 00:00 2023-08-25T08:47:08.375 INFO:teuthology.orchestra.run.smithi099.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-08-25T08:47:08.376 INFO:teuthology.orchestra.run.smithi157.stdout:(28/131): ceph-grafana-dashboards-18.0.0-5703.g 397 kB/s | 23 kB 00:00 2023-08-25T08:47:08.401 INFO:teuthology.orchestra.run.smithi157.stdout:(29/131): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 2.2 MB/s | 132 kB 00:00 2023-08-25T08:47:08.409 INFO:teuthology.orchestra.run.smithi099.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 799 kB/s | 27 kB 00:00 2023-08-25T08:47:08.422 INFO:teuthology.orchestra.run.smithi181.stdout:(113/131): postgresql-server-10.21-2.module+el8 7.8 MB/s | 5.1 MB 00:00 2023-08-25T08:47:08.439 INFO:teuthology.orchestra.run.smithi181.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.4 MB/s | 155 kB 00:00 2023-08-25T08:47:08.451 INFO:teuthology.orchestra.run.smithi099.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-08-25T08:47:08.456 INFO:teuthology.orchestra.run.smithi181.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 534 kB/s | 18 kB 00:00 2023-08-25T08:47:08.491 INFO:teuthology.orchestra.run.smithi181.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.1 MB/s | 559 kB 00:00 2023-08-25T08:47:08.515 INFO:teuthology.orchestra.run.smithi181.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.8 MB/s | 605 kB 00:00 2023-08-25T08:47:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 585 kB/s | 44 kB 00:00 2023-08-25T08:47:08.541 INFO:teuthology.orchestra.run.smithi181.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-08-25T08:47:08.567 INFO:teuthology.orchestra.run.smithi181.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-08-25T08:47:08.591 INFO:teuthology.orchestra.run.smithi181.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-08-25T08:47:08.616 INFO:teuthology.orchestra.run.smithi181.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-08-25T08:47:08.626 INFO:teuthology.orchestra.run.smithi099.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-08-25T08:47:08.642 INFO:teuthology.orchestra.run.smithi181.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-08-25T08:47:08.667 INFO:teuthology.orchestra.run.smithi181.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-08-25T08:47:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-08-25T08:47:08.693 INFO:teuthology.orchestra.run.smithi181.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-08-25T08:47:08.728 INFO:teuthology.orchestra.run.smithi099.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-08-25T08:47:08.743 INFO:teuthology.orchestra.run.smithi181.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.4 MB/s | 644 kB 00:00 2023-08-25T08:47:08.767 INFO:teuthology.orchestra.run.smithi181.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-08-25T08:47:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-08-25T08:47:08.793 INFO:teuthology.orchestra.run.smithi181.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-08-25T08:47:08.811 INFO:teuthology.orchestra.run.smithi099.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-08-25T08:47:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-08-25T08:47:08.843 INFO:teuthology.orchestra.run.smithi181.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.6 MB/s | 2.3 MB 00:00 2023-08-25T08:47:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-08-25T08:47:08.887 INFO:teuthology.orchestra.run.smithi099.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-08-25T08:47:08.894 INFO:teuthology.orchestra.run.smithi181.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-08-25T08:47:08.902 INFO:teuthology.orchestra.run.smithi157.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 15 MB/s | 7.4 MB 00:00 2023-08-25T08:47:08.929 INFO:teuthology.orchestra.run.smithi099.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-08-25T08:47:08.952 INFO:teuthology.orchestra.run.smithi157.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5703.gba 4.7 MB/s | 242 kB 00:00 2023-08-25T08:47:08.986 INFO:teuthology.orchestra.run.smithi157.stdout:(32/131): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 1.4 MB/s | 47 kB 00:00 2023-08-25T08:47:09.008 INFO:teuthology.orchestra.run.smithi099.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 2.6 MB/s | 198 kB 00:00 2023-08-25T08:47:09.036 INFO:teuthology.orchestra.run.smithi157.stdout:(33/131): ceph-mgr-dashboard-18.0.0-5703.gba6b3 2.7 MB/s | 1.7 MB 00:00 2023-08-25T08:47:09.053 INFO:teuthology.orchestra.run.smithi157.stdout:(34/131): ceph-prometheus-alerts-18.0.0-5703.gb 216 kB/s | 15 kB 00:00 2023-08-25T08:47:09.120 INFO:teuthology.orchestra.run.smithi157.stdout:(35/131): cephadm-18.0.0-5703.gba6b39e4.el8.noa 2.5 MB/s | 171 kB 00:00 2023-08-25T08:47:09.146 INFO:teuthology.orchestra.run.smithi099.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 19 MB/s | 4.9 MB 00:00 2023-08-25T08:47:09.169 INFO:teuthology.orchestra.run.smithi181.stdout:(130/131): librbd1-18.0.0-5703.gba6b39e4.el8.x8 9.7 MB/s | 3.6 MB 00:00 2023-08-25T08:47:09.170 INFO:teuthology.orchestra.run.smithi157.stdout:(36/131): cephfs-top-18.0.0-5703.gba6b39e4.el8. 359 kB/s | 18 kB 00:00 2023-08-25T08:47:09.188 INFO:teuthology.orchestra.run.smithi099.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-08-25T08:47:09.196 INFO:teuthology.orchestra.run.smithi157.stdout:(37/131): ceph-volume-18.0.0-5703.gba6b39e4.el8 1.6 MB/s | 261 kB 00:00 2023-08-25T08:47:09.261 INFO:teuthology.orchestra.run.smithi099.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-08-25T08:47:09.281 INFO:teuthology.orchestra.run.smithi157.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 707 kB/s | 59 kB 00:00 2023-08-25T08:47:09.290 INFO:teuthology.orchestra.run.smithi099.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.5 MB/s | 354 kB 00:00 2023-08-25T08:47:09.329 INFO:teuthology.orchestra.run.smithi157.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.9 MB/s | 306 kB 00:00 2023-08-25T08:47:09.355 INFO:teuthology.orchestra.run.smithi157.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 999 kB/s | 75 kB 00:00 2023-08-25T08:47:09.356 INFO:teuthology.orchestra.run.smithi099.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2023-08-25T08:47:09.405 INFO:teuthology.orchestra.run.smithi157.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 6.2 MB/s | 476 kB 00:00 2023-08-25T08:47:09.430 INFO:teuthology.orchestra.run.smithi157.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 580 kB/s | 44 kB 00:00 2023-08-25T08:47:09.440 INFO:teuthology.orchestra.run.smithi099.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-08-25T08:47:09.458 INFO:teuthology.orchestra.run.smithi157.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-08-25T08:47:09.481 INFO:teuthology.orchestra.run.smithi157.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2023-08-25T08:47:09.511 INFO:teuthology.orchestra.run.smithi099.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 699 kB/s | 47 kB 00:00 2023-08-25T08:47:09.512 INFO:teuthology.orchestra.run.smithi157.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 324 kB/s | 16 kB 00:00 2023-08-25T08:47:09.557 INFO:teuthology.orchestra.run.smithi157.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-08-25T08:47:09.573 INFO:teuthology.orchestra.run.smithi099.stdout:(102/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-08-25T08:47:09.592 INFO:teuthology.orchestra.run.smithi157.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-08-25T08:47:09.599 INFO:teuthology.orchestra.run.smithi099.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.1 MB/s | 200 kB 00:00 2023-08-25T08:47:09.607 INFO:teuthology.orchestra.run.smithi157.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-08-25T08:47:09.624 INFO:teuthology.orchestra.run.smithi099.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 850 kB/s | 43 kB 00:00 2023-08-25T08:47:09.633 INFO:teuthology.orchestra.run.smithi157.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2023-08-25T08:47:09.649 INFO:teuthology.orchestra.run.smithi099.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-08-25T08:47:09.658 INFO:teuthology.orchestra.run.smithi157.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-08-25T08:47:09.674 INFO:teuthology.orchestra.run.smithi099.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 961 kB/s | 48 kB 00:00 2023-08-25T08:47:09.701 INFO:teuthology.orchestra.run.smithi099.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-08-25T08:47:09.708 INFO:teuthology.orchestra.run.smithi157.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 982 kB/s | 49 kB 00:00 2023-08-25T08:47:09.726 INFO:teuthology.orchestra.run.smithi099.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-08-25T08:47:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:(131/131): librados2-18.0.0-5703.gba6b39e4.el8. 2.9 MB/s | 3.7 MB 00:01 2023-08-25T08:47:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-08-25T08:47:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:Total 13 MB/s | 206 MB 00:16 2023-08-25T08:47:10.199 INFO:teuthology.orchestra.run.smithi157.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 8.7 MB/s | 1.2 MB 00:00 2023-08-25T08:47:10.199 INFO:teuthology.orchestra.run.smithi157.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 702 kB/s | 59 kB 00:00 2023-08-25T08:47:10.199 INFO:teuthology.orchestra.run.smithi157.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-08-25T08:47:10.199 INFO:teuthology.orchestra.run.smithi157.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-08-25T08:47:10.199 INFO:teuthology.orchestra.run.smithi157.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2023-08-25T08:47:10.199 INFO:teuthology.orchestra.run.smithi157.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 671 kB/s | 34 kB 00:00 2023-08-25T08:47:10.199 INFO:teuthology.orchestra.run.smithi157.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 4.6 MB/s | 196 kB 00:00 2023-08-25T08:47:10.200 INFO:teuthology.orchestra.run.smithi157.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2023-08-25T08:47:10.200 INFO:teuthology.orchestra.run.smithi157.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2023-08-25T08:47:10.200 INFO:teuthology.orchestra.run.smithi157.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 475 kB/s | 24 kB 00:00 2023-08-25T08:47:10.200 INFO:teuthology.orchestra.run.smithi157.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2023-08-25T08:47:10.200 INFO:teuthology.orchestra.run.smithi157.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2023-08-25T08:47:10.200 INFO:teuthology.orchestra.run.smithi157.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-08-25T08:47:10.200 INFO:teuthology.orchestra.run.smithi099.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.8 MB/s | 109 kB 00:00 2023-08-25T08:47:10.201 INFO:teuthology.orchestra.run.smithi099.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-08-25T08:47:10.201 INFO:teuthology.orchestra.run.smithi099.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-08-25T08:47:10.201 INFO:teuthology.orchestra.run.smithi099.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 320 kB/s | 19 kB 00:00 2023-08-25T08:47:10.201 INFO:teuthology.orchestra.run.smithi099.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-08-25T08:47:10.201 INFO:teuthology.orchestra.run.smithi099.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-08-25T08:47:10.201 INFO:teuthology.orchestra.run.smithi099.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 307 kB/s | 18 kB 00:00 2023-08-25T08:47:10.201 INFO:teuthology.orchestra.run.smithi099.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-08-25T08:47:10.202 INFO:teuthology.orchestra.run.smithi099.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-08-25T08:47:10.202 INFO:teuthology.orchestra.run.smithi099.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-08-25T08:47:10.202 INFO:teuthology.orchestra.run.smithi099.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-08-25T08:47:10.202 INFO:teuthology.orchestra.run.smithi099.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2023-08-25T08:47:10.202 INFO:teuthology.orchestra.run.smithi099.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 713 kB/s | 60 kB 00:00 2023-08-25T08:47:10.204 INFO:teuthology.orchestra.run.smithi099.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-08-25T08:47:10.232 INFO:teuthology.orchestra.run.smithi099.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-08-25T08:47:10.255 INFO:teuthology.orchestra.run.smithi099.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 768 kB/s | 39 kB 00:00 2023-08-25T08:47:10.279 INFO:teuthology.orchestra.run.smithi157.stdout:(65/131): ceph-common-18.0.0-5703.gba6b39e4.el8 1.4 MB/s | 22 MB 00:15 2023-08-25T08:47:10.289 INFO:teuthology.orchestra.run.smithi099.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-08-25T08:47:10.304 INFO:teuthology.orchestra.run.smithi157.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 253 kB/s | 61 kB 00:00 2023-08-25T08:47:10.316 INFO:teuthology.orchestra.run.smithi099.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-08-25T08:47:10.330 INFO:teuthology.orchestra.run.smithi157.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 859 kB/s | 251 kB 00:00 2023-08-25T08:47:10.339 INFO:teuthology.orchestra.run.smithi099.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-08-25T08:47:10.346 INFO:teuthology.orchestra.run.smithi157.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 550 kB/s | 23 kB 00:00 2023-08-25T08:47:10.399 INFO:teuthology.orchestra.run.smithi157.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-08-25T08:47:10.422 INFO:teuthology.orchestra.run.smithi157.stdout:(70/131): python3-webtest-2.0.33-1.el8.noarch.r 600 kB/s | 85 kB 00:00 2023-08-25T08:47:10.473 INFO:teuthology.orchestra.run.smithi157.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-08-25T08:47:10.497 INFO:teuthology.orchestra.run.smithi157.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-08-25T08:47:10.540 INFO:teuthology.orchestra.run.smithi157.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 811 kB/s | 54 kB 00:00 2023-08-25T08:47:10.565 INFO:teuthology.orchestra.run.smithi157.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.1 MB/s | 259 kB 00:00 2023-08-25T08:47:10.598 INFO:teuthology.orchestra.run.smithi157.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-08-25T08:47:10.624 INFO:teuthology.orchestra.run.smithi157.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 662 kB/s | 39 kB 00:00 2023-08-25T08:47:10.657 INFO:teuthology.orchestra.run.smithi157.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-08-25T08:47:10.691 INFO:teuthology.orchestra.run.smithi157.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-08-25T08:47:10.708 INFO:teuthology.orchestra.run.smithi157.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-08-25T08:47:10.733 INFO:teuthology.orchestra.run.smithi157.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-08-25T08:47:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:(128/131): librbd1-18.0.0-5703.gba6b39e4.el8.x8 8.7 MB/s | 3.6 MB 00:00 2023-08-25T08:47:10.775 INFO:teuthology.orchestra.run.smithi157.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-08-25T08:47:10.815 INFO:teuthology.orchestra.run.smithi099.stdout:(129/131): librados2-18.0.0-5703.gba6b39e4.el8. 7.4 MB/s | 3.7 MB 00:00 2023-08-25T08:47:10.818 INFO:teuthology.orchestra.run.smithi157.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 641 kB/s | 27 kB 00:00 2023-08-25T08:47:10.848 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-08-25T08:47:10.856 INFO:teuthology.orchestra.run.smithi099.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 7.8 MB/s | 798 kB 00:00 2023-08-25T08:47:10.889 INFO:teuthology.orchestra.run.smithi157.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 4.5 MB/s | 320 kB 00:00 2023-08-25T08:47:10.934 INFO:teuthology.orchestra.run.smithi157.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 945 kB/s | 44 kB 00:00 2023-08-25T08:47:10.970 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-08-25T08:47:10.971 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-08-25T08:47:11.009 INFO:teuthology.orchestra.run.smithi157.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-08-25T08:47:11.060 INFO:teuthology.orchestra.run.smithi157.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-08-25T08:47:11.102 INFO:teuthology.orchestra.run.smithi157.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-08-25T08:47:11.161 INFO:teuthology.orchestra.run.smithi157.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-08-25T08:47:11.186 INFO:teuthology.orchestra.run.smithi157.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-08-25T08:47:11.211 INFO:teuthology.orchestra.run.smithi157.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.4 MB/s | 126 kB 00:00 2023-08-25T08:47:11.236 INFO:teuthology.orchestra.run.smithi157.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 395 kB/s | 20 kB 00:00 2023-08-25T08:47:11.263 INFO:teuthology.orchestra.run.smithi157.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-08-25T08:47:11.404 INFO:teuthology.orchestra.run.smithi157.stdout:(93/131): python3-scipy-1.0.0-21.module+el8.5.0 14 MB/s | 14 MB 00:01 2023-08-25T08:47:11.438 INFO:teuthology.orchestra.run.smithi157.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 3.5 MB/s | 709 kB 00:00 2023-08-25T08:47:11.563 INFO:teuthology.orchestra.run.smithi157.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 38 MB/s | 4.7 MB 00:00 2023-08-25T08:47:11.596 INFO:teuthology.orchestra.run.smithi157.stdout:(96/131): libpq-13.5-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2023-08-25T08:47:11.621 INFO:teuthology.orchestra.run.smithi157.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-08-25T08:47:11.655 INFO:teuthology.orchestra.run.smithi157.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-08-25T08:47:11.747 INFO:teuthology.orchestra.run.smithi157.stdout:(99/131): openblas-threads-0.3.15-3.el8.x86_64. 10 MB/s | 4.9 MB 00:00 2023-08-25T08:47:11.780 INFO:teuthology.orchestra.run.smithi157.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-08-25T08:47:11.823 INFO:teuthology.orchestra.run.smithi157.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-08-25T08:47:11.858 INFO:teuthology.orchestra.run.smithi157.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-08-25T08:47:11.881 INFO:teuthology.orchestra.run.smithi157.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2023-08-25T08:47:11.907 INFO:teuthology.orchestra.run.smithi157.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 853 kB/s | 43 kB 00:00 2023-08-25T08:47:11.932 INFO:teuthology.orchestra.run.smithi157.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-08-25T08:47:11.959 INFO:teuthology.orchestra.run.smithi157.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 954 kB/s | 48 kB 00:00 2023-08-25T08:47:11.982 INFO:teuthology.orchestra.run.smithi157.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-08-25T08:47:12.008 INFO:teuthology.orchestra.run.smithi157.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-08-25T08:47:12.034 INFO:teuthology.orchestra.run.smithi157.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-08-25T08:47:12.059 INFO:teuthology.orchestra.run.smithi157.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-08-25T08:47:12.097 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-08-25T08:47:12.100 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-08-25T08:47:12.126 INFO:teuthology.orchestra.run.smithi157.stdout:(111/131): postgresql-server-10.21-2.module+el8 10 MB/s | 5.1 MB 00:00 2023-08-25T08:47:12.144 INFO:teuthology.orchestra.run.smithi157.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 223 kB/s | 19 kB 00:00 2023-08-25T08:47:12.176 INFO:teuthology.orchestra.run.smithi157.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-08-25T08:47:12.201 INFO:teuthology.orchestra.run.smithi157.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-08-25T08:47:12.226 INFO:teuthology.orchestra.run.smithi157.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 356 kB/s | 18 kB 00:00 2023-08-25T08:47:12.260 INFO:teuthology.orchestra.run.smithi157.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.7 MB/s | 559 kB 00:00 2023-08-25T08:47:12.285 INFO:teuthology.orchestra.run.smithi157.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.0 MB/s | 605 kB 00:00 2023-08-25T08:47:12.313 INFO:teuthology.orchestra.run.smithi157.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-08-25T08:47:12.338 INFO:teuthology.orchestra.run.smithi157.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-08-25T08:47:12.370 INFO:teuthology.orchestra.run.smithi157.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-08-25T08:47:12.404 INFO:teuthology.orchestra.run.smithi157.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.0 MB/s | 60 kB 00:00 2023-08-25T08:47:12.421 INFO:teuthology.orchestra.run.smithi157.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-08-25T08:47:12.448 INFO:teuthology.orchestra.run.smithi157.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-08-25T08:47:12.471 INFO:teuthology.orchestra.run.smithi157.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 766 kB/s | 39 kB 00:00 2023-08-25T08:47:12.514 INFO:teuthology.orchestra.run.smithi157.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-08-25T08:47:12.530 INFO:teuthology.orchestra.run.smithi157.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-08-25T08:47:12.556 INFO:teuthology.orchestra.run.smithi157.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-08-25T08:47:12.605 INFO:teuthology.orchestra.run.smithi157.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.7 MB/s | 2.3 MB 00:00 2023-08-25T08:47:12.681 INFO:teuthology.orchestra.run.smithi157.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 10 MB/s | 798 kB 00:00 2023-08-25T08:47:12.940 INFO:teuthology.orchestra.run.smithi157.stdout:(130/131): librbd1-18.0.0-5703.gba6b39e4.el8.x8 9.5 MB/s | 3.6 MB 00:00 2023-08-25T08:47:14.164 INFO:teuthology.orchestra.run.smithi157.stdout:(131/131): librados2-18.0.0-5703.gba6b39e4.el8. 2.3 MB/s | 3.7 MB 00:01 2023-08-25T08:47:14.167 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-08-25T08:47:14.168 INFO:teuthology.orchestra.run.smithi157.stdout:Total 10 MB/s | 206 MB 00:19 2023-08-25T08:47:14.977 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-08-25T08:47:15.097 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-08-25T08:47:15.098 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-08-25T08:47:16.209 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-08-25T08:47:16.214 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-08-25T08:47:17.214 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-08-25T08:47:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-25T08:47:17.557 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-25T08:47:17.768 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-25T08:47:17.986 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T08:47:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T08:47:18.361 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T08:47:18.767 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T08:47:18.994 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:47:19.190 INFO:teuthology.orchestra.run.smithi099.stdout:(131/131): ceph-test-18.0.0-5703.gba6b39e4.el8. 3.0 MB/s | 48 MB 00:16 2023-08-25T08:47:19.192 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-08-25T08:47:19.193 INFO:teuthology.orchestra.run.smithi099.stdout:Total 9.2 MB/s | 206 MB 00:22 2023-08-25T08:47:19.391 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:47:19.612 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:47:19.736 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:47:19.902 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:47:19.998 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:47:20.171 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T08:47:20.822 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-08-25T08:47:20.956 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-08-25T08:47:20.957 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-08-25T08:47:20.977 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-25T08:47:21.189 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-08-25T08:47:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-25T08:47:21.375 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-25T08:47:21.535 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-25T08:47:21.635 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-25T08:47:21.783 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-25T08:47:21.922 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-25T08:47:21.960 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T08:47:22.060 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T08:47:22.073 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T08:47:22.145 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T08:47:22.285 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T08:47:22.310 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T08:47:22.360 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T08:47:22.493 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T08:47:22.497 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-08-25T08:47:22.503 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-08-25T08:47:22.663 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T08:47:22.696 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T08:47:22.835 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T08:47:22.909 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:47:23.310 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:47:23.454 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T08:47:23.536 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:47:23.652 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T08:47:23.670 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:47:23.844 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:47:23.949 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:47:24.126 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T08:47:24.252 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T08:47:24.363 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-25T08:47:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T08:47:24.571 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-25T08:47:24.930 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-25T08:47:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T08:47:25.198 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-25T08:47:25.343 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T08:47:25.433 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-25T08:47:25.450 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T08:47:25.593 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T08:47:25.643 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-25T08:47:25.673 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T08:47:25.817 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T08:47:25.871 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-25T08:47:25.949 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T08:47:26.074 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-25T08:47:26.101 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T08:47:26.290 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-25T08:47:26.542 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-25T08:47:26.699 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 26/134 2023-08-25T08:47:26.716 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T08:47:26.828 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 27/134 2023-08-25T08:47:26.892 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T08:47:27.518 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T08:47:27.701 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T08:47:27.705 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 28/134 2023-08-25T08:47:27.726 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-08-25T08:47:27.923 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-25T08:47:28.007 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-25T08:47:28.109 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-25T08:47:28.253 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-25T08:47:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-25T08:47:28.392 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T08:47:28.492 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-25T08:47:28.532 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T08:47:28.657 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-25T08:47:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T08:47:28.726 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T08:47:28.910 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-25T08:47:28.924 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T08:47:29.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-25T08:47:29.306 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-25T08:47:29.392 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T08:47:29.506 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-25T08:47:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:47:29.752 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-25T08:47:29.923 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 26/134 2023-08-25T08:47:30.061 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 27/134 2023-08-25T08:47:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:47:30.323 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:47:30.473 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:47:30.497 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T08:47:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:47:30.818 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-25T08:47:30.820 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:47:30.956 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 28/134 2023-08-25T08:47:31.003 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-25T08:47:31.006 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T08:47:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-25T08:47:31.273 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-25T08:47:31.330 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-25T08:47:31.372 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 36/134 2023-08-25T08:47:31.536 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-25T08:47:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-25T08:47:31.546 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 37/134 2023-08-25T08:47:31.716 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-25T08:47:31.783 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-25T08:47:31.918 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-25T08:47:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-25T08:47:31.975 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T08:47:32.111 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-25T08:47:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-25T08:47:32.270 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-25T08:47:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T08:47:32.516 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-25T08:47:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T08:47:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T08:47:32.723 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-25T08:47:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T08:47:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T08:47:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T08:47:33.116 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-25T08:47:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T08:47:33.395 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-25T08:47:33.585 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-25T08:47:33.753 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T08:47:33.812 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-25T08:47:34.008 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T08:47:34.060 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-25T08:47:34.068 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-25T08:47:34.173 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T08:47:34.262 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-25T08:47:34.416 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-25T08:47:34.461 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-25T08:47:34.621 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-25T08:47:34.657 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 36/134 2023-08-25T08:47:34.813 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-25T08:47:34.836 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 37/134 2023-08-25T08:47:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T08:47:34.998 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-25T08:47:35.071 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-25T08:47:35.089 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T08:47:35.209 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-25T08:47:35.236 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-25T08:47:35.394 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-25T08:47:35.402 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-25T08:47:35.552 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-25T08:47:35.688 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-25T08:47:35.815 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-25T08:47:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T08:47:35.967 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-25T08:47:36.015 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-25T08:47:36.180 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-25T08:47:36.204 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-25T08:47:36.398 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-25T08:47:36.482 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-25T08:47:36.648 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-25T08:47:36.678 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-25T08:47:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-25T08:47:36.801 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-25T08:47:36.868 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-25T08:47:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-25T08:47:36.983 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T08:47:37.035 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T08:47:37.062 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-25T08:47:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-25T08:47:37.271 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-25T08:47:37.277 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-25T08:47:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-25T08:47:37.494 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-25T08:47:37.583 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-25T08:47:37.652 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T08:47:37.660 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-25T08:47:37.730 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T08:47:37.792 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-25T08:47:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 26/134 2023-08-25T08:47:37.986 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-25T08:47:37.995 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-25T08:47:38.026 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 27/134 2023-08-25T08:47:38.167 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-25T08:47:38.204 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-25T08:47:38.340 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-25T08:47:38.369 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-25T08:47:38.510 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-25T08:47:38.535 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-25T08:47:38.775 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-25T08:47:38.858 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-25T08:47:38.973 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-25T08:47:39.097 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 28/134 2023-08-25T08:47:39.098 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-25T08:47:39.120 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-25T08:47:39.251 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-25T08:47:39.364 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-25T08:47:39.439 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-25T08:47:39.485 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-25T08:47:39.597 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-25T08:47:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-25T08:47:39.728 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-25T08:47:39.775 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-25T08:47:39.837 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-25T08:47:39.916 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-25T08:47:40.015 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-25T08:47:40.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-25T08:47:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T08:47:40.154 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-25T08:47:40.157 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T08:47:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T08:47:40.391 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-25T08:47:40.415 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-25T08:47:40.610 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-25T08:47:40.618 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-25T08:47:40.765 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T08:47:40.812 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-25T08:47:40.844 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T08:47:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-25T08:47:41.070 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-25T08:47:41.231 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-25T08:47:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-25T08:47:41.419 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-25T08:47:41.421 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-25T08:47:41.586 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-25T08:47:41.620 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-25T08:47:41.839 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-25T08:47:42.072 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-25T08:47:42.220 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-25T08:47:42.275 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T08:47:42.376 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-25T08:47:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-25T08:47:42.664 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-25T08:47:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-25T08:47:42.775 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-25T08:47:42.900 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-25T08:47:42.989 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-25T08:47:43.011 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-25T08:47:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-25T08:47:43.159 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-25T08:47:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-25T08:47:43.296 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-25T08:47:43.353 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 36/134 2023-08-25T08:47:43.520 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-25T08:47:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 37/134 2023-08-25T08:47:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-25T08:47:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-25T08:47:43.727 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-25T08:47:43.852 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-25T08:47:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-25T08:47:43.929 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-25T08:47:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-25T08:47:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-25T08:47:44.178 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-25T08:47:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-25T08:47:44.829 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-25T08:47:44.832 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-25T08:47:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-25T08:47:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-25T08:47:44.849 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-25T08:47:44.871 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-25T08:47:45.042 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T08:47:45.117 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T08:47:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-25T08:47:45.290 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 93/134 2023-08-25T08:47:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-25T08:47:45.668 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-25T08:47:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-25T08:47:45.954 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-25T08:47:46.196 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-25T08:47:46.211 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-25T08:47:46.508 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-25T08:47:46.553 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-25T08:47:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-25T08:47:46.852 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-25T08:47:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-25T08:47:47.114 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-25T08:47:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-25T08:47:47.308 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-25T08:47:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-25T08:47:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-25T08:47:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-25T08:47:48.102 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-25T08:47:48.249 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-25T08:47:48.314 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T08:47:48.371 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T08:47:48.523 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-25T08:47:48.539 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 93/134 2023-08-25T08:47:48.811 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-25T08:47:48.959 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:47:48.995 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-25T08:47:49.165 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:47:49.179 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-25T08:47:49.381 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T08:47:49.454 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T08:47:49.699 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-25T08:47:49.754 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:47:49.941 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-25T08:47:50.019 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T08:47:50.100 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T08:47:50.111 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T08:47:50.190 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T08:47:50.281 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T08:47:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-25T08:47:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-25T08:47:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-25T08:47:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-25T08:47:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-25T08:47:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-25T08:47:51.799 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-25T08:47:51.949 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-25T08:47:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-25T08:47:52.232 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:47:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-25T08:47:52.447 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:47:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-25T08:47:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-25T08:47:53.043 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-25T08:47:53.045 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:47:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-25T08:47:53.317 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T08:47:53.389 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T08:47:53.533 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-25T08:47:53.564 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T08:47:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-25T08:47:54.025 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-25T08:47:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-25T08:47:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-25T08:47:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-25T08:47:56.030 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-25T08:47:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-25T08:47:56.580 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-25T08:47:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-25T08:47:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-25T08:47:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-25T08:47:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-25T08:47:58.534 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T08:47:58.619 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T08:47:58.826 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 93/134 2023-08-25T08:48:03.728 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:48:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:48:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T08:48:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T08:48:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T08:48:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T08:48:08.839 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T08:48:08.840 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2023-08-25T08:48:08.840 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2023-08-25T08:48:08.840 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-08-25T08:48:08.840 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2023-08-25T08:48:08.841 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2023-08-25T08:48:08.841 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2023-08-25T08:48:08.841 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2023-08-25T08:48:08.841 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2023-08-25T08:48:08.841 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-08-25T08:48:08.841 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:48:09.195 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T08:48:10.784 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T08:48:10.990 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T08:48:11.211 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T08:48:11.404 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T08:48:11.857 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T08:48:12.028 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T08:48:12.045 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T08:48:12.243 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T08:48:12.244 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2023-08-25T08:48:12.244 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2023-08-25T08:48:12.244 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-08-25T08:48:12.244 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /run 2023-08-25T08:48:12.244 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2023-08-25T08:48:12.244 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2023-08-25T08:48:12.244 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2023-08-25T08:48:12.245 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2023-08-25T08:48:12.245 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-08-25T08:48:12.245 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:48:12.577 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T08:48:13.744 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T08:48:13.919 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T08:48:14.084 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 102/134 2023-08-25T08:48:14.161 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T08:48:14.294 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 103/134 2023-08-25T08:48:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T08:48:14.477 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T08:48:14.558 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T08:48:14.700 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T08:48:14.760 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T08:48:15.032 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T08:48:15.214 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T08:48:15.418 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T08:48:15.435 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T08:48:15.994 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T08:48:16.177 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T08:48:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T08:48:16.644 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 107/134 2023-08-25T08:48:16.867 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T08:48:16.902 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T08:48:17.084 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T08:48:17.095 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T08:48:17.183 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T08:48:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T08:48:17.448 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 102/134 2023-08-25T08:48:17.650 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 103/134 2023-08-25T08:48:17.834 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T08:48:18.054 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T08:48:18.338 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T08:48:18.397 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 110/134 2023-08-25T08:48:18.585 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T08:48:18.626 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T08:48:18.777 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T08:48:18.818 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T08:48:18.977 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T08:48:19.305 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T08:48:19.330 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T08:48:19.483 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T08:48:19.510 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T08:48:19.574 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T08:48:19.925 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 107/134 2023-08-25T08:48:20.183 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T08:48:20.218 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T08:48:20.375 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T08:48:20.386 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T08:48:21.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 110/134 2023-08-25T08:48:21.873 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T08:48:21.916 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T08:48:22.067 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T08:48:22.108 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T08:48:22.259 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T08:48:22.613 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T08:48:22.758 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T08:48:24.095 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T08:48:24.380 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 115/134 2023-08-25T08:48:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T08:48:24.740 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T08:48:25.600 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T08:48:25.600 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2023-08-25T08:48:25.600 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2023-08-25T08:48:25.600 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-08-25T08:48:25.601 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /run 2023-08-25T08:48:25.601 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2023-08-25T08:48:25.601 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2023-08-25T08:48:25.601 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2023-08-25T08:48:25.601 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2023-08-25T08:48:25.601 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-08-25T08:48:25.601 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:48:25.646 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-25T08:48:25.953 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T08:48:25.966 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-25T08:48:26.153 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 119/134 2023-08-25T08:48:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T08:48:26.563 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 121/134 2023-08-25T08:48:26.735 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T08:48:27.343 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T08:48:27.617 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 115/134 2023-08-25T08:48:27.697 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T08:48:27.757 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T08:48:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T08:48:27.948 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T08:48:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T08:48:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T08:48:28.815 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-25T08:48:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T08:48:28.929 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:29.086 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T08:48:29.123 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-25T08:48:29.127 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T08:48:29.208 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:29.226 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:29.285 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 119/134 2023-08-25T08:48:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 124/134 2023-08-25T08:48:29.675 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T08:48:29.675 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 121/134 2023-08-25T08:48:29.707 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 125/134 2023-08-25T08:48:29.799 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T08:48:30.016 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 126/134 2023-08-25T08:48:30.343 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-25T08:48:30.483 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:30.491 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:30.668 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-25T08:48:30.858 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T08:48:30.944 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T08:48:31.141 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T08:48:31.153 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T08:48:31.154 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:31.336 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:31.336 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T08:48:31.571 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T08:48:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T08:48:31.811 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:31.941 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T08:48:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 102/134 2023-08-25T08:48:32.198 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:32.216 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:32.307 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:32.307 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 1/134 2023-08-25T08:48:32.308 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2/134 2023-08-25T08:48:32.308 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 3/134 2023-08-25T08:48:32.308 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 4/134 2023-08-25T08:48:32.308 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/134 2023-08-25T08:48:32.308 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:48:32.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:48:32.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:48:32.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T08:48:32.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 10/134 2023-08-25T08:48:32.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 11/134 2023-08-25T08:48:32.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 12/134 2023-08-25T08:48:32.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 13/134 2023-08-25T08:48:32.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 14/134 2023-08-25T08:48:32.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 15/134 2023-08-25T08:48:32.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 16/134 2023-08-25T08:48:32.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 17/134 2023-08-25T08:48:32.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 18/134 2023-08-25T08:48:32.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 19/134 2023-08-25T08:48:32.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/134 2023-08-25T08:48:32.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 21/134 2023-08-25T08:48:32.311 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 22/134 2023-08-25T08:48:32.311 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 23/134 2023-08-25T08:48:32.311 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 24/134 2023-08-25T08:48:32.311 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 25/134 2023-08-25T08:48:32.311 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 26/134 2023-08-25T08:48:32.311 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 27/134 2023-08-25T08:48:32.311 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 28/134 2023-08-25T08:48:32.312 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/134 2023-08-25T08:48:32.312 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 30/134 2023-08-25T08:48:32.312 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 31/134 2023-08-25T08:48:32.320 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/134 2023-08-25T08:48:32.320 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/134 2023-08-25T08:48:32.320 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 34/134 2023-08-25T08:48:32.321 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/134 2023-08-25T08:48:32.321 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 36/134 2023-08-25T08:48:32.321 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 37/134 2023-08-25T08:48:32.322 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 38/134 2023-08-25T08:48:32.322 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-25T08:48:32.322 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-25T08:48:32.322 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-25T08:48:32.323 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-25T08:48:32.323 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-25T08:48:32.323 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-25T08:48:32.323 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-25T08:48:32.324 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-25T08:48:32.324 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-25T08:48:32.324 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-25T08:48:32.324 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-25T08:48:32.325 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-25T08:48:32.325 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-25T08:48:32.325 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-25T08:48:32.325 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-25T08:48:32.326 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-25T08:48:32.326 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-25T08:48:32.326 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-25T08:48:32.326 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-25T08:48:32.326 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-25T08:48:32.326 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-25T08:48:32.327 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-25T08:48:32.327 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-25T08:48:32.327 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-25T08:48:32.327 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-25T08:48:32.327 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-25T08:48:32.327 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-25T08:48:32.328 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-25T08:48:32.328 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-25T08:48:32.328 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-25T08:48:32.328 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-25T08:48:32.328 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-25T08:48:32.329 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-25T08:48:32.329 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-25T08:48:32.329 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-25T08:48:32.329 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-25T08:48:32.329 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-25T08:48:32.330 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-25T08:48:32.330 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-25T08:48:32.330 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-25T08:48:32.330 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-25T08:48:32.330 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-25T08:48:32.331 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-25T08:48:32.331 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-25T08:48:32.331 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-25T08:48:32.331 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-25T08:48:32.332 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-25T08:48:32.332 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-25T08:48:32.332 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-25T08:48:32.332 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-25T08:48:32.332 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-25T08:48:32.332 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-25T08:48:32.333 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-25T08:48:32.333 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-25T08:48:32.333 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-25T08:48:32.333 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-25T08:48:32.333 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-25T08:48:32.333 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-25T08:48:32.333 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-25T08:48:32.334 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-25T08:48:32.334 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-25T08:48:32.334 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-25T08:48:32.334 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-25T08:48:32.334 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-25T08:48:32.334 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-25T08:48:32.334 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-25T08:48:32.335 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-25T08:48:32.335 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-25T08:48:32.335 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-25T08:48:32.335 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-25T08:48:32.335 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-25T08:48:32.335 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-25T08:48:32.335 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-25T08:48:32.336 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-25T08:48:32.336 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-25T08:48:32.336 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-25T08:48:32.336 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-25T08:48:32.336 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-25T08:48:32.336 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-25T08:48:32.336 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-25T08:48:32.337 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-25T08:48:32.337 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T08:48:32.337 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-25T08:48:32.337 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T08:48:32.337 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-25T08:48:32.337 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-25T08:48:32.337 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-25T08:48:32.337 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-25T08:48:32.338 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-25T08:48:32.338 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:32.338 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 129/134 2023-08-25T08:48:32.338 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-25T08:48:32.338 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 131/134 2023-08-25T08:48:32.338 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:32.339 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-25T08:48:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 103/134 2023-08-25T08:48:32.373 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 124/134 2023-08-25T08:48:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T08:48:32.689 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 125/134 2023-08-25T08:48:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T08:48:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T08:48:33.004 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 126/134 2023-08-25T08:48:33.300 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-25T08:48:33.448 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:33.703 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-25T08:48:33.814 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T08:48:33.894 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T08:48:34.055 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T08:48:34.131 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T08:48:34.143 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T08:48:34.144 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:34.284 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T08:48:34.367 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:34.368 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T08:48:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T08:48:34.594 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T08:48:34.796 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:34.827 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 107/134 2023-08-25T08:48:34.946 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:34.946 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:52 ago on Fri 25 Aug 2023 08:46:42 AM UTC. 2023-08-25T08:48:35.068 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T08:48:35.132 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T08:48:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 1/134 2023-08-25T08:48:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2/134 2023-08-25T08:48:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 3/134 2023-08-25T08:48:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 4/134 2023-08-25T08:48:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/134 2023-08-25T08:48:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:48:35.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:48:35.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:48:35.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T08:48:35.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 10/134 2023-08-25T08:48:35.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 11/134 2023-08-25T08:48:35.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 12/134 2023-08-25T08:48:35.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 13/134 2023-08-25T08:48:35.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 14/134 2023-08-25T08:48:35.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 15/134 2023-08-25T08:48:35.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 16/134 2023-08-25T08:48:35.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 17/134 2023-08-25T08:48:35.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 18/134 2023-08-25T08:48:35.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 19/134 2023-08-25T08:48:35.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/134 2023-08-25T08:48:35.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 21/134 2023-08-25T08:48:35.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 22/134 2023-08-25T08:48:35.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 23/134 2023-08-25T08:48:35.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 24/134 2023-08-25T08:48:35.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 25/134 2023-08-25T08:48:35.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 26/134 2023-08-25T08:48:35.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 27/134 2023-08-25T08:48:35.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 28/134 2023-08-25T08:48:35.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/134 2023-08-25T08:48:35.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 30/134 2023-08-25T08:48:35.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 31/134 2023-08-25T08:48:35.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/134 2023-08-25T08:48:35.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/134 2023-08-25T08:48:35.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 34/134 2023-08-25T08:48:35.325 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/134 2023-08-25T08:48:35.325 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 36/134 2023-08-25T08:48:35.325 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 37/134 2023-08-25T08:48:35.325 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 38/134 2023-08-25T08:48:35.325 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-25T08:48:35.325 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-25T08:48:35.325 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-25T08:48:35.325 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-25T08:48:35.326 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-25T08:48:35.326 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-25T08:48:35.326 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-25T08:48:35.326 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-25T08:48:35.326 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-25T08:48:35.326 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-25T08:48:35.326 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-25T08:48:35.326 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-25T08:48:35.327 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-25T08:48:35.327 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-25T08:48:35.327 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-25T08:48:35.327 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-25T08:48:35.327 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-25T08:48:35.327 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-25T08:48:35.327 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-25T08:48:35.327 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-25T08:48:35.328 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-25T08:48:35.328 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-25T08:48:35.328 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-25T08:48:35.328 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-25T08:48:35.328 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-25T08:48:35.328 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-25T08:48:35.328 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-25T08:48:35.329 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-25T08:48:35.329 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-25T08:48:35.329 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-25T08:48:35.329 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-25T08:48:35.329 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-25T08:48:35.329 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-25T08:48:35.329 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-25T08:48:35.329 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-25T08:48:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-25T08:48:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-25T08:48:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-25T08:48:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-25T08:48:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-25T08:48:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-25T08:48:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-25T08:48:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-25T08:48:35.331 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-25T08:48:35.331 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-25T08:48:35.331 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-25T08:48:35.331 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-25T08:48:35.331 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-25T08:48:35.331 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-25T08:48:35.331 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-25T08:48:35.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-25T08:48:35.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-25T08:48:35.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-25T08:48:35.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-25T08:48:35.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-25T08:48:35.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-25T08:48:35.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-25T08:48:35.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-25T08:48:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-25T08:48:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-25T08:48:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-25T08:48:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-25T08:48:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-25T08:48:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-25T08:48:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-25T08:48:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-25T08:48:35.334 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-25T08:48:35.334 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-25T08:48:35.334 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-25T08:48:35.334 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-25T08:48:35.334 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-25T08:48:35.334 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-25T08:48:35.334 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-25T08:48:35.334 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-25T08:48:35.335 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-25T08:48:35.335 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-25T08:48:35.335 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-25T08:48:35.335 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-25T08:48:35.335 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-25T08:48:35.335 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-25T08:48:35.335 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-25T08:48:35.336 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T08:48:35.336 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-25T08:48:35.336 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T08:48:35.336 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-25T08:48:35.336 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-25T08:48:35.336 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-25T08:48:35.336 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-25T08:48:35.337 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-25T08:48:35.337 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:35.337 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 129/134 2023-08-25T08:48:35.337 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-25T08:48:35.337 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 131/134 2023-08-25T08:48:35.337 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:35.337 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-25T08:48:35.338 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T08:48:35.353 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T08:48:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 110/134 2023-08-25T08:48:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T08:48:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T08:48:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T08:48:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T08:48:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T08:48:37.746 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:37.746 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:55 ago on Fri 25 Aug 2023 08:46:42 AM UTC. 2023-08-25T08:48:37.756 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2023-08-25T08:48:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T08:48:37.969 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:48:37.969 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-08-25T08:48:37.969 INFO:teuthology.orchestra.run.smithi181.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-25T08:48:37.969 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.969 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.970 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-08-25T08:48:37.970 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-25T08:48:37.970 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.1-7.el8.noarch 2023-08-25T08:48:37.971 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.0.4-10.el8.x86_64 2023-08-25T08:48:37.971 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.971 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.971 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.971 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.971 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.972 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.972 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.972 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.972 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.972 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.972 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.972 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.973 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.973 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.973 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.973 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.973 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.973 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.973 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.974 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.974 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.974 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.974 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:37.974 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-25T08:48:37.974 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.1-9.el8.x86_64 2023-08-25T08:48:37.974 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-25T08:48:37.974 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-25T08:48:37.975 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-3.el8.x86_64 2023-08-25T08:48:37.975 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-25T08:48:37.975 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-25T08:48:37.975 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-25T08:48:37.975 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.975 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.975 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.976 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-25T08:48:37.976 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-25T08:48:37.976 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-25T08:48:37.976 INFO:teuthology.orchestra.run.smithi181.stdout: libpq-13.5-1.el8.x86_64 2023-08-25T08:48:37.976 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-25T08:48:37.976 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-25T08:48:37.976 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.977 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.977 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-25T08:48:37.977 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.977 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-25T08:48:37.977 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-25T08:48:37.978 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-25T08:48:37.978 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-25T08:48:37.978 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-25T08:48:37.978 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-25T08:48:37.978 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-25T08:48:37.978 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-25T08:48:37.978 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-25T08:48:37.979 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-25T08:48:37.979 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-25T08:48:37.979 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-25T08:48:37.979 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-25T08:48:37.979 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-25T08:48:37.979 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:37.979 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:37.979 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:37.980 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-25T08:48:37.980 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-25T08:48:37.980 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-25T08:48:37.980 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-25T08:48:37.980 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-25T08:48:37.980 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.981 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.981 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.981 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-25T08:48:37.981 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-25T08:48:37.981 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-25T08:48:37.981 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-25T08:48:37.981 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-25T08:48:37.982 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-25T08:48:37.982 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-25T08:48:37.982 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-25T08:48:37.982 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-25T08:48:37.982 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-25T08:48:37.982 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-25T08:48:37.982 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-25T08:48:37.983 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-25T08:48:37.983 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-25T08:48:37.983 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-25T08:48:37.983 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-25T08:48:37.983 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-25T08:48:37.983 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-25T08:48:37.983 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-25T08:48:37.984 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-25T08:48:37.984 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-25T08:48:37.984 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-25T08:48:37.984 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2023-08-25T08:48:37.984 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-25T08:48:37.984 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-25T08:48:37.985 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-25T08:48:37.985 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-25T08:48:37.985 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-25T08:48:37.985 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-25T08:48:37.985 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.985 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.985 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-25T08:48:37.986 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-25T08:48:37.986 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.986 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-25T08:48:37.986 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-25T08:48:37.986 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-25T08:48:37.986 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-25T08:48:37.986 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-25T08:48:37.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-25T08:48:37.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-25T08:48:37.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-25T08:48:37.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-25T08:48:37.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-25T08:48:37.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-25T08:48:37.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-25T08:48:37.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-25T08:48:37.988 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.988 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.988 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:37.988 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-25T08:48:37.988 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-25T08:48:37.988 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-25T08:48:37.988 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-25T08:48:37.989 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-25T08:48:37.989 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-25T08:48:37.989 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:48:37.989 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-08-25T08:48:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T08:48:38.148 DEBUG:teuthology.parallel:result is None 2023-08-25T08:48:40.627 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2023-08-25T08:48:40.840 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:48:40.840 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2023-08-25T08:48:40.841 INFO:teuthology.orchestra.run.smithi157.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-25T08:48:40.841 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.841 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.841 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-08-25T08:48:40.841 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-25T08:48:40.841 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.1-7.el8.noarch 2023-08-25T08:48:40.841 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.0.4-10.el8.x86_64 2023-08-25T08:48:40.842 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.842 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.842 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.842 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.842 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.842 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.842 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.842 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.843 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.843 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.843 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.843 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.843 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.843 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.843 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.844 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.844 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.844 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.844 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.844 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.844 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.844 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.845 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:40.845 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-25T08:48:40.845 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.1-9.el8.x86_64 2023-08-25T08:48:40.845 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-25T08:48:40.845 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-25T08:48:40.845 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-3.el8.x86_64 2023-08-25T08:48:40.845 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-25T08:48:40.845 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-25T08:48:40.846 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-25T08:48:40.846 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.846 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.846 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.846 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-25T08:48:40.846 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-25T08:48:40.846 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-25T08:48:40.846 INFO:teuthology.orchestra.run.smithi157.stdout: libpq-13.5-1.el8.x86_64 2023-08-25T08:48:40.847 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-25T08:48:40.847 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-25T08:48:40.847 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.847 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.847 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-25T08:48:40.847 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.847 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-25T08:48:40.847 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-25T08:48:40.848 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-25T08:48:40.848 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-25T08:48:40.848 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-25T08:48:40.848 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-25T08:48:40.848 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-25T08:48:40.848 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-25T08:48:40.848 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-25T08:48:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-25T08:48:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-25T08:48:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-25T08:48:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-25T08:48:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-25T08:48:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:40.850 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-25T08:48:40.850 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-25T08:48:40.850 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-25T08:48:40.850 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-25T08:48:40.850 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-25T08:48:40.850 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.850 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.850 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.851 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-25T08:48:40.851 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-25T08:48:40.851 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-25T08:48:40.851 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-25T08:48:40.851 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-25T08:48:40.851 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-25T08:48:40.851 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-25T08:48:40.851 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-25T08:48:40.852 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-25T08:48:40.852 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-25T08:48:40.852 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-25T08:48:40.852 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-25T08:48:40.852 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-25T08:48:40.852 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-25T08:48:40.852 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-25T08:48:40.852 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-25T08:48:40.853 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-25T08:48:40.853 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-25T08:48:40.853 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-25T08:48:40.853 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-25T08:48:40.853 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-25T08:48:40.853 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-25T08:48:40.853 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-2.6-1.el8.noarch 2023-08-25T08:48:40.854 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-25T08:48:40.854 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-25T08:48:40.854 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-25T08:48:40.854 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-25T08:48:40.854 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-25T08:48:40.854 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-25T08:48:40.854 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.854 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.855 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-25T08:48:40.855 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-25T08:48:40.855 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.855 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-25T08:48:40.855 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-25T08:48:40.855 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-25T08:48:40.855 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-25T08:48:40.855 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-25T08:48:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-25T08:48:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-25T08:48:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-25T08:48:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-25T08:48:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-25T08:48:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-25T08:48:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-25T08:48:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-25T08:48:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-25T08:48:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-25T08:48:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-25T08:48:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-25T08:48:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-25T08:48:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-25T08:48:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-08-25T08:48:40.858 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-08-25T08:48:41.023 DEBUG:teuthology.parallel:result is None 2023-08-25T08:48:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T08:48:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 115/134 2023-08-25T08:48:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T08:48:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T08:48:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-25T08:48:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-25T08:48:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 119/134 2023-08-25T08:48:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T08:48:46.170 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 121/134 2023-08-25T08:48:46.324 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T08:48:49.599 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:49.893 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T08:48:50.232 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 124/134 2023-08-25T08:48:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 125/134 2023-08-25T08:48:50.815 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 126/134 2023-08-25T08:48:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-25T08:48:51.289 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:51.311 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-25T08:48:51.780 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T08:48:51.884 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T08:48:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T08:48:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T08:48:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T08:48:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T08:48:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:53.134 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:53.135 INFO:teuthology.orchestra.run.smithi099.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-25T08:48:53.135 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:48:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 1/134 2023-08-25T08:48:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2/134 2023-08-25T08:48:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 3/134 2023-08-25T08:48:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 4/134 2023-08-25T08:48:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/134 2023-08-25T08:48:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T08:48:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T08:48:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T08:48:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T08:48:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 10/134 2023-08-25T08:48:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 11/134 2023-08-25T08:48:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 12/134 2023-08-25T08:48:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 13/134 2023-08-25T08:48:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 14/134 2023-08-25T08:48:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 15/134 2023-08-25T08:48:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 16/134 2023-08-25T08:48:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 17/134 2023-08-25T08:48:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 18/134 2023-08-25T08:48:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 19/134 2023-08-25T08:48:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/134 2023-08-25T08:48:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 21/134 2023-08-25T08:48:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 22/134 2023-08-25T08:48:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 23/134 2023-08-25T08:48:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 24/134 2023-08-25T08:48:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 25/134 2023-08-25T08:48:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 26/134 2023-08-25T08:48:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 27/134 2023-08-25T08:48:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 28/134 2023-08-25T08:48:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/134 2023-08-25T08:48:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 30/134 2023-08-25T08:48:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 31/134 2023-08-25T08:48:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/134 2023-08-25T08:48:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/134 2023-08-25T08:48:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 34/134 2023-08-25T08:48:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/134 2023-08-25T08:48:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 36/134 2023-08-25T08:48:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 37/134 2023-08-25T08:48:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 38/134 2023-08-25T08:48:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-25T08:48:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-25T08:48:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-25T08:48:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-25T08:48:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-25T08:48:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-25T08:48:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-25T08:48:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-25T08:48:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-25T08:48:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-25T08:48:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-25T08:48:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-25T08:48:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-25T08:48:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-25T08:48:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-25T08:48:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-25T08:48:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-25T08:48:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-25T08:48:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-25T08:48:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-25T08:48:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-25T08:48:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-25T08:48:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-25T08:48:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-25T08:48:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-25T08:48:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-25T08:48:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-25T08:48:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-25T08:48:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-25T08:48:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-25T08:48:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-25T08:48:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-25T08:48:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-25T08:48:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-25T08:48:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-25T08:48:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-25T08:48:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-25T08:48:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-25T08:48:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-25T08:48:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-25T08:48:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-25T08:48:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-25T08:48:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-25T08:48:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-25T08:48:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-25T08:48:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-25T08:48:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-25T08:48:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-25T08:48:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-25T08:48:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-25T08:48:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-25T08:48:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-25T08:48:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-25T08:48:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-25T08:48:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-25T08:48:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-25T08:48:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-25T08:48:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-25T08:48:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-25T08:48:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-25T08:48:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-25T08:48:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-25T08:48:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-25T08:48:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-25T08:48:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-25T08:48:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-25T08:48:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-25T08:48:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-25T08:48:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-25T08:48:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-25T08:48:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-25T08:48:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-25T08:48:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-25T08:48:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-25T08:48:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-25T08:48:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-25T08:48:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-25T08:48:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-25T08:48:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-25T08:48:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-25T08:48:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-25T08:48:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T08:48:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-25T08:48:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T08:48:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-25T08:48:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-25T08:48:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-25T08:48:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-25T08:48:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-25T08:48:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T08:48:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 129/134 2023-08-25T08:48:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-25T08:48:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 131/134 2023-08-25T08:48:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T08:48:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-25T08:48:56.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T08:48:56.516 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:12 ago on Fri 25 Aug 2023 08:46:44 AM UTC. 2023-08-25T08:48:59.315 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2023-08-25T08:48:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:48:59.554 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2023-08-25T08:48:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-25T08:48:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.554 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-08-25T08:48:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-25T08:48:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.1-7.el8.noarch 2023-08-25T08:48:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.0.4-10.el8.x86_64 2023-08-25T08:48:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T08:48:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-25T08:48:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.1-9.el8.x86_64 2023-08-25T08:48:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-25T08:48:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-25T08:48:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-3.el8.x86_64 2023-08-25T08:48:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-25T08:48:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-25T08:48:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-25T08:48:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-25T08:48:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-25T08:48:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-25T08:48:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: libpq-13.5-1.el8.x86_64 2023-08-25T08:48:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-25T08:48:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-25T08:48:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-25T08:48:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-25T08:48:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-25T08:48:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-25T08:48:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-25T08:48:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-25T08:48:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-25T08:48:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-25T08:48:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-25T08:48:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-25T08:48:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-25T08:48:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-25T08:48:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-25T08:48:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-25T08:48:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-25T08:48:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T08:48:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-25T08:48:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-25T08:48:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-25T08:48:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-25T08:48:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-25T08:48:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-25T08:48:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-25T08:48:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-25T08:48:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-25T08:48:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-25T08:48:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-25T08:48:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-25T08:48:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-25T08:48:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-25T08:48:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-25T08:48:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-25T08:48:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-25T08:48:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-25T08:48:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-25T08:48:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-25T08:48:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-25T08:48:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-25T08:48:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-25T08:48:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-25T08:48:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-25T08:48:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-25T08:48:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-25T08:48:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-2.6-1.el8.noarch 2023-08-25T08:48:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-25T08:48:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-25T08:48:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-25T08:48:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-25T08:48:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-25T08:48:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-25T08:48:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-25T08:48:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-25T08:48:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-25T08:48:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-25T08:48:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-25T08:48:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-25T08:48:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-25T08:48:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-25T08:48:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-25T08:48:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-25T08:48:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-25T08:48:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-25T08:48:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-25T08:48:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-25T08:48:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-25T08:48:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T08:48:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-25T08:48:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-25T08:48:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-25T08:48:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-25T08:48:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-25T08:48:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-25T08:48:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:48:59.574 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-08-25T08:48:59.900 DEBUG:teuthology.parallel:result is None 2023-08-25T08:48:59.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:49:00.044 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T08:49:00.105 INFO:teuthology.orchestra.run.smithi099.stdout:18.0.0-5703.gba6b39e4.el8 2023-08-25T08:49:00.105 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el8 2023-08-25T08:49:00.106 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T08:49:00.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:49:00.247 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T08:49:00.277 INFO:teuthology.orchestra.run.smithi157.stdout:18.0.0-5703.gba6b39e4.el8 2023-08-25T08:49:00.278 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el8 2023-08-25T08:49:00.278 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T08:49:00.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:49:00.425 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T08:49:00.457 INFO:teuthology.orchestra.run.smithi181.stdout:18.0.0-5703.gba6b39e4.el8 2023-08-25T08:49:00.458 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el8 2023-08-25T08:49:00.458 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T08:49:00.459 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-25T08:49:00.459 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:49:00.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T08:49:00.566 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:49:00.566 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T08:49:00.615 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:49:00.615 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T08:49:00.663 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-25T08:49:00.664 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:49:00.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T08:49:00.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T08:49:00.843 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:49:00.844 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T08:49:00.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T08:49:00.948 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:49:00.948 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T08:49:00.982 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T08:49:01.051 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-25T08:49:01.052 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:49:01.052 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T08:49:01.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T08:49:01.227 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:49:01.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T08:49:01.260 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T08:49:01.332 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:49:01.332 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T08:49:01.364 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T08:49:01.435 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-25T08:49:01.436 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:49:01.436 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T08:49:01.504 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T08:49:01.612 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:49:01.613 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T08:49:01.657 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T08:49:01.728 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:49:01.729 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T08:49:01.760 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T08:49:01.831 INFO:teuthology.run_tasks:Running task cephadm... 2023-08-25T08:49:02.054 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-08-25T08:49:02.055 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:49:02.065 INFO:tasks.cephadm:Cluster fsid is 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:02.066 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-08-25T08:49:02.066 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.99', 'mon.b': '172.21.15.157', 'mon.c': '172.21.15.181'} 2023-08-25T08:49:02.067 INFO:tasks.cephadm:First mon is mon.a on smithi099 2023-08-25T08:49:02.067 INFO:tasks.cephadm:First mgr is x 2023-08-25T08:49:02.067 INFO:tasks.cephadm:Normalizing hostnames... 2023-08-25T08:49:02.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2023-08-25T08:49:02.154 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2023-08-25T08:49:02.188 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2023-08-25T08:49:02.222 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-08-25T08:49:02.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:49:02.445 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72276/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-5703-gba6b39e4', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-5703.gba6b39e4'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-08-24 23:23:18.626495', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023aug3-b5', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-08-25T08:49:02.597 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023aug3-b5, sha1 ba6b39e4f4f77fb95c747099b53db0467d435f56 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T08:49:02.598 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/x86_64/flavors/default/cephadm 2023-08-25T08:49:02.598 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:03.187 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-rw-r--. 1 ubuntu ubuntu 705688 Aug 25 08:49 /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:03.188 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:03.574 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-rw-r--. 1 ubuntu ubuntu 705688 Aug 25 08:49 /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:03.574 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:03.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-rw-r--. 1 ubuntu ubuntu 705688 Aug 25 08:49 /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:03.959 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:04.010 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:04.033 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-25T08:49:04.091 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 on all hosts... 2023-08-25T08:49:04.091 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 pull 2023-08-25T08:49:04.096 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 pull 2023-08-25T08:49:04.098 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 pull 2023-08-25T08:49:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T08:49:05.487 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T08:49:09.235 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T08:49:30.934 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-08-25T08:49:30.935 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T08:49:30.935 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef", 2023-08-25T08:49:30.935 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2023-08-25T08:49:30.935 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e263fcaadb67f0bae2627ea3f04151ef1bf468f74f698991bba88a3316474934" 2023-08-25T08:49:30.935 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-08-25T08:49:30.935 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-08-25T08:49:32.072 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-08-25T08:49:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T08:49:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef", 2023-08-25T08:49:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2023-08-25T08:49:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e263fcaadb67f0bae2627ea3f04151ef1bf468f74f698991bba88a3316474934" 2023-08-25T08:49:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-08-25T08:49:32.074 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-08-25T08:49:42.100 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T08:49:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T08:49:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef", 2023-08-25T08:49:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2023-08-25T08:49:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e263fcaadb67f0bae2627ea3f04151ef1bf468f74f698991bba88a3316474934" 2023-08-25T08:49:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T08:49:42.102 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T08:49:42.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2023-08-25T08:49:42.200 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2023-08-25T08:49:42.238 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2023-08-25T08:49:42.277 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2023-08-25T08:49:42.344 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2023-08-25T08:49:42.378 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2023-08-25T08:49:42.411 INFO:tasks.cephadm:Writing seed config... 2023-08-25T08:49:42.413 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-08-25T08:49:42.413 INFO:tasks.cephadm: override: [client] debug client = 20 2023-08-25T08:49:42.413 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-08-25T08:49:42.414 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-08-25T08:49:42.414 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-08-25T08:49:42.414 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-08-25T08:49:42.414 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-08-25T08:49:42.415 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-08-25T08:49:42.415 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-08-25T08:49:42.415 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-08-25T08:49:42.416 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-08-25T08:49:42.416 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-08-25T08:49:42.416 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-08-25T08:49:42.416 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-08-25T08:49:42.416 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-08-25T08:49:42.417 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-08-25T08:49:42.417 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-08-25T08:49:42.417 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-08-25T08:49:42.417 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-08-25T08:49:42.417 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-08-25T08:49:42.417 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-08-25T08:49:42.418 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-08-25T08:49:42.418 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-08-25T08:49:42.418 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-08-25T08:49:42.418 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-08-25T08:49:42.418 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-08-25T08:49:42.419 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-08-25T08:49:42.419 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-08-25T08:49:42.419 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-08-25T08:49:42.419 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-08-25T08:49:42.419 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-08-25T08:49:42.420 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-08-25T08:49:42.420 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-08-25T08:49:42.420 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-08-25T08:49:42.420 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-08-25T08:49:42.420 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-08-25T08:49:42.421 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-08-25T08:49:42.421 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-08-25T08:49:42.421 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-08-25T08:49:42.421 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-08-25T08:49:42.421 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-08-25T08:49:42.422 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-08-25T08:49:42.422 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-08-25T08:49:42.422 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-08-25T08:49:42.423 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:49:42.423 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-08-25T08:49:42.466 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3d4188ec-4324-11ee-9b3d-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-08-25T08:49:42.467 DEBUG:teuthology.orchestra.run.smithi099:mon.a> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.a.service 2023-08-25T08:49:42.511 DEBUG:teuthology.orchestra.run.smithi099:mgr.x> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.x.service 2023-08-25T08:49:42.556 INFO:tasks.cephadm:Bootstrapping... 2023-08-25T08:49:42.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 -v bootstrap --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.99 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-08-25T08:49:42.577 INFO:journalctl@ceph.mon.a.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:49:42.622 INFO:journalctl@ceph.mgr.x.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:49:42.776 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-08-25T08:49:42.776 INFO:teuthology.orchestra.run.smithi099.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56', '-v', 'bootstrap', '--fsid', '3d4188ec-4324-11ee-9b3d-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.99', '--skip-admin-label'] 2023-08-25T08:49:42.947 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.1.1 2023-08-25T08:49:42.948 INFO:teuthology.orchestra.run.smithi099.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-08-25T08:49:42.948 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying podman|docker is present... 2023-08-25T08:49:43.118 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.1.1 2023-08-25T08:49:43.119 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying lvm2 is present... 2023-08-25T08:49:43.119 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying time synchronization is in place... 2023-08-25T08:49:43.133 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-08-25T08:49:43.133 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-08-25T08:49:43.145 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-08-25T08:49:43.146 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2023-08-25T08:49:43.159 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2023-08-25T08:49:43.171 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2023-08-25T08:49:43.172 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2023-08-25T08:49:43.172 INFO:teuthology.orchestra.run.smithi099.stdout:Repeating the final host check... 2023-08-25T08:49:43.340 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.1.1 2023-08-25T08:49:43.341 INFO:teuthology.orchestra.run.smithi099.stdout:podman (/bin/podman) version 4.1.1 is present 2023-08-25T08:49:43.341 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl is present 2023-08-25T08:49:43.341 INFO:teuthology.orchestra.run.smithi099.stdout:lvcreate is present 2023-08-25T08:49:43.353 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-08-25T08:49:43.354 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-08-25T08:49:43.366 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-08-25T08:49:43.367 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2023-08-25T08:49:43.379 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2023-08-25T08:49:43.391 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2023-08-25T08:49:43.392 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2023-08-25T08:49:43.392 INFO:teuthology.orchestra.run.smithi099.stdout:Host looks OK 2023-08-25T08:49:43.392 INFO:teuthology.orchestra.run.smithi099.stdout:Cluster fsid: 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:43.392 INFO:teuthology.orchestra.run.smithi099.stdout:Acquiring lock 140000869853672 on /run/cephadm/3d4188ec-4324-11ee-9b3d-001a4aab830c.lock 2023-08-25T08:49:43.393 INFO:teuthology.orchestra.run.smithi099.stdout:Lock 140000869853672 acquired on /run/cephadm/3d4188ec-4324-11ee-9b3d-001a4aab830c.lock 2023-08-25T08:49:43.393 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 3300 ... 2023-08-25T08:49:43.393 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 6789 ... 2023-08-25T08:49:43.393 INFO:teuthology.orchestra.run.smithi099.stdout:Base mon IP(s) is [172.21.15.99:3300, 172.21.15.99:6789], mon addrv is [v2:172.21.15.99:3300,v1:172.21.15.99:6789] 2023-08-25T08:49:43.401 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-08-25T08:49:43.401 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.99 metric 100 2023-08-25T08:49:43.407 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-08-25T08:49:43.408 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-08-25T08:49:43.408 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-08-25T08:49:43.415 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-08-25T08:49:43.415 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-08-25T08:49:43.415 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-08-25T08:49:43.416 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-08-25T08:49:43.416 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 fe80::4045:82b9:bb79:8b92/64 scope link noprefixroute 2023-08-25T08:49:43.416 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-08-25T08:49:43.417 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2023-08-25T08:49:43.417 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2023-08-25T08:49:43.417 INFO:teuthology.orchestra.run.smithi099.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-08-25T08:49:43.417 INFO:teuthology.orchestra.run.smithi099.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-08-25T08:49:43.418 INFO:teuthology.orchestra.run.smithi099.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T08:49:43.887 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef 2023-08-25T08:49:43.887 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T08:49:43.888 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Getting image source signatures 2023-08-25T08:49:43.888 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:57753a2dbe9e592c0393092b3155420b9dbb1f83f68296cfecb3d2f029e1ad16 2023-08-25T08:49:43.888 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:6cfdaf403fb5ea185397a3afe33998eb8d2277a1b4936b2369ceaaceb6049ebf 2023-08-25T08:49:43.888 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying config sha256:95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef 2023-08-25T08:49:43.888 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Writing manifest to image destination 2023-08-25T08:49:43.888 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Storing signatures 2023-08-25T08:49:44.861 INFO:teuthology.orchestra.run.smithi099.stdout:ceph: stdout ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T08:49:45.716 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph version: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T08:49:45.717 INFO:teuthology.orchestra.run.smithi099.stdout:Extracting ceph user uid/gid from container image... 2023-08-25T08:49:45.732 INFO:teuthology.orchestra.run.smithi099.stdout:stat: stdout 167 167 2023-08-25T08:49:45.733 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial keys... 2023-08-25T08:49:46.507 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAqa+hkScaNBhAAL+XO6dWYxLrVcAs7BA7xbw== 2023-08-25T08:49:47.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAqa+hkzWe3NRAA/nu1FpOMNJ1WEiE9FOl+wg== 2023-08-25T08:49:48.160 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAra+hkdW/LKRAAtk5yyy0yp4Nb72LWrJJqLg== 2023-08-25T08:49:48.161 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial monmap... 2023-08-25T08:49:49.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-08-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-08-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-08-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool for a [v2:172.21.15.99:3300,v1:172.21.15.99:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-08-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = pacific 2023-08-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: set fsid to 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-08-25T08:49:49.438 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:49:49.438 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mon... 2023-08-25T08:49:50.217 INFO:teuthology.orchestra.run.smithi099.stdout:create mon.a on 2023-08-25T08:49:50.470 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-08-25T08:49:50.668 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-08-25T08:49:50.906 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c.target → /etc/systemd/system/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c.target. 2023-08-25T08:49:50.906 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c.target → /etc/systemd/system/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c.target. 2023-08-25T08:49:51.272 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.a 2023-08-25T08:49:51.272 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.a.service: Unit ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.a.service not loaded. 2023-08-25T08:49:51.466 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c.target.wants/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@.service. 2023-08-25T08:49:51.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:51 smithi099 systemd[1]: Starting Ceph mon.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:49:52.213 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:51 smithi099 podman[97097]: 2023-08-25T08:49:52.270 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-25T08:49:52.271 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2023-08-25T08:49:52.287 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-25T08:49:52.287 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2023-08-25T08:49:52.287 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2023-08-25T08:49:52.288 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2023-08-25T08:49:52.288 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon to start... 2023-08-25T08:49:52.288 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon... 2023-08-25T08:49:52.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:52 smithi099 bash[97097]: 420a5b6fd05aa14a956cdf7c574b3127a8e4bb9b9edcf840df2bc183273f5a4b 2023-08-25T08:49:52.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:52 smithi099 systemd[1]: Started Ceph mon.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:49:52.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:52 smithi099 ceph-mon[97137]: mkfs 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:52.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:52 smithi099 ceph-mon[97137]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-25T08:49:53.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout cluster: 2023-08-25T08:49:53.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout id: 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:53.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-08-25T08:49:53.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:49:53.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout services: 2023-08-25T08:49:53.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.678666s) 2023-08-25T08:49:53.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-08-25T08:49:53.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-08-25T08:49:53.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:49:53.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout data: 2023-08-25T08:49:53.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-08-25T08:49:53.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-08-25T08:49:53.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-08-25T08:49:53.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pgs: 2023-08-25T08:49:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:49:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.846+0000 7f57b96b1700 1 Processor -- start 2023-08-25T08:49:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.846+0000 7f57b96b1700 1 -- start start 2023-08-25T08:49:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.846+0000 7f57b96b1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:49:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.846+0000 7f57b96b1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b407a8a0 con 0x7f57b407be70 2023-08-25T08:49:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.847+0000 7f57aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:49:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.847+0000 7f57aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49464/0 (socket says 172.21.15.99:49464) 2023-08-25T08:49:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.847+0000 7f57aaffd700 1 -- 172.21.15.99:0/136676653 learned_addr learned my addr 172.21.15.99:0/136676653 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:49:53.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.847+0000 7f57aaffd700 1 -- 172.21.15.99:0/136676653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b407a9e0 con 0x7f57b407be70 2023-08-25T08:49:53.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.848+0000 7f57aaffd700 1 --2- 172.21.15.99:0/136676653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b407a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57a40098a0 tx=0x7f57a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c89ebc2b38402d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:49:53.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.848+0000 7f57b27fc700 1 -- 172.21.15.99:0/136676653 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a4004160 con 0x7f57b407be70 2023-08-25T08:49:53.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.848+0000 7f57b27fc700 1 -- 172.21.15.99:0/136676653 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a40042c0 con 0x7f57b407be70 2023-08-25T08:49:53.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.848+0000 7f57b27fc700 1 -- 172.21.15.99:0/136676653 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a40045a0 con 0x7f57b407be70 2023-08-25T08:49:53.380 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.848+0000 7f57b96b1700 1 -- 172.21.15.99:0/136676653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 msgr2=0x7f57b407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:49:53.380 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.848+0000 7f57b96b1700 1 --2- 172.21.15.99:0/136676653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b407a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57a40098a0 tx=0x7f57a4009bb0 comp rx=0 tx=0).stop 2023-08-25T08:49:53.380 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.849+0000 7f57b96b1700 1 -- 172.21.15.99:0/136676653 shutdown_connections 2023-08-25T08:49:53.380 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.849+0000 7f57b96b1700 1 --2- 172.21.15.99:0/136676653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b407a2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:49:53.380 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.849+0000 7f57b96b1700 1 -- 172.21.15.99:0/136676653 >> 172.21.15.99:0/136676653 conn(0x7f57b41016e0 msgr2=0x7f57b4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:49:53.380 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.849+0000 7f57b96b1700 1 -- 172.21.15.99:0/136676653 shutdown_connections 2023-08-25T08:49:53.380 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.849+0000 7f57b96b1700 1 -- 172.21.15.99:0/136676653 wait complete. 2023-08-25T08:49:53.381 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.850+0000 7f57b96b1700 1 Processor -- start 2023-08-25T08:49:53.381 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.850+0000 7f57b96b1700 1 -- start start 2023-08-25T08:49:53.381 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.850+0000 7f57b96b1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:49:53.381 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.850+0000 7f57b96b1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419bd60 con 0x7f57b407be70 2023-08-25T08:49:53.381 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.850+0000 7f57aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:49:53.381 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.850+0000 7f57aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b419b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49480/0 (socket says 172.21.15.99:49480) 2023-08-25T08:49:53.381 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.850+0000 7f57aaffd700 1 -- 172.21.15.99:0/967747215 learned_addr learned my addr 172.21.15.99:0/967747215 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:49:53.382 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.851+0000 7f57aaffd700 1 -- 172.21.15.99:0/967747215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a4009550 con 0x7f57b407be70 2023-08-25T08:49:53.382 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.851+0000 7f57aaffd700 1 --2- 172.21.15.99:0/967747215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b419b820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f57b407b340 tx=0x7f57a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:49:53.382 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.851+0000 7f57b0ff9700 1 -- 172.21.15.99:0/967747215 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a4011630 con 0x7f57b407be70 2023-08-25T08:49:53.382 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.851+0000 7f57b0ff9700 1 -- 172.21.15.99:0/967747215 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a4011790 con 0x7f57b407be70 2023-08-25T08:49:53.382 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.851+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b419bf60 con 0x7f57b407be70 2023-08-25T08:49:53.382 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.851+0000 7f57b0ff9700 1 -- 172.21.15.99:0/967747215 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a4011a30 con 0x7f57b407be70 2023-08-25T08:49:53.382 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.851+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b419c380 con 0x7f57b407be70 2023-08-25T08:49:53.382 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.852+0000 7f57b0ff9700 1 -- 172.21.15.99:0/967747215 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f57a4022070 con 0x7f57b407be70 2023-08-25T08:49:53.383 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.852+0000 7f57b0ff9700 1 -- 172.21.15.99:0/967747215 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f57a401c450 con 0x7f57b407be70 2023-08-25T08:49:53.383 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.852+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b4195700 con 0x7f57b407be70 2023-08-25T08:49:53.383 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.856+0000 7f57b0ff9700 1 -- 172.21.15.99:0/967747215 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f57a401b450 con 0x7f57b407be70 2023-08-25T08:49:53.383 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.927+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f57b407ad50 con 0x7f57b407be70 2023-08-25T08:49:53.383 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.927+0000 7f57b0ff9700 1 -- 172.21.15.99:0/967747215 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f57a401b5f0 con 0x7f57b407be70 2023-08-25T08:49:53.383 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.928+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 msgr2=0x7f57b419b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:49:53.383 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.928+0000 7f57b96b1700 1 --2- 172.21.15.99:0/967747215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b419b820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f57b407b340 tx=0x7f57a4004750 comp rx=0 tx=0).stop 2023-08-25T08:49:53.384 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.928+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 shutdown_connections 2023-08-25T08:49:53.384 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.928+0000 7f57b96b1700 1 --2- 172.21.15.99:0/967747215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b407be70 0x7f57b419b820 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:49:53.384 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.928+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 >> 172.21.15.99:0/967747215 conn(0x7f57b41016e0 msgr2=0x7f57b4103020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:49:53.384 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.928+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 shutdown_connections 2023-08-25T08:49:53.384 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:52.928+0000 7f57b96b1700 1 -- 172.21.15.99:0/967747215 wait complete. 2023-08-25T08:49:53.384 INFO:teuthology.orchestra.run.smithi099.stdout:mon is available 2023-08-25T08:49:53.384 INFO:teuthology.orchestra.run.smithi099.stdout:Assimilating anything we can from ceph.conf... 2023-08-25T08:49:53.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:53 smithi099 ceph-mon[97137]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-25T08:49:53.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:53 smithi099 ceph-mon[97137]: monmap e1: 1 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0]} removed_ranks: {} 2023-08-25T08:49:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:53 smithi099 ceph-mon[97137]: fsmap 2023-08-25T08:49:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:53 smithi099 ceph-mon[97137]: osdmap e1: 0 total, 0 up, 0 in 2023-08-25T08:49:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:53 smithi099 ceph-mon[97137]: mgrmap e1: no daemons active 2023-08-25T08:49:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:53 smithi099 ceph-mon[97137]: from='client.? 172.21.15.99:0/967747215' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-08-25T08:49:54.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:54 smithi099 ceph-mon[97137]: from='client.? 172.21.15.99:0/3544794960' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-08-25T08:49:54.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:54 smithi099 ceph-mon[97137]: from='client.? 172.21.15.99:0/3544794960' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-08-25T08:49:55.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:49:55.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2023-08-25T08:49:55.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:55.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.99:3300,v1:172.21.15.99:6789] 2023-08-25T08:49:55.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-08-25T08:49:55.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-08-25T08:49:55.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-08-25T08:49:55.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-08-25T08:49:55.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:49:55.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2023-08-25T08:49:55.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-08-25T08:49:55.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:49:55.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2023-08-25T08:49:55.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-08-25T08:49:55.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-08-25T08:49:55.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-08-25T08:49:55.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-08-25T08:49:55.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-08-25T08:49:55.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.912+0000 7f0077c8f700 1 Processor -- start 2023-08-25T08:49:55.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.912+0000 7f0077c8f700 1 -- start start 2023-08-25T08:49:55.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.913+0000 7f0077c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:49:55.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.913+0000 7f0077c8f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007007a8a0 con 0x7f007007be70 2023-08-25T08:49:55.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.913+0000 7f0075a2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:49:55.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.913+0000 7f0075a2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49490/0 (socket says 172.21.15.99:49490) 2023-08-25T08:49:55.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.913+0000 7f0075a2b700 1 -- 172.21.15.99:0/2989230612 learned_addr learned my addr 172.21.15.99:0/2989230612 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:49:55.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.914+0000 7f0075a2b700 1 -- 172.21.15.99:0/2989230612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007007a9e0 con 0x7f007007be70 2023-08-25T08:49:55.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.914+0000 7f0075a2b700 1 --2- 172.21.15.99:0/2989230612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007007a2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f006c0098a0 tx=0x7f006c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e91f86e086f337 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:49:55.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.914+0000 7f0074a29700 1 -- 172.21.15.99:0/2989230612 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f006c004160 con 0x7f007007be70 2023-08-25T08:49:55.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.914+0000 7f0074a29700 1 -- 172.21.15.99:0/2989230612 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c0042c0 con 0x7f007007be70 2023-08-25T08:49:55.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.914+0000 7f0074a29700 1 -- 172.21.15.99:0/2989230612 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f006c0045a0 con 0x7f007007be70 2023-08-25T08:49:55.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.915+0000 7f0077c8f700 1 -- 172.21.15.99:0/2989230612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 msgr2=0x7f007007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:49:55.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.915+0000 7f0077c8f700 1 --2- 172.21.15.99:0/2989230612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007007a2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f006c0098a0 tx=0x7f006c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:49:55.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.915+0000 7f0077c8f700 1 -- 172.21.15.99:0/2989230612 shutdown_connections 2023-08-25T08:49:55.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.915+0000 7f0077c8f700 1 --2- 172.21.15.99:0/2989230612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007007a2d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:49:55.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.915+0000 7f0077c8f700 1 -- 172.21.15.99:0/2989230612 >> 172.21.15.99:0/2989230612 conn(0x7f00701016e0 msgr2=0x7f0070103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:49:55.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.915+0000 7f0077c8f700 1 -- 172.21.15.99:0/2989230612 shutdown_connections 2023-08-25T08:49:55.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.915+0000 7f0077c8f700 1 -- 172.21.15.99:0/2989230612 wait complete. 2023-08-25T08:49:55.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.916+0000 7f0077c8f700 1 Processor -- start 2023-08-25T08:49:55.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.916+0000 7f0077c8f700 1 -- start start 2023-08-25T08:49:55.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.916+0000 7f0077c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:49:55.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.916+0000 7f0077c8f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007019bd60 con 0x7f007007be70 2023-08-25T08:49:55.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.917+0000 7f0075a2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:49:55.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.917+0000 7f0075a2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007019b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49498/0 (socket says 172.21.15.99:49498) 2023-08-25T08:49:55.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.917+0000 7f0075a2b700 1 -- 172.21.15.99:0/3544794960 learned_addr learned my addr 172.21.15.99:0/3544794960 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:49:55.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.917+0000 7f0075a2b700 1 -- 172.21.15.99:0/3544794960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c009550 con 0x7f007007be70 2023-08-25T08:49:55.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.917+0000 7f0075a2b700 1 --2- 172.21.15.99:0/3544794960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007019b820 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f007007b340 tx=0x7f006c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:49:55.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.917+0000 7f0066ffd700 1 -- 172.21.15.99:0/3544794960 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f006c011630 con 0x7f007007be70 2023-08-25T08:49:55.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.918+0000 7f0066ffd700 1 -- 172.21.15.99:0/3544794960 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c011790 con 0x7f007007be70 2023-08-25T08:49:55.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.918+0000 7f0066ffd700 1 -- 172.21.15.99:0/3544794960 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f006c011a30 con 0x7f007007be70 2023-08-25T08:49:55.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.918+0000 7f0077c8f700 1 -- 172.21.15.99:0/3544794960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007019bf60 con 0x7f007007be70 2023-08-25T08:49:55.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.918+0000 7f0077c8f700 1 -- 172.21.15.99:0/3544794960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007019c380 con 0x7f007007be70 2023-08-25T08:49:55.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.918+0000 7f0066ffd700 1 -- 172.21.15.99:0/3544794960 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f006c022070 con 0x7f007007be70 2023-08-25T08:49:55.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.918+0000 7f0066ffd700 1 -- 172.21.15.99:0/3544794960 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f006c01c790 con 0x7f007007be70 2023-08-25T08:49:55.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.919+0000 7f0064ff9700 1 -- 172.21.15.99:0/3544794960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00580052f0 con 0x7f007007be70 2023-08-25T08:49:55.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:53.921+0000 7f0066ffd700 1 -- 172.21.15.99:0/3544794960 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f006c01bcf0 con 0x7f007007be70 2023-08-25T08:49:55.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.001+0000 7f0064ff9700 1 -- 172.21.15.99:0/3544794960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0058002400 con 0x7f007007be70 2023-08-25T08:49:55.765 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.005+0000 7f0066ffd700 1 -- 172.21.15.99:0/3544794960 <== mon.0 v2:172.21.15.99:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f006c01be90 con 0x7f007007be70 2023-08-25T08:49:55.765 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.005+0000 7f0066ffd700 1 -- 172.21.15.99:0/3544794960 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f006c037610 con 0x7f007007be70 2023-08-25T08:49:55.765 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.006+0000 7f0064ff9700 1 -- 172.21.15.99:0/3544794960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 msgr2=0x7f007019b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:49:55.765 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.006+0000 7f0064ff9700 1 --2- 172.21.15.99:0/3544794960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007019b820 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f007007b340 tx=0x7f006c004750 comp rx=0 tx=0).stop 2023-08-25T08:49:55.765 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.006+0000 7f0064ff9700 1 -- 172.21.15.99:0/3544794960 shutdown_connections 2023-08-25T08:49:55.765 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.006+0000 7f0064ff9700 1 --2- 172.21.15.99:0/3544794960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007007be70 0x7f007019b820 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:49:55.765 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.006+0000 7f0064ff9700 1 -- 172.21.15.99:0/3544794960 >> 172.21.15.99:0/3544794960 conn(0x7f00701016e0 msgr2=0x7f0070103020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:49:55.766 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.006+0000 7f0064ff9700 1 -- 172.21.15.99:0/3544794960 shutdown_connections 2023-08-25T08:49:55.766 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:54.006+0000 7f0064ff9700 1 -- 172.21.15.99:0/3544794960 wait complete. 2023-08-25T08:49:55.766 INFO:teuthology.orchestra.run.smithi099.stdout:Generating new minimal ceph.conf... 2023-08-25T08:49:56.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:56 smithi099 ceph-mon[97137]: from='client.? 172.21.15.99:0/156524950' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:49:56.935 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.293+0000 7f620a6cd700 1 Processor -- start 2023-08-25T08:49:56.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.294+0000 7f620a6cd700 1 -- start start 2023-08-25T08:49:56.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.294+0000 7f620a6cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:49:56.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.294+0000 7f620a6cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620407a8e0 con 0x7f620407beb0 2023-08-25T08:49:56.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.294+0000 7f6203fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:49:56.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.294+0000 7f6203fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49508/0 (socket says 172.21.15.99:49508) 2023-08-25T08:49:56.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.294+0000 7f6203fff700 1 -- 172.21.15.99:0/2080888970 learned_addr learned my addr 172.21.15.99:0/2080888970 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:49:56.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.295+0000 7f6203fff700 1 -- 172.21.15.99:0/2080888970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620407aa20 con 0x7f620407beb0 2023-08-25T08:49:56.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.295+0000 7f6203fff700 1 --2- 172.21.15.99:0/2080888970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620407a310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61f40098a0 tx=0x7f61f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab0bf7a6914cf01 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:49:56.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.295+0000 7f6202ffd700 1 -- 172.21.15.99:0/2080888970 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61f4004160 con 0x7f620407beb0 2023-08-25T08:49:56.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.295+0000 7f6202ffd700 1 -- 172.21.15.99:0/2080888970 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61f40042c0 con 0x7f620407beb0 2023-08-25T08:49:56.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.295+0000 7f6202ffd700 1 -- 172.21.15.99:0/2080888970 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61f4011a00 con 0x7f620407beb0 2023-08-25T08:49:56.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.296+0000 7f620a6cd700 1 -- 172.21.15.99:0/2080888970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 msgr2=0x7f620407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:49:56.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.296+0000 7f620a6cd700 1 --2- 172.21.15.99:0/2080888970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620407a310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61f40098a0 tx=0x7f61f4009bb0 comp rx=0 tx=0).stop 2023-08-25T08:49:56.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.296+0000 7f620a6cd700 1 -- 172.21.15.99:0/2080888970 shutdown_connections 2023-08-25T08:49:56.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.296+0000 7f620a6cd700 1 --2- 172.21.15.99:0/2080888970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620407a310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:49:56.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.296+0000 7f620a6cd700 1 -- 172.21.15.99:0/2080888970 >> 172.21.15.99:0/2080888970 conn(0x7f62041016e0 msgr2=0x7f6204103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:49:56.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.296+0000 7f620a6cd700 1 -- 172.21.15.99:0/2080888970 shutdown_connections 2023-08-25T08:49:56.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.296+0000 7f620a6cd700 1 -- 172.21.15.99:0/2080888970 wait complete. 2023-08-25T08:49:56.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.297+0000 7f620a6cd700 1 Processor -- start 2023-08-25T08:49:56.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.298+0000 7f620a6cd700 1 -- start start 2023-08-25T08:49:56.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.298+0000 7f620a6cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:49:56.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.298+0000 7f620a6cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620419c5d0 con 0x7f620407beb0 2023-08-25T08:49:56.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.298+0000 7f6203fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:49:56.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f6203fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620419c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49518/0 (socket says 172.21.15.99:49518) 2023-08-25T08:49:56.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f6203fff700 1 -- 172.21.15.99:0/156524950 learned_addr learned my addr 172.21.15.99:0/156524950 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:49:56.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f6203fff700 1 -- 172.21.15.99:0/156524950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f4009550 con 0x7f620407beb0 2023-08-25T08:49:56.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f6203fff700 1 --2- 172.21.15.99:0/156524950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620419c090 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009fb0 tx=0x7f61f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:49:56.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f62017fa700 1 -- 172.21.15.99:0/156524950 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61f4003a40 con 0x7f620407beb0 2023-08-25T08:49:56.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f62017fa700 1 -- 172.21.15.99:0/156524950 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61f4003ba0 con 0x7f620407beb0 2023-08-25T08:49:56.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620419c7d0 con 0x7f620407beb0 2023-08-25T08:49:56.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f62017fa700 1 -- 172.21.15.99:0/156524950 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61f401a750 con 0x7f620407beb0 2023-08-25T08:49:56.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.299+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620419cbf0 con 0x7f620407beb0 2023-08-25T08:49:56.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.300+0000 7f62017fa700 1 -- 172.21.15.99:0/156524950 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f61f401ac80 con 0x7f620407beb0 2023-08-25T08:49:56.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.300+0000 7f62017fa700 1 -- 172.21.15.99:0/156524950 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f61f40193c0 con 0x7f620407beb0 2023-08-25T08:49:56.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.301+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204196100 con 0x7f620407beb0 2023-08-25T08:49:56.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.303+0000 7f62017fa700 1 -- 172.21.15.99:0/156524950 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f61f402de30 con 0x7f620407beb0 2023-08-25T08:49:56.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.373+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6204061f50 con 0x7f620407beb0 2023-08-25T08:49:56.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.373+0000 7f62017fa700 1 -- 172.21.15.99:0/156524950 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f61f4024b60 con 0x7f620407beb0 2023-08-25T08:49:56.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.374+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 msgr2=0x7f620419c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:49:56.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.374+0000 7f620a6cd700 1 --2- 172.21.15.99:0/156524950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620419c090 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009fb0 tx=0x7f61f4011ad0 comp rx=0 tx=0).stop 2023-08-25T08:49:56.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.375+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 shutdown_connections 2023-08-25T08:49:56.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.375+0000 7f620a6cd700 1 --2- 172.21.15.99:0/156524950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f620407beb0 0x7f620419c090 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:49:56.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.375+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 >> 172.21.15.99:0/156524950 conn(0x7f62041016e0 msgr2=0x7f6204102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:49:56.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.375+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 shutdown_connections 2023-08-25T08:49:56.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:56.375+0000 7f620a6cd700 1 -- 172.21.15.99:0/156524950 wait complete. 2023-08-25T08:49:56.943 INFO:teuthology.orchestra.run.smithi099.stdout:Restarting the monitor... 2023-08-25T08:49:57.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:56 smithi099 systemd[1]: Stopping Ceph mon.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:49:57.651 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:57 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-a[97113]: 2023-08-25T08:49:57.374+0000 7ff3d3def700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T08:49:57.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:57 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-a[97113]: 2023-08-25T08:49:57.374+0000 7ff3d3def700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-08-25T08:49:57.934 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:57 smithi099 bash[97474]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-a 2023-08-25T08:49:58.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 systemd[1]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.a.service: Succeeded. 2023-08-25T08:49:58.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 systemd[1]: Stopped Ceph mon.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:49:58.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 systemd[1]: Starting Ceph mon.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:49:58.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 podman[97600]: 2023-08-25T08:49:58.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: set uid:gid to 167:167 (ceph:ceph) 2023-08-25T08:49:58.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev), process ceph-mon, pid 7 2023-08-25T08:49:58.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: pidfile_write: ignore empty --pid-file 2023-08-25T08:49:58.966 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: load: jerasure load: lrc 2023-08-25T08:49:58.966 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: RocksDB version: 7.9.2 2023-08-25T08:49:58.966 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Git sha 0 2023-08-25T08:49:58.966 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Compile date 2023-08-24 22:08:31 2023-08-25T08:49:58.966 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: DB SUMMARY 2023-08-25T08:49:58.966 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: DB Session ID: QHHPRSWIXRB4KZDN01ZS 2023-08-25T08:49:58.966 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: CURRENT file: CURRENT 2023-08-25T08:49:58.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: IDENTITY file: IDENTITY 2023-08-25T08:49:58.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-08-25T08:49:58.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-08-25T08:49:58.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99274 ; 2023-08-25T08:49:58.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.error_if_exists: 0 2023-08-25T08:49:58.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.create_if_missing: 0 2023-08-25T08:49:58.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.paranoid_checks: 1 2023-08-25T08:49:58.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-25T08:49:58.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-25T08:49:58.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-25T08:49:58.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.env: 0x563a220dada0 2023-08-25T08:49:58.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.fs: PosixFileSystem 2023-08-25T08:49:58.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.info_log: 0x563a23b27440 2023-08-25T08:49:58.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_file_opening_threads: 16 2023-08-25T08:49:58.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.statistics: (nil) 2023-08-25T08:49:58.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.use_fsync: 0 2023-08-25T08:49:58.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_log_file_size: 0 2023-08-25T08:49:58.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-25T08:49:58.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-25T08:49:58.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.keep_log_file_num: 1000 2023-08-25T08:49:58.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.recycle_log_file_num: 0 2023-08-25T08:49:58.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.allow_fallocate: 1 2023-08-25T08:49:58.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.allow_mmap_reads: 0 2023-08-25T08:49:58.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.allow_mmap_writes: 0 2023-08-25T08:49:58.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.use_direct_reads: 0 2023-08-25T08:49:58.971 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-25T08:49:58.971 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.create_missing_column_families: 0 2023-08-25T08:49:58.971 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.db_log_dir: 2023-08-25T08:49:58.971 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.wal_dir: 2023-08-25T08:49:58.971 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-25T08:49:58.971 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-25T08:49:58.973 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-25T08:49:58.973 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-25T08:49:58.973 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-25T08:49:58.973 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-25T08:49:58.973 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.advise_random_on_open: 1 2023-08-25T08:49:58.973 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.db_write_buffer_size: 0 2023-08-25T08:49:58.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.write_buffer_manager: 0x563a23ae14a0 2023-08-25T08:49:58.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-25T08:49:58.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-25T08:49:58.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-25T08:49:58.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.rate_limiter: (nil) 2023-08-25T08:49:58.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-25T08:49:58.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.wal_recovery_mode: 2 2023-08-25T08:49:58.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.enable_thread_tracking: 0 2023-08-25T08:49:58.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.enable_pipelined_write: 0 2023-08-25T08:49:58.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.unordered_write: 0 2023-08-25T08:49:58.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-25T08:49:58.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-25T08:49:58.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-25T08:49:58.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-25T08:49:58.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.row_cache: None 2023-08-25T08:49:58.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.wal_filter: None 2023-08-25T08:49:58.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-25T08:49:58.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.allow_ingest_behind: 0 2023-08-25T08:49:58.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.two_write_queues: 0 2023-08-25T08:49:58.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.manual_wal_flush: 0 2023-08-25T08:49:58.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.wal_compression: 0 2023-08-25T08:49:58.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.atomic_flush: 0 2023-08-25T08:49:58.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-25T08:49:58.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-25T08:49:58.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-25T08:49:58.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.log_readahead_size: 0 2023-08-25T08:49:58.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-25T08:49:58.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.best_efforts_recovery: 0 2023-08-25T08:49:58.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-25T08:49:58.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-25T08:49:58.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.allow_data_in_errors: 0 2023-08-25T08:49:58.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.db_host_id: __hostname__ 2023-08-25T08:49:58.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-25T08:49:58.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_background_jobs: 2 2023-08-25T08:49:58.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_background_compactions: -1 2023-08-25T08:49:58.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_subcompactions: 1 2023-08-25T08:49:58.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-25T08:49:58.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-25T08:49:58.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-25T08:49:58.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_total_wal_size: 0 2023-08-25T08:49:58.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-25T08:49:58.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-25T08:49:58.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-25T08:49:58.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-25T08:49:58.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_open_files: -1 2023-08-25T08:49:58.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bytes_per_sync: 0 2023-08-25T08:49:58.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-25T08:49:58.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-25T08:49:58.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_readahead_size: 0 2023-08-25T08:49:58.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_background_flushes: -1 2023-08-25T08:49:58.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Compression algorithms supported: 2023-08-25T08:49:58.981 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-25T08:49:58.981 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: kZSTD supported: 0 2023-08-25T08:49:58.981 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: kXpressCompression supported: 0 2023-08-25T08:49:58.981 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: kLZ4HCCompression supported: 1 2023-08-25T08:49:58.981 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: kZlibCompression supported: 1 2023-08-25T08:49:58.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: kSnappyCompression supported: 1 2023-08-25T08:49:58.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: kLZ4Compression supported: 1 2023-08-25T08:49:58.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: kBZip2Compression supported: 0 2023-08-25T08:49:58.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-25T08:49:58.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-25T08:49:58.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-08-25T08:49:58.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-25T08:49:58.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-25T08:49:58.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.merge_operator: 2023-08-25T08:49:58.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_filter: None 2023-08-25T08:49:58.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_filter_factory: None 2023-08-25T08:49:58.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.sst_partitioner_factory: None 2023-08-25T08:49:58.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-25T08:49:58.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-25T08:49:58.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563a23b27540) 2023-08-25T08:49:58.984 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks: 1 2023-08-25T08:49:58.984 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-25T08:49:58.984 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-25T08:49:58.984 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_top_level_index_and_filter: 1 2023-08-25T08:49:58.984 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_type: 0 2023-08-25T08:49:58.984 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_index_type: 0 2023-08-25T08:49:58.984 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_shortening: 1 2023-08-25T08:49:58.985 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-25T08:49:58.985 INFO:journalctl@ceph.mon.a.smithi099.stdout: checksum: 4 2023-08-25T08:49:58.985 INFO:journalctl@ceph.mon.a.smithi099.stdout: no_block_cache: 0 2023-08-25T08:49:58.985 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache: 0x563a23b7f1f0 2023-08-25T08:49:58.985 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_name: BinnedLRUCache 2023-08-25T08:49:58.985 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_options: 2023-08-25T08:49:58.985 INFO:journalctl@ceph.mon.a.smithi099.stdout: capacity : 536870912 2023-08-25T08:49:58.985 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_shard_bits : 4 2023-08-25T08:49:58.986 INFO:journalctl@ceph.mon.a.smithi099.stdout: strict_capacity_limit : 0 2023-08-25T08:49:58.986 INFO:journalctl@ceph.mon.a.smithi099.stdout: high_pri_pool_ratio: 0.000 2023-08-25T08:49:58.986 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_compressed: (nil) 2023-08-25T08:49:58.986 INFO:journalctl@ceph.mon.a.smithi099.stdout: persistent_cache: (nil) 2023-08-25T08:49:58.986 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size: 4096 2023-08-25T08:49:58.986 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size_deviation: 10 2023-08-25T08:49:58.986 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_restart_interval: 16 2023-08-25T08:49:58.986 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_block_restart_interval: 1 2023-08-25T08:49:58.987 INFO:journalctl@ceph.mon.a.smithi099.stdout: metadata_block_size: 4096 2023-08-25T08:49:58.987 INFO:journalctl@ceph.mon.a.smithi099.stdout: partition_filters: 0 2023-08-25T08:49:58.987 INFO:journalctl@ceph.mon.a.smithi099.stdout: use_delta_encoding: 1 2023-08-25T08:49:58.987 INFO:journalctl@ceph.mon.a.smithi099.stdout: filter_policy: bloomfilter 2023-08-25T08:49:58.987 INFO:journalctl@ceph.mon.a.smithi099.stdout: whole_key_filtering: 1 2023-08-25T08:49:58.987 INFO:journalctl@ceph.mon.a.smithi099.stdout: verify_compression: 0 2023-08-25T08:49:58.987 INFO:journalctl@ceph.mon.a.smithi099.stdout: read_amp_bytes_per_bit: 0 2023-08-25T08:49:58.987 INFO:journalctl@ceph.mon.a.smithi099.stdout: format_version: 5 2023-08-25T08:49:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_index_compression: 1 2023-08-25T08:49:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_align: 0 2023-08-25T08:49:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout: max_auto_readahead_size: 262144 2023-08-25T08:49:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout: prepopulate_block_cache: 0 2023-08-25T08:49:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout: initial_auto_readahead_size: 8192 2023-08-25T08:49:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_file_reads_for_auto_readahead: 2 2023-08-25T08:49:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.write_buffer_size: 33554432 2023-08-25T08:49:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_write_buffer_number: 2 2023-08-25T08:49:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression: NoCompression 2023-08-25T08:49:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression: Disabled 2023-08-25T08:49:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.prefix_extractor: nullptr 2023-08-25T08:49:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-25T08:49:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.num_levels: 7 2023-08-25T08:49:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-25T08:49:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-25T08:49:58.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-25T08:49:58.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-25T08:49:58.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-25T08:49:58.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-25T08:49:58.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-25T08:49:58.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-25T08:49:58.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-25T08:49:58.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-25T08:49:58.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-25T08:49:58.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-25T08:49:58.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-25T08:49:58.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.level: 32767 2023-08-25T08:49:58.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.strategy: 0 2023-08-25T08:49:58.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-25T08:49:58.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-25T08:49:58.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-25T08:49:58.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-25T08:49:58.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.enabled: false 2023-08-25T08:49:58.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-25T08:49:58.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-25T08:49:58.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-25T08:49:58.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-25T08:49:58.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.target_file_size_base: 67108864 2023-08-25T08:49:58.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-25T08:49:58.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-25T08:49:58.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-25T08:49:58.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-25T08:49:58.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-25T08:49:58.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-25T08:49:58.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-25T08:49:58.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-25T08:49:58.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-25T08:49:58.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-25T08:49:58.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-25T08:49:58.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-25T08:49:58.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-25T08:49:58.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-25T08:49:58.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.arena_block_size: 1048576 2023-08-25T08:49:58.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-25T08:49:58.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-25T08:49:58.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.disable_auto_compactions: 0 2023-08-25T08:49:58.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-25T08:49:58.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-25T08:49:58.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-25T08:49:58.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-25T08:49:58.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-25T08:49:58.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-25T08:49:58.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-25T08:49:58.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-25T08:49:58.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-25T08:49:58.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-25T08:49:58.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.table_properties_collectors: 2023-08-25T08:49:58.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.inplace_update_support: 0 2023-08-25T08:49:58.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-25T08:49:58.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-25T08:49:58.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-25T08:49:58.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-25T08:49:58.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.bloom_locality: 0 2023-08-25T08:49:58.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.max_successive_merges: 0 2023-08-25T08:49:58.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-25T08:49:58.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.paranoid_file_checks: 0 2023-08-25T08:49:58.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.force_consistency_checks: 1 2023-08-25T08:49:58.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.report_bg_io_stats: 0 2023-08-25T08:49:58.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.ttl: 2592000 2023-08-25T08:49:58.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-25T08:49:58.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-25T08:49:58.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-25T08:49:58.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.enable_blob_files: false 2023-08-25T08:49:58.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.min_blob_size: 0 2023-08-25T08:49:58.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.blob_file_size: 268435456 2023-08-25T08:49:58.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-25T08:49:58.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-25T08:49:58.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-25T08:49:59.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-25T08:49:59.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-25T08:49:59.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.blob_file_starting_level: 0 2023-08-25T08:49:59.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-25T08:49:59.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-08-25T08:49:59.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-08-25T08:49:59.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6afd8706-c210-476d-a494-b96009f5e83f 2023-08-25T08:49:59.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953398897883, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-08-25T08:49:59.001 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-08-25T08:49:59.001 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953398899844, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94271, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 323, "table_properties": {"data_size": 92276, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13656, "raw_average_key_size": 49, "raw_value_size": 84828, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 274, "num_filter_entries": 274, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1692953398, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6afd8706-c210-476d-a494-b96009f5e83f", "db_session_id": "QHHPRSWIXRB4KZDN01ZS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-08-25T08:49:59.001 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953398899978, "job": 1, "event": "recovery_finished"} 2023-08-25T08:49:59.001 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-08-25T08:49:59.001 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-25T08:49:59.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563a23c1c000 2023-08-25T08:49:59.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: DB pointer 0x563a23c08000 2023-08-25T08:49:59.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-25T08:49:59.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-25T08:49:59.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** DB Stats ** 2023-08-25T08:49:59.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T08:49:59.003 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-25T08:49:59.003 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T08:49:59.003 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T08:49:59.003 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-25T08:49:59.003 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T08:49:59.003 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T08:49:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2023-08-25T08:49:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2023-08-25T08:49:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T08:49:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-25T08:49:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout: L0 2/0 93.90 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-25T08:49:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout: Sum 2/0 93.90 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-25T08:49:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-25T08:49:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2023-08-25T08:49:59.005 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2023-08-25T08:49:59.005 INFO:journalctl@ceph.mon.a.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T08:49:59.005 INFO:journalctl@ceph.mon.a.smithi099.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-25T08:49:59.005 INFO:journalctl@ceph.mon.a.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 66.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-25T08:49:59.005 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2023-08-25T08:49:59.005 INFO:journalctl@ceph.mon.a.smithi099.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-25T08:49:59.005 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2023-08-25T08:49:59.006 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T08:49:59.006 INFO:journalctl@ceph.mon.a.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-25T08:49:59.006 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-25T08:49:59.006 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-25T08:49:59.006 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-25T08:49:59.006 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-25T08:49:59.006 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative compaction: 0.00 GB write, 14.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T08:49:59.006 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval compaction: 0.00 GB write, 14.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T08:49:59.007 INFO:journalctl@ceph.mon.a.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-25T08:49:59.007 INFO:journalctl@ceph.mon.a.smithi099.stdout: Block cache BinnedLRUCache@0x563a23b7f1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-08-25T08:49:59.007 INFO:journalctl@ceph.mon.a.smithi099.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-08-25T08:49:59.007 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2023-08-25T08:49:59.007 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-25T08:49:59.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: starting mon.a rank 0 at public addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] at bind addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:59.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a@-1(???) e1 preinit fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:49:59.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a@-1(???).mds e1 new map 2023-08-25T08:49:59.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a@-1(???).mds e1 print_map 2023-08-25T08:49:59.008 INFO:journalctl@ceph.mon.a.smithi099.stdout: e1 2023-08-25T08:49:59.008 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-25T08:49:59.008 INFO:journalctl@ceph.mon.a.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T08:49:59.008 INFO:journalctl@ceph.mon.a.smithi099.stdout: legacy client fscid: -1 2023-08-25T08:49:59.008 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2023-08-25T08:49:59.008 INFO:journalctl@ceph.mon.a.smithi099.stdout: No filesystems configured 2023-08-25T08:49:59.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-25T08:49:59.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:49:59.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:49:59.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:49:59.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-08-25T08:49:59.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta expand map: {default=false} 2023-08-25T08:49:59.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta from 'false' to 'false' 2023-08-25T08:49:59.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta expanded map: {default=false} 2023-08-25T08:49:59.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta expand map: {default=info} 2023-08-25T08:49:59.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta from 'info' to 'info' 2023-08-25T08:49:59.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta expanded map: {default=info} 2023-08-25T08:49:59.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta expand map: {default=daemon} 2023-08-25T08:49:59.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-25T08:49:59.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta expanded map: {default=daemon} 2023-08-25T08:49:59.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta expand map: {default=debug} 2023-08-25T08:49:59.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta from 'debug' to 'debug' 2023-08-25T08:49:59.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: expand_channel_meta expanded map: {default=debug} 2023-08-25T08:49:59.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 bash[97600]: eeef5ed71b057088b802b6509a275f2e98c9180bcf012a07365a62bd711524ed 2023-08-25T08:49:59.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-25T08:49:59.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: monmap e1: 1 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0]} removed_ranks: {} 2023-08-25T08:49:59.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: fsmap 2023-08-25T08:49:59.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: osdmap e1: 0 total, 0 up, 0 in 2023-08-25T08:49:59.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 ceph-mon[97641]: mgrmap e1: no daemons active 2023-08-25T08:49:59.012 INFO:teuthology.orchestra.run.smithi099.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-08-25T08:49:59.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:49:58 smithi099 systemd[1]: Started Ceph mon.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:49:59.968 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.481+0000 7fd4e9113700 1 Processor -- start 2023-08-25T08:49:59.969 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.481+0000 7fd4e9113700 1 -- start start 2023-08-25T08:49:59.969 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.482+0000 7fd4e9113700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:49:59.969 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.482+0000 7fd4e9113700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e407a8a0 con 0x7fd4e407be70 2023-08-25T08:49:59.970 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.482+0000 7fd4e2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:49:59.970 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.482+0000 7fd4e2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35750/0 (socket says 172.21.15.99:35750) 2023-08-25T08:49:59.970 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.482+0000 7fd4e2d9d700 1 -- 172.21.15.99:0/1476995276 learned_addr learned my addr 172.21.15.99:0/1476995276 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:49:59.971 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.483+0000 7fd4e2d9d700 1 -- 172.21.15.99:0/1476995276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e407a9e0 con 0x7fd4e407be70 2023-08-25T08:49:59.971 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.483+0000 7fd4e2d9d700 1 --2- 172.21.15.99:0/1476995276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e407a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40098a0 tx=0x7fd4d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91593c752dbdb974 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:49:59.971 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.483+0000 7fd4e1d9b700 1 -- 172.21.15.99:0/1476995276 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d4004160 con 0x7fd4e407be70 2023-08-25T08:49:59.971 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.483+0000 7fd4e1d9b700 1 -- 172.21.15.99:0/1476995276 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4d40042c0 con 0x7fd4e407be70 2023-08-25T08:49:59.972 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.483+0000 7fd4e1d9b700 1 -- 172.21.15.99:0/1476995276 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d4011a00 con 0x7fd4e407be70 2023-08-25T08:49:59.972 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.484+0000 7fd4e9113700 1 -- 172.21.15.99:0/1476995276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 msgr2=0x7fd4e407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:49:59.972 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.484+0000 7fd4e9113700 1 --2- 172.21.15.99:0/1476995276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e407a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40098a0 tx=0x7fd4d4009bb0 comp rx=0 tx=0).stop 2023-08-25T08:49:59.973 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.484+0000 7fd4e9113700 1 -- 172.21.15.99:0/1476995276 shutdown_connections 2023-08-25T08:49:59.973 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.484+0000 7fd4e9113700 1 --2- 172.21.15.99:0/1476995276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e407a2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:49:59.973 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.484+0000 7fd4e9113700 1 -- 172.21.15.99:0/1476995276 >> 172.21.15.99:0/1476995276 conn(0x7fd4e41016e0 msgr2=0x7fd4e4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:49:59.974 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.484+0000 7fd4e9113700 1 -- 172.21.15.99:0/1476995276 shutdown_connections 2023-08-25T08:49:59.974 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.484+0000 7fd4e9113700 1 -- 172.21.15.99:0/1476995276 wait complete. 2023-08-25T08:49:59.974 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.485+0000 7fd4e9113700 1 Processor -- start 2023-08-25T08:49:59.974 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.485+0000 7fd4e9113700 1 -- start start 2023-08-25T08:49:59.975 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.485+0000 7fd4e9113700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e419c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:49:59.975 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.485+0000 7fd4e9113700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e419c5b0 con 0x7fd4e407be70 2023-08-25T08:49:59.975 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.486+0000 7fd4e2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e419c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:49:59.976 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.486+0000 7fd4e2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e419c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35766/0 (socket says 172.21.15.99:35766) 2023-08-25T08:49:59.976 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.486+0000 7fd4e2d9d700 1 -- 172.21.15.99:0/3917112225 learned_addr learned my addr 172.21.15.99:0/3917112225 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:49:59.976 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.486+0000 7fd4e2d9d700 1 -- 172.21.15.99:0/3917112225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d4009550 con 0x7fd4e407be70 2023-08-25T08:49:59.976 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.486+0000 7fd4e2d9d700 1 --2- 172.21.15.99:0/3917112225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e419c070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009130 tx=0x7fd4d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:49:59.977 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.487+0000 7fd4cbfff700 1 -- 172.21.15.99:0/3917112225 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d4003b70 con 0x7fd4e407be70 2023-08-25T08:49:59.977 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.487+0000 7fd4cbfff700 1 -- 172.21.15.99:0/3917112225 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4d4003cd0 con 0x7fd4e407be70 2023-08-25T08:49:59.977 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.487+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e419c7b0 con 0x7fd4e407be70 2023-08-25T08:49:59.978 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.487+0000 7fd4cbfff700 1 -- 172.21.15.99:0/3917112225 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d40197d0 con 0x7fd4e407be70 2023-08-25T08:49:59.978 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.487+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e419cbd0 con 0x7fd4e407be70 2023-08-25T08:49:59.978 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.487+0000 7fd4cbfff700 1 -- 172.21.15.99:0/3917112225 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd4d4019a30 con 0x7fd4e407be70 2023-08-25T08:49:59.978 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.487+0000 7fd4cbfff700 1 -- 172.21.15.99:0/3917112225 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd4d400b720 con 0x7fd4e407be70 2023-08-25T08:49:59.979 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.488+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e4061f50 con 0x7fd4e407be70 2023-08-25T08:49:59.979 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.491+0000 7fd4cbfff700 1 -- 172.21.15.99:0/3917112225 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd4d402a060 con 0x7fd4e407be70 2023-08-25T08:49:59.979 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.559+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fd4e410c510 con 0x7fd4e407be70 2023-08-25T08:49:59.980 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.562+0000 7fd4cbfff700 1 -- 172.21.15.99:0/3917112225 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd4d401abb0 con 0x7fd4e407be70 2023-08-25T08:49:59.980 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.563+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 msgr2=0x7fd4e419c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:49:59.980 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.563+0000 7fd4e9113700 1 --2- 172.21.15.99:0/3917112225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e419c070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009130 tx=0x7fd4d4011ad0 comp rx=0 tx=0).stop 2023-08-25T08:49:59.981 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.563+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 shutdown_connections 2023-08-25T08:49:59.981 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.563+0000 7fd4e9113700 1 --2- 172.21.15.99:0/3917112225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4e407be70 0x7fd4e419c070 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:49:59.981 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.563+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 >> 172.21.15.99:0/3917112225 conn(0x7fd4e41016e0 msgr2=0x7fd4e4102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:49:59.981 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.563+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 shutdown_connections 2023-08-25T08:49:59.982 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:49:59.563+0000 7fd4e9113700 1 -- 172.21.15.99:0/3917112225 wait complete. 2023-08-25T08:49:59.982 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote config to /etc/ceph/ceph.conf 2023-08-25T08:49:59.986 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-08-25T08:49:59.986 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mgr... 2023-08-25T08:49:59.987 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:9283 ... 2023-08-25T08:49:59.987 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:8765 ... 2023-08-25T08:50:00.362 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.x 2023-08-25T08:50:00.362 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.x.service: Unit ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.x.service not loaded. 2023-08-25T08:50:00.559 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c.target.wants/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@.service. 2023-08-25T08:50:00.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:00 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3917112225' entity='client.admin' 2023-08-25T08:50:00.813 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:00 smithi099 systemd[1]: Starting Ceph mgr.x for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:50:01.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:01 smithi099 podman[97872]: 2023-08-25T08:50:01.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:01 smithi099 bash[97872]: 50bd29f18fb27f7ec390b8ededac6afec50040e779ee474f5f34f185cc57500b 2023-08-25T08:50:01.364 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-25T08:50:01.365 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2023-08-25T08:50:01.381 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-25T08:50:01.382 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2023-08-25T08:50:01.382 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2023-08-25T08:50:01.382 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2023-08-25T08:50:01.398 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-25T08:50:01.399 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2023-08-25T08:50:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-25T08:50:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2023-08-25T08:50:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2023-08-25T08:50:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-08-25T08:50:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr to start... 2023-08-25T08:50:01.420 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr... 2023-08-25T08:50:01.682 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:01 smithi099 systemd[1]: Started Ceph mgr.x for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:50:01.683 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:01.456+0000 7f9a13d7d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T08:50:01.683 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:01.565+0000 7f9a13d7d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T08:50:01.683 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:01.681+0000 7f9a13d7d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T08:50:02.044 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:01.809+0000 7f9a13d7d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T08:50:02.044 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:02.043+0000 7f9a13d7d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T08:50:02.300 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:02 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1303050804' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-25T08:50:02.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:50:02.597 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2023-08-25T08:50:02.597 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:50:02.598 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2023-08-25T08:50:02.598 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-25T08:50:02.598 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-25T08:50:02.599 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-25T08:50:02.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:02.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-25T08:50:02.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-25T08:50:02.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2023-08-25T08:50:02.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:02.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-25T08:50:02.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2023-08-25T08:50:02.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:02.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-08-25T08:50:02.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-25T08:50:02.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:02.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-25T08:50:02.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-25T08:50:02.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:02.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-25T08:50:02.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:02.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-25T08:50:02.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-25T08:50:02.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-25T08:50:02.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-25T08:50:02.609 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-25T08:50:02.609 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-25T08:50:02.609 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:02.609 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-25T08:50:02.609 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-25T08:50:02.609 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-25T08:50:02.609 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-25T08:50:02.609 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-25T08:50:02.610 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-25T08:50:02.610 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-25T08:50:02.610 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-25T08:50:02.610 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-25T08:50:02.610 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:02.610 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-25T08:50:02.610 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:02.611 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-25T08:50:02.611 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-25T08:50:02.611 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:02.611 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-25T08:50:02.611 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2023-08-25T08:50:02.611 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-25T08:50:02.611 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-25T08:50:02.611 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-25T08:50:02.612 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-25T08:50:02.612 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2023-08-25T08:50:02.612 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:02.612 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T08:50:02.612 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:02.612 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-25T08:50:02.612 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:02.613 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2023-08-25T08:49:52.252050+0000", 2023-08-25T08:50:02.613 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T08:50:02.613 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:02.613 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-25T08:50:02.613 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2023-08-25T08:50:02.613 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.973+0000 7fd5ac457700 1 Processor -- start 2023-08-25T08:50:02.613 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.973+0000 7fd5ac457700 1 -- start start 2023-08-25T08:50:02.613 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.973+0000 7fd5ac457700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:02.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.974+0000 7fd5ac457700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a407a8c0 con 0x7fd5a407be90 2023-08-25T08:50:02.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.974+0000 7fd5aa1f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:02.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.974+0000 7fd5aa1f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35806/0 (socket says 172.21.15.99:35806) 2023-08-25T08:50:02.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.974+0000 7fd5aa1f3700 1 -- 172.21.15.99:0/914085487 learned_addr learned my addr 172.21.15.99:0/914085487 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:02.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.975+0000 7fd5aa1f3700 1 -- 172.21.15.99:0/914085487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a407aa00 con 0x7fd5a407be90 2023-08-25T08:50:02.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.975+0000 7fd5aa1f3700 1 --2- 172.21.15.99:0/914085487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a407a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00098a0 tx=0x7fd5a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c90048973f61862 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:02.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.975+0000 7fd5a91f1700 1 -- 172.21.15.99:0/914085487 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5a0004160 con 0x7fd5a407be90 2023-08-25T08:50:02.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.975+0000 7fd5a91f1700 1 -- 172.21.15.99:0/914085487 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5a00042c0 con 0x7fd5a407be90 2023-08-25T08:50:02.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.975+0000 7fd5a91f1700 1 -- 172.21.15.99:0/914085487 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5a0011a00 con 0x7fd5a407be90 2023-08-25T08:50:02.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.976+0000 7fd5ac457700 1 -- 172.21.15.99:0/914085487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 msgr2=0x7fd5a407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:02.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.976+0000 7fd5ac457700 1 --2- 172.21.15.99:0/914085487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a407a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00098a0 tx=0x7fd5a0009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:02.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.976+0000 7fd5ac457700 1 -- 172.21.15.99:0/914085487 shutdown_connections 2023-08-25T08:50:02.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.976+0000 7fd5ac457700 1 --2- 172.21.15.99:0/914085487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a407a2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:02.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.976+0000 7fd5ac457700 1 -- 172.21.15.99:0/914085487 >> 172.21.15.99:0/914085487 conn(0x7fd5a41016e0 msgr2=0x7fd5a4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:02.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.976+0000 7fd5ac457700 1 -- 172.21.15.99:0/914085487 shutdown_connections 2023-08-25T08:50:02.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.976+0000 7fd5ac457700 1 -- 172.21.15.99:0/914085487 wait complete. 2023-08-25T08:50:02.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.977+0000 7fd5ac457700 1 Processor -- start 2023-08-25T08:50:02.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.978+0000 7fd5ac457700 1 -- start start 2023-08-25T08:50:02.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.978+0000 7fd5ac457700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:02.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.978+0000 7fd5ac457700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a419c5f0 con 0x7fd5a407be90 2023-08-25T08:50:02.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.978+0000 7fd5aa1f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:02.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.978+0000 7fd5aa1f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a419c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35812/0 (socket says 172.21.15.99:35812) 2023-08-25T08:50:02.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.978+0000 7fd5aa1f3700 1 -- 172.21.15.99:0/1303050804 learned_addr learned my addr 172.21.15.99:0/1303050804 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:02.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.978+0000 7fd5aa1f3700 1 -- 172.21.15.99:0/1303050804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a0009550 con 0x7fd5a407be90 2023-08-25T08:50:02.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.979+0000 7fd5aa1f3700 1 --2- 172.21.15.99:0/1303050804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a419c0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009130 tx=0x7fd5a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:02.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.979+0000 7fd59b7fe700 1 -- 172.21.15.99:0/1303050804 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5a0003ab0 con 0x7fd5a407be90 2023-08-25T08:50:02.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.979+0000 7fd59b7fe700 1 -- 172.21.15.99:0/1303050804 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5a0003c10 con 0x7fd5a407be90 2023-08-25T08:50:02.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.979+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a419c7f0 con 0x7fd5a407be90 2023-08-25T08:50:02.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.979+0000 7fd59b7fe700 1 -- 172.21.15.99:0/1303050804 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5a00247d0 con 0x7fd5a407be90 2023-08-25T08:50:02.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.979+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a419cc10 con 0x7fd5a407be90 2023-08-25T08:50:02.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.980+0000 7fd59b7fe700 1 -- 172.21.15.99:0/1303050804 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd5a0024a50 con 0x7fd5a407be90 2023-08-25T08:50:02.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.980+0000 7fd59b7fe700 1 -- 172.21.15.99:0/1303050804 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd5a002d440 con 0x7fd5a407be90 2023-08-25T08:50:02.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.980+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a4196100 con 0x7fd5a407be90 2023-08-25T08:50:02.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:01.983+0000 7fd59b7fe700 1 -- 172.21.15.99:0/1303050804 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd5a001a430 con 0x7fd5a407be90 2023-08-25T08:50:02.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.059+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd5a4061f50 con 0x7fd5a407be90 2023-08-25T08:50:02.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.059+0000 7fd59b7fe700 1 -- 172.21.15.99:0/1303050804 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd5a0031070 con 0x7fd5a407be90 2023-08-25T08:50:02.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.060+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 msgr2=0x7fd5a419c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:02.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.060+0000 7fd5ac457700 1 --2- 172.21.15.99:0/1303050804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a419c0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009130 tx=0x7fd5a0011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:02.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.060+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 shutdown_connections 2023-08-25T08:50:02.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.060+0000 7fd5ac457700 1 --2- 172.21.15.99:0/1303050804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a407be90 0x7fd5a419c0b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:02.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.060+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 >> 172.21.15.99:0/1303050804 conn(0x7fd5a41016e0 msgr2=0x7fd5a4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:02.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.060+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 shutdown_connections 2023-08-25T08:50:02.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:02.060+0000 7fd5ac457700 1 -- 172.21.15.99:0/1303050804 wait complete. 2023-08-25T08:50:02.619 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (1/15)... 2023-08-25T08:50:02.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:02.527+0000 7f9a13d7d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T08:50:02.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:02.696+0000 7f9a13d7d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T08:50:03.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:02.811+0000 7f9a13d7d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T08:50:03.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:02.933+0000 7f9a13d7d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T08:50:03.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:03.468+0000 7f9a13d7d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T08:50:04.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:04 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:04.337+0000 7f9a13d7d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T08:50:04.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:04 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:04.443+0000 7f9a13d7d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T08:50:05.549 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:05 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/760695359' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-25T08:50:05.550 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:05.430+0000 7f9a13d7d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:50:05.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:05.549+0000 7f9a13d7d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T08:50:05.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:05.778+0000 7f9a13d7d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:50:06.424 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:06.141+0000 7f9a13d7d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T08:50:06.425 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:06.423+0000 7f9a13d7d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T08:50:06.811 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:06.618+0000 7f9a13d7d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T08:50:07.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:06.962+0000 7f9a13d7d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T08:50:08.300 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:07 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:07.881+0000 7f9a13d7d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T08:50:08.301 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:08.056+0000 7f9a13d7d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T08:50:08.301 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:08.300+0000 7f9a13d7d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T08:50:08.561 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:08.445+0000 7f9a13d7d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T08:50:09.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:08.718+0000 7f9a13d7d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T08:50:10.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:09.776+0000 7f9a13d7d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T08:50:10.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: Activating manager daemon x 2023-08-25T08:50:10.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: mgrmap e2: x(active, starting, since 0.00375333s) 2023-08-25T08:50:10.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T08:50:10.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T08:50:10.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T08:50:10.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:50:10.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T08:50:10.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: Manager daemon x is now available 2023-08-25T08:50:10.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T08:50:10.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T08:50:10.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' 2023-08-25T08:50:10.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' 2023-08-25T08:50:10.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:09 smithi099 ceph-mon[97641]: from='mgr.14100 172.21.15.99:0/461701272' entity='mgr.x' 2023-08-25T08:50:12.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:11 smithi099 ceph-mon[97641]: mgrmap e3: x(active, since 1.00961s) 2023-08-25T08:50:13.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:12 smithi099 ceph-mon[97641]: mgrmap e4: x(active, since 2s) 2023-08-25T08:50:14.007 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:50:14.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2023-08-25T08:50:14.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:50:14.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2023-08-25T08:50:14.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-25T08:50:14.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-25T08:50:14.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-25T08:50:14.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:14.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-25T08:50:14.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-25T08:50:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2023-08-25T08:50:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-25T08:50:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2023-08-25T08:50:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-08-25T08:50:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-25T08:50:14.012 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:14.012 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-25T08:50:14.012 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-25T08:50:14.012 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:14.012 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-25T08:50:14.012 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:14.012 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-25T08:50:14.013 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-25T08:50:14.013 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-25T08:50:14.019 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-25T08:50:14.019 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-25T08:50:14.019 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-25T08:50:14.019 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:14.019 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-25T08:50:14.020 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-25T08:50:14.021 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-25T08:50:14.021 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-25T08:50:14.021 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-25T08:50:14.021 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-25T08:50:14.021 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-25T08:50:14.021 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-25T08:50:14.021 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-25T08:50:14.022 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:14.022 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-25T08:50:14.022 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:14.022 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-25T08:50:14.022 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-25T08:50:14.022 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:14.022 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-25T08:50:14.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2023-08-25T08:50:14.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-25T08:50:14.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-25T08:50:14.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-25T08:50:14.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-25T08:50:14.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2023-08-25T08:50:14.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:14.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T08:50:14.024 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:14.024 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-25T08:50:14.024 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:14.024 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2023-08-25T08:49:52.252050+0000", 2023-08-25T08:50:14.024 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T08:50:14.024 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:14.024 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-25T08:50:14.025 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2023-08-25T08:50:14.025 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.129+0000 7f4e4b0e9700 1 Processor -- start 2023-08-25T08:50:14.025 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.130+0000 7f4e4b0e9700 1 -- start start 2023-08-25T08:50:14.025 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.130+0000 7f4e4b0e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e44106540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:14.025 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.130+0000 7f4e4b0e9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e44106b10 con 0x7f4e44106120 2023-08-25T08:50:14.025 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.130+0000 7f4e48e85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e44106540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:14.025 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.130+0000 7f4e48e85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e44106540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35828/0 (socket says 172.21.15.99:35828) 2023-08-25T08:50:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.130+0000 7f4e48e85700 1 -- 172.21.15.99:0/3377816968 learned_addr learned my addr 172.21.15.99:0/3377816968 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.131+0000 7f4e48e85700 1 -- 172.21.15.99:0/3377816968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44107320 con 0x7f4e44106120 2023-08-25T08:50:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.131+0000 7f4e48e85700 1 --2- 172.21.15.99:0/3377816968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e44106540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e380098a0 tx=0x7f4e38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bff85273900c6d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.131+0000 7f4e437fe700 1 -- 172.21.15.99:0/3377816968 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e38004160 con 0x7f4e44106120 2023-08-25T08:50:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.131+0000 7f4e437fe700 1 -- 172.21.15.99:0/3377816968 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4e380042c0 con 0x7f4e44106120 2023-08-25T08:50:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.131+0000 7f4e437fe700 1 -- 172.21.15.99:0/3377816968 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e38011a00 con 0x7f4e44106120 2023-08-25T08:50:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.132+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/3377816968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 msgr2=0x7f4e44106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.132+0000 7f4e4b0e9700 1 --2- 172.21.15.99:0/3377816968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e44106540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e380098a0 tx=0x7f4e38009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:14.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.132+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/3377816968 shutdown_connections 2023-08-25T08:50:14.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.132+0000 7f4e4b0e9700 1 --2- 172.21.15.99:0/3377816968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e44106540 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:14.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.132+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/3377816968 >> 172.21.15.99:0/3377816968 conn(0x7f4e441016e0 msgr2=0x7f4e44103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:14.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.132+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/3377816968 shutdown_connections 2023-08-25T08:50:14.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.132+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/3377816968 wait complete. 2023-08-25T08:50:14.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.133+0000 7f4e4b0e9700 1 Processor -- start 2023-08-25T08:50:14.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.133+0000 7f4e4b0e9700 1 -- start start 2023-08-25T08:50:14.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.134+0000 7f4e4b0e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e4419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:14.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.134+0000 7f4e4b0e9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4419c5f0 con 0x7f4e44106120 2023-08-25T08:50:14.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.134+0000 7f4e48e85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e4419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:14.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.134+0000 7f4e48e85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e4419c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35842/0 (socket says 172.21.15.99:35842) 2023-08-25T08:50:14.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.134+0000 7f4e48e85700 1 -- 172.21.15.99:0/760695359 learned_addr learned my addr 172.21.15.99:0/760695359 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:14.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.134+0000 7f4e48e85700 1 -- 172.21.15.99:0/760695359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38009550 con 0x7f4e44106120 2023-08-25T08:50:14.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.135+0000 7f4e48e85700 1 --2- 172.21.15.99:0/760695359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e4419c0b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009130 tx=0x7f4e38011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:14.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.135+0000 7f4e41ffb700 1 -- 172.21.15.99:0/760695359 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e38003ab0 con 0x7f4e44106120 2023-08-25T08:50:14.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.135+0000 7f4e41ffb700 1 -- 172.21.15.99:0/760695359 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4e38003c10 con 0x7f4e44106120 2023-08-25T08:50:14.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.135+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4419c7f0 con 0x7f4e44106120 2023-08-25T08:50:14.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.135+0000 7f4e41ffb700 1 -- 172.21.15.99:0/760695359 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e380247d0 con 0x7f4e44106120 2023-08-25T08:50:14.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.135+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4419cc10 con 0x7f4e44106120 2023-08-25T08:50:14.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.135+0000 7f4e41ffb700 1 -- 172.21.15.99:0/760695359 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e38024d00 con 0x7f4e44106120 2023-08-25T08:50:14.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.135+0000 7f4e41ffb700 1 -- 172.21.15.99:0/760695359 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4e380193c0 con 0x7f4e44106120 2023-08-25T08:50:14.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.136+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e44196300 con 0x7f4e44106120 2023-08-25T08:50:14.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.139+0000 7f4e41ffb700 1 -- 172.21.15.99:0/760695359 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4e3801abb0 con 0x7f4e44106120 2023-08-25T08:50:14.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.214+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4e44061f50 con 0x7f4e44106120 2023-08-25T08:50:14.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.215+0000 7f4e41ffb700 1 -- 172.21.15.99:0/760695359 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f4e3801ad50 con 0x7f4e44106120 2023-08-25T08:50:14.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.215+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 msgr2=0x7f4e4419c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:14.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.215+0000 7f4e4b0e9700 1 --2- 172.21.15.99:0/760695359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e4419c0b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009130 tx=0x7f4e38011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:14.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.216+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 shutdown_connections 2023-08-25T08:50:14.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.216+0000 7f4e4b0e9700 1 --2- 172.21.15.99:0/760695359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e44106120 0x7f4e4419c0b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:14.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.216+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 >> 172.21.15.99:0/760695359 conn(0x7f4e441016e0 msgr2=0x7f4e44102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:14.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.216+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 shutdown_connections 2023-08-25T08:50:14.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:05.216+0000 7f4e4b0e9700 1 -- 172.21.15.99:0/760695359 wait complete. 2023-08-25T08:50:14.031 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (2/15)... 2023-08-25T08:50:17.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:50:17.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2023-08-25T08:50:17.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:50:17.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2023-08-25T08:50:17.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-25T08:50:17.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-25T08:50:17.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-25T08:50:17.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:17.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-25T08:50:17.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-25T08:50:17.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2023-08-25T08:50:17.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:17.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-25T08:50:17.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2023-08-25T08:50:17.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:17.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2023-08-25T08:50:17.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-25T08:50:17.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:17.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-25T08:50:17.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-25T08:50:17.203 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:17.203 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-25T08:50:17.203 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:17.203 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-25T08:50:17.203 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-25T08:50:17.203 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-25T08:50:17.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-25T08:50:17.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-25T08:50:17.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-25T08:50:17.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:17.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-25T08:50:17.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-25T08:50:17.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-25T08:50:17.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-25T08:50:17.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-25T08:50:17.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-25T08:50:17.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-25T08:50:17.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-25T08:50:17.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-25T08:50:17.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:17.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-25T08:50:17.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:17.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-25T08:50:17.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-25T08:50:17.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:17.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-25T08:50:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-25T08:50:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-25T08:50:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-25T08:50:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-25T08:50:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-25T08:50:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2023-08-25T08:50:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2023-08-25T08:50:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T08:50:17.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:17.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-25T08:50:17.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T08:50:17.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2023-08-25T08:49:52.252050+0000", 2023-08-25T08:50:17.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T08:50:17.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2023-08-25T08:50:17.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-25T08:50:17.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2023-08-25T08:50:17.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.531+0000 7faab22e3700 1 Processor -- start 2023-08-25T08:50:17.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.532+0000 7faab22e3700 1 -- start start 2023-08-25T08:50:17.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.532+0000 7faab22e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac106540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:17.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.532+0000 7faab22e3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac106b10 con 0x7faaac106120 2023-08-25T08:50:17.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.532+0000 7faaabfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac106540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:17.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.532+0000 7faaabfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac106540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43956/0 (socket says 172.21.15.99:43956) 2023-08-25T08:50:17.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.532+0000 7faaabfff700 1 -- 172.21.15.99:0/1730201217 learned_addr learned my addr 172.21.15.99:0/1730201217 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:17.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.533+0000 7faaabfff700 1 -- 172.21.15.99:0/1730201217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac107320 con 0x7faaac106120 2023-08-25T08:50:17.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.533+0000 7faaabfff700 1 --2- 172.21.15.99:0/1730201217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac106540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0098a0 tx=0x7faa9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31c607dab97d7df server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:17.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.533+0000 7faaaaffd700 1 -- 172.21.15.99:0/1730201217 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa9c004160 con 0x7faaac106120 2023-08-25T08:50:17.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.533+0000 7faaaaffd700 1 -- 172.21.15.99:0/1730201217 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa9c0042c0 con 0x7faaac106120 2023-08-25T08:50:17.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.533+0000 7faaaaffd700 1 -- 172.21.15.99:0/1730201217 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa9c011a00 con 0x7faaac106120 2023-08-25T08:50:17.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.534+0000 7faab22e3700 1 -- 172.21.15.99:0/1730201217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 msgr2=0x7faaac106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:17.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.534+0000 7faab22e3700 1 --2- 172.21.15.99:0/1730201217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac106540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0098a0 tx=0x7faa9c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:17.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.534+0000 7faab22e3700 1 -- 172.21.15.99:0/1730201217 shutdown_connections 2023-08-25T08:50:17.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.534+0000 7faab22e3700 1 --2- 172.21.15.99:0/1730201217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac106540 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:17.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.534+0000 7faab22e3700 1 -- 172.21.15.99:0/1730201217 >> 172.21.15.99:0/1730201217 conn(0x7faaac1016e0 msgr2=0x7faaac103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:17.222 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.534+0000 7faab22e3700 1 -- 172.21.15.99:0/1730201217 shutdown_connections 2023-08-25T08:50:17.222 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.534+0000 7faab22e3700 1 -- 172.21.15.99:0/1730201217 wait complete. 2023-08-25T08:50:17.222 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.535+0000 7faab22e3700 1 Processor -- start 2023-08-25T08:50:17.222 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.535+0000 7faab22e3700 1 -- start start 2023-08-25T08:50:17.224 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.535+0000 7faab22e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac07bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:17.224 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.535+0000 7faab22e3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac07c180 con 0x7faaac106120 2023-08-25T08:50:17.224 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faaabfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac07bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:17.225 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faaabfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac07bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43962/0 (socket says 172.21.15.99:43962) 2023-08-25T08:50:17.225 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faaabfff700 1 -- 172.21.15.99:0/2040026591 learned_addr learned my addr 172.21.15.99:0/2040026591 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:17.225 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faaabfff700 1 -- 172.21.15.99:0/2040026591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c009550 con 0x7faaac106120 2023-08-25T08:50:17.225 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faaabfff700 1 --2- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac07bc40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009130 tx=0x7faa9c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:17.225 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faaa97fa700 1 -- 172.21.15.99:0/2040026591 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa9c003ab0 con 0x7faaac106120 2023-08-25T08:50:17.225 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac07a290 con 0x7faaac106120 2023-08-25T08:50:17.226 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faaa97fa700 1 -- 172.21.15.99:0/2040026591 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa9c003c10 con 0x7faaac106120 2023-08-25T08:50:17.226 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.536+0000 7faaa97fa700 1 -- 172.21.15.99:0/2040026591 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa9c0247d0 con 0x7faaac106120 2023-08-25T08:50:17.226 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.537+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac07a6b0 con 0x7faaac106120 2023-08-25T08:50:17.226 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.537+0000 7faaa97fa700 1 -- 172.21.15.99:0/2040026591 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7faa9c01a430 con 0x7faaac106120 2023-08-25T08:50:17.226 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.538+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac061f50 con 0x7faaac106120 2023-08-25T08:50:17.226 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.538+0000 7faaa97fa700 1 --2- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7faa940380c0 0x7faa9403a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:17.227 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.538+0000 7faaa97fa700 1 -- 172.21.15.99:0/2040026591 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faa9c04bcd0 con 0x7faaac106120 2023-08-25T08:50:17.227 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.538+0000 7faaab7fe700 1 --2- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7faa940380c0 0x7faa9403a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:17.227 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.539+0000 7faaab7fe700 1 --2- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7faa940380c0 0x7faa9403a580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faaa0006fd0 tx=0x7faaa0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:17.227 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.544+0000 7faaa97fa700 1 -- 172.21.15.99:0/2040026591 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7faa9c018350 con 0x7faaac106120 2023-08-25T08:50:17.227 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.818+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7faaac07b0c0 con 0x7faaac106120 2023-08-25T08:50:17.227 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.819+0000 7faaa97fa700 1 -- 172.21.15.99:0/2040026591 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7faa9c018350 con 0x7faaac106120 2023-08-25T08:50:17.227 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7faa940380c0 msgr2=0x7faa9403a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:17.228 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 --2- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7faa940380c0 0x7faa9403a580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faaa0006fd0 tx=0x7faaa0006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:17.228 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 msgr2=0x7faaac07bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:17.228 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 --2- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac07bc40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009130 tx=0x7faa9c011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:17.228 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 shutdown_connections 2023-08-25T08:50:17.228 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 --2- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7faa940380c0 0x7faa9403a580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:17.228 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 --2- 172.21.15.99:0/2040026591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac106120 0x7faaac07bc40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:17.229 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 >> 172.21.15.99:0/2040026591 conn(0x7faaac1016e0 msgr2=0x7faaac102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:17.229 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 shutdown_connections 2023-08-25T08:50:17.229 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:16.821+0000 7faab22e3700 1 -- 172.21.15.99:0/2040026591 wait complete. 2023-08-25T08:50:17.229 INFO:teuthology.orchestra.run.smithi099.stdout:mgr is available 2023-08-25T08:50:17.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:16 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2040026591' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-25T08:50:18.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:17 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2287491790' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-08-25T08:50:18.394 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:50:18.395 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2023-08-25T08:50:18.395 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:50:18.395 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-08-25T08:50:18.396 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-08-25T08:50:18.396 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-08-25T08:50:18.396 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-08-25T08:50:18.397 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:50:18.397 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2023-08-25T08:50:18.397 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-08-25T08:50:18.397 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2023-08-25T08:50:18.397 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2023-08-25T08:50:18.398 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-08-25T08:50:18.398 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-08-25T08:50:18.398 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-08-25T08:50:18.398 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-08-25T08:50:18.398 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-08-25T08:50:18.399 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.706+0000 7f419b89b700 1 Processor -- start 2023-08-25T08:50:18.399 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.707+0000 7f419b89b700 1 -- start start 2023-08-25T08:50:18.399 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.707+0000 7f419b89b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:18.399 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.707+0000 7f419b89b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419407a8c0 con 0x7f419407be90 2023-08-25T08:50:18.399 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.707+0000 7f4199637700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:18.399 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.708+0000 7f4199637700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34012/0 (socket says 172.21.15.99:34012) 2023-08-25T08:50:18.400 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.708+0000 7f4199637700 1 -- 172.21.15.99:0/1033493289 learned_addr learned my addr 172.21.15.99:0/1033493289 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:18.400 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.708+0000 7f4199637700 1 -- 172.21.15.99:0/1033493289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419407aa00 con 0x7f419407be90 2023-08-25T08:50:18.400 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.708+0000 7f4199637700 1 --2- 172.21.15.99:0/1033493289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419407a2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41900098a0 tx=0x7f4190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c52c55f2aee61d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:18.400 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.708+0000 7f418bfff700 1 -- 172.21.15.99:0/1033493289 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4190004160 con 0x7f419407be90 2023-08-25T08:50:18.400 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.708+0000 7f418bfff700 1 -- 172.21.15.99:0/1033493289 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41900042c0 con 0x7f419407be90 2023-08-25T08:50:18.401 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.708+0000 7f418bfff700 1 -- 172.21.15.99:0/1033493289 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4190011a00 con 0x7f419407be90 2023-08-25T08:50:18.401 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.709+0000 7f419b89b700 1 -- 172.21.15.99:0/1033493289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 msgr2=0x7f419407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:18.401 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.709+0000 7f419b89b700 1 --2- 172.21.15.99:0/1033493289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419407a2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41900098a0 tx=0x7f4190009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:18.401 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.709+0000 7f419b89b700 1 -- 172.21.15.99:0/1033493289 shutdown_connections 2023-08-25T08:50:18.401 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.709+0000 7f419b89b700 1 --2- 172.21.15.99:0/1033493289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419407a2f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:18.401 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.709+0000 7f419b89b700 1 -- 172.21.15.99:0/1033493289 >> 172.21.15.99:0/1033493289 conn(0x7f41941016e0 msgr2=0x7f4194103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:18.401 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.709+0000 7f419b89b700 1 -- 172.21.15.99:0/1033493289 shutdown_connections 2023-08-25T08:50:18.402 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.709+0000 7f419b89b700 1 -- 172.21.15.99:0/1033493289 wait complete. 2023-08-25T08:50:18.402 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.710+0000 7f419b89b700 1 Processor -- start 2023-08-25T08:50:18.402 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.710+0000 7f419b89b700 1 -- start start 2023-08-25T08:50:18.402 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.711+0000 7f419b89b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:18.402 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.711+0000 7f419b89b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419419c5f0 con 0x7f419407be90 2023-08-25T08:50:18.403 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.711+0000 7f4199637700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:18.403 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.711+0000 7f4199637700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419419c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34024/0 (socket says 172.21.15.99:34024) 2023-08-25T08:50:18.403 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.711+0000 7f4199637700 1 -- 172.21.15.99:0/2287491790 learned_addr learned my addr 172.21.15.99:0/2287491790 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:18.403 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.711+0000 7f4199637700 1 -- 172.21.15.99:0/2287491790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190009550 con 0x7f419407be90 2023-08-25T08:50:18.403 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.711+0000 7f4199637700 1 --2- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419419c0b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4190009130 tx=0x7f4190011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:18.403 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.712+0000 7f418a7fc700 1 -- 172.21.15.99:0/2287491790 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4190003ab0 con 0x7f419407be90 2023-08-25T08:50:18.403 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.712+0000 7f418a7fc700 1 -- 172.21.15.99:0/2287491790 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4190003c10 con 0x7f419407be90 2023-08-25T08:50:18.404 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.712+0000 7f418a7fc700 1 -- 172.21.15.99:0/2287491790 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41900247d0 con 0x7f419407be90 2023-08-25T08:50:18.404 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.712+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419419c7f0 con 0x7f419407be90 2023-08-25T08:50:18.404 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.712+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419419cc10 con 0x7f419407be90 2023-08-25T08:50:18.404 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.713+0000 7f418a7fc700 1 -- 172.21.15.99:0/2287491790 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7f419001abb0 con 0x7f419407be90 2023-08-25T08:50:18.404 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.713+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4194196100 con 0x7f419407be90 2023-08-25T08:50:18.404 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.713+0000 7f418a7fc700 1 --2- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f4180038110 0x7f418003a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:18.405 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.713+0000 7f418a7fc700 1 -- 172.21.15.99:0/2287491790 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f419004bcb0 con 0x7f419407be90 2023-08-25T08:50:18.405 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.713+0000 7f4198e36700 1 --2- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f4180038110 0x7f418003a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:18.405 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.714+0000 7f4198e36700 1 --2- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f4180038110 0x7f418003a5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4184006fd0 tx=0x7f4184006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:18.405 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.718+0000 7f418a7fc700 1 -- 172.21.15.99:0/2287491790 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4190018710 con 0x7f419407be90 2023-08-25T08:50:18.405 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.915+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4194061fa0 con 0x7f419407be90 2023-08-25T08:50:18.405 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.916+0000 7f418a7fc700 1 -- 172.21.15.99:0/2287491790 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f4190018350 con 0x7f419407be90 2023-08-25T08:50:18.406 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f4180038110 msgr2=0x7f418003a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:18.406 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 --2- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f4180038110 0x7f418003a5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4184006fd0 tx=0x7f4184006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:18.406 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 msgr2=0x7f419419c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:18.406 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 --2- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419419c0b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4190009130 tx=0x7f4190011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:18.406 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 shutdown_connections 2023-08-25T08:50:18.406 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 --2- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f4180038110 0x7f418003a5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:18.406 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 --2- 172.21.15.99:0/2287491790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419407be90 0x7f419419c0b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:18.407 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 >> 172.21.15.99:0/2287491790 conn(0x7f41941016e0 msgr2=0x7f4194102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:18.407 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 shutdown_connections 2023-08-25T08:50:18.407 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:17.918+0000 7f419b89b700 1 -- 172.21.15.99:0/2287491790 wait complete. 2023-08-25T08:50:18.407 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling cephadm module... 2023-08-25T08:50:20.177 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:19 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2966837970' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-08-25T08:50:20.177 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: ignoring --setuser ceph since I am not root 2023-08-25T08:50:20.178 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: ignoring --setgroup ceph since I am not root 2023-08-25T08:50:20.368 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.940+0000 7f8dd59cb700 1 Processor -- start 2023-08-25T08:50:20.369 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.940+0000 7f8dd59cb700 1 -- start start 2023-08-25T08:50:20.369 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.940+0000 7f8dd59cb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:20.369 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.940+0000 7f8dd59cb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd007a8a0 con 0x7f8dd007be70 2023-08-25T08:50:20.369 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.940+0000 7f8dceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:20.369 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.941+0000 7f8dceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34034/0 (socket says 172.21.15.99:34034) 2023-08-25T08:50:20.369 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.941+0000 7f8dceffd700 1 -- 172.21.15.99:0/2585673304 learned_addr learned my addr 172.21.15.99:0/2585673304 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:20.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.941+0000 7f8dceffd700 1 -- 172.21.15.99:0/2585673304 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd007a9e0 con 0x7f8dd007be70 2023-08-25T08:50:20.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.941+0000 7f8dceffd700 1 --2- 172.21.15.99:0/2585673304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd007a2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00098a0 tx=0x7f8dc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7d40d59e65b38b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:20.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.941+0000 7f8dcdffb700 1 -- 172.21.15.99:0/2585673304 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dc0004160 con 0x7f8dd007be70 2023-08-25T08:50:20.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.941+0000 7f8dcdffb700 1 -- 172.21.15.99:0/2585673304 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8dc00042c0 con 0x7f8dd007be70 2023-08-25T08:50:20.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.941+0000 7f8dcdffb700 1 -- 172.21.15.99:0/2585673304 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dc0011a00 con 0x7f8dd007be70 2023-08-25T08:50:20.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.942+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2585673304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 msgr2=0x7f8dd007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:20.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.942+0000 7f8dd59cb700 1 --2- 172.21.15.99:0/2585673304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd007a2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00098a0 tx=0x7f8dc0009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:20.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.942+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2585673304 shutdown_connections 2023-08-25T08:50:20.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.942+0000 7f8dd59cb700 1 --2- 172.21.15.99:0/2585673304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd007a2d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:20.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.942+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2585673304 >> 172.21.15.99:0/2585673304 conn(0x7f8dd01016e0 msgr2=0x7f8dd0103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:20.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.942+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2585673304 shutdown_connections 2023-08-25T08:50:20.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.942+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2585673304 wait complete. 2023-08-25T08:50:20.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.943+0000 7f8dd59cb700 1 Processor -- start 2023-08-25T08:50:20.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.943+0000 7f8dd59cb700 1 -- start start 2023-08-25T08:50:20.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.944+0000 7f8dd59cb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:20.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.944+0000 7f8dd59cb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd019c5b0 con 0x7f8dd007be70 2023-08-25T08:50:20.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.944+0000 7f8dceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:20.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.944+0000 7f8dceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd019c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34050/0 (socket says 172.21.15.99:34050) 2023-08-25T08:50:20.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.944+0000 7f8dceffd700 1 -- 172.21.15.99:0/2966837970 learned_addr learned my addr 172.21.15.99:0/2966837970 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:20.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.944+0000 7f8dceffd700 1 -- 172.21.15.99:0/2966837970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0009550 con 0x7f8dd007be70 2023-08-25T08:50:20.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.944+0000 7f8dceffd700 1 --2- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd019c070 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009130 tx=0x7f8dc0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:20.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.944+0000 7f8dd49c9700 1 -- 172.21.15.99:0/2966837970 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dc0003b70 con 0x7f8dd007be70 2023-08-25T08:50:20.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.945+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd019c7b0 con 0x7f8dd007be70 2023-08-25T08:50:20.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.945+0000 7f8dd49c9700 1 -- 172.21.15.99:0/2966837970 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8dc0003cd0 con 0x7f8dd007be70 2023-08-25T08:50:20.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.945+0000 7f8dd49c9700 1 -- 172.21.15.99:0/2966837970 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dc00197d0 con 0x7f8dd007be70 2023-08-25T08:50:20.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.945+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd019cbd0 con 0x7f8dd007be70 2023-08-25T08:50:20.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.946+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd0061f50 con 0x7f8dd007be70 2023-08-25T08:50:20.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.946+0000 7f8dd49c9700 1 -- 172.21.15.99:0/2966837970 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7f8dc001abb0 con 0x7f8dd007be70 2023-08-25T08:50:20.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.946+0000 7f8dd49c9700 1 --2- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f8db8038110 0x7f8db803a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:20.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.946+0000 7f8dd49c9700 1 -- 172.21.15.99:0/2966837970 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8dc004c100 con 0x7f8dd007be70 2023-08-25T08:50:20.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.947+0000 7f8dce7fc700 1 --2- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f8db8038110 0x7f8db803a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:20.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.947+0000 7f8dce7fc700 1 --2- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f8db8038110 0x7f8db803a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4006fd0 tx=0x7f8dc4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:20.377 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:18.951+0000 7f8dd49c9700 1 -- 172.21.15.99:0/2966837970 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8dc007ee00 con 0x7f8dd007be70 2023-08-25T08:50:20.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.201+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f8dd010c510 con 0x7f8dd007be70 2023-08-25T08:50:20.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.918+0000 7f8dd49c9700 1 -- 172.21.15.99:0/2966837970 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f8dc001a7b0 con 0x7f8dd007be70 2023-08-25T08:50:20.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.920+0000 7f8dd49c9700 1 -- 172.21.15.99:0/2966837970 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f8dc0016490 con 0x7f8dd007be70 2023-08-25T08:50:20.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.922+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f8db8038110 msgr2=0x7f8db803a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:20.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.922+0000 7f8dd59cb700 1 --2- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f8db8038110 0x7f8db803a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4006fd0 tx=0x7f8dc4006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:20.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.922+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 msgr2=0x7f8dd019c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:20.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.922+0000 7f8dd59cb700 1 --2- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd019c070 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009130 tx=0x7f8dc0011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:20.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.923+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 shutdown_connections 2023-08-25T08:50:20.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.923+0000 7f8dd59cb700 1 --2- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f8db8038110 0x7f8db803a5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:20.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.923+0000 7f8dd59cb700 1 --2- 172.21.15.99:0/2966837970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dd007be70 0x7f8dd019c070 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:20.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.923+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 >> 172.21.15.99:0/2966837970 conn(0x7f8dd01016e0 msgr2=0x7f8dd0102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:20.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.923+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 shutdown_connections 2023-08-25T08:50:20.379 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:19.923+0000 7f8dd59cb700 1 -- 172.21.15.99:0/2966837970 wait complete. 2023-08-25T08:50:20.535 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:20.176+0000 7f7902f51200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T08:50:20.535 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:20.281+0000 7f7902f51200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T08:50:20.535 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:20.404+0000 7f7902f51200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T08:50:20.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:20.534+0000 7f7902f51200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T08:50:20.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:20.778+0000 7f7902f51200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T08:50:21.194 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2966837970' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-08-25T08:50:21.194 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:20 smithi099 ceph-mon[97641]: mgrmap e5: x(active, since 10s) 2023-08-25T08:50:21.540 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:21 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:21.251+0000 7f7902f51200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T08:50:21.541 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:21 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:21.420+0000 7f7902f51200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T08:50:21.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2023-08-25T08:50:21.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-08-25T08:50:21.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-25T08:50:21.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-08-25T08:50:21.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-08-25T08:50:21.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2023-08-25T08:50:21.552 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.894+0000 7f0418ba0700 1 Processor -- start 2023-08-25T08:50:21.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.894+0000 7f0418ba0700 1 -- start start 2023-08-25T08:50:21.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.894+0000 7f0418ba0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f0414105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:21.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.894+0000 7f0418ba0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0414105a10 con 0x7f0414105020 2023-08-25T08:50:21.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.895+0000 7f041259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f0414105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:21.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.895+0000 7f041259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f0414105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34068/0 (socket says 172.21.15.99:34068) 2023-08-25T08:50:21.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.895+0000 7f041259c700 1 -- 172.21.15.99:0/3739213583 learned_addr learned my addr 172.21.15.99:0/3739213583 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:21.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.895+0000 7f041259c700 1 -- 172.21.15.99:0/3739213583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0414106220 con 0x7f0414105020 2023-08-25T08:50:21.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.895+0000 7f041259c700 1 --2- 172.21.15.99:0/3739213583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f0414105440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f04040098a0 tx=0x7f0404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243edc87a505be25 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:21.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f041159a700 1 -- 172.21.15.99:0/3739213583 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0404004160 con 0x7f0414105020 2023-08-25T08:50:21.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f041159a700 1 -- 172.21.15.99:0/3739213583 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04040042c0 con 0x7f0414105020 2023-08-25T08:50:21.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f041159a700 1 -- 172.21.15.99:0/3739213583 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0404011a00 con 0x7f0414105020 2023-08-25T08:50:21.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f0418ba0700 1 -- 172.21.15.99:0/3739213583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 msgr2=0x7f0414105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:21.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f0418ba0700 1 --2- 172.21.15.99:0/3739213583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f0414105440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f04040098a0 tx=0x7f0404009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:21.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f0418ba0700 1 -- 172.21.15.99:0/3739213583 shutdown_connections 2023-08-25T08:50:21.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f0418ba0700 1 --2- 172.21.15.99:0/3739213583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f0414105440 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:21.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f0418ba0700 1 -- 172.21.15.99:0/3739213583 >> 172.21.15.99:0/3739213583 conn(0x7f04141005c0 msgr2=0x7f0414102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:21.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.896+0000 7f0418ba0700 1 -- 172.21.15.99:0/3739213583 shutdown_connections 2023-08-25T08:50:21.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.897+0000 7f0418ba0700 1 -- 172.21.15.99:0/3739213583 wait complete. 2023-08-25T08:50:21.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.898+0000 7f0418ba0700 1 Processor -- start 2023-08-25T08:50:21.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.898+0000 7f0418ba0700 1 -- start start 2023-08-25T08:50:21.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.899+0000 7f0418ba0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f04141901b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:21.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.899+0000 7f0418ba0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04141906f0 con 0x7f0414105020 2023-08-25T08:50:21.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.899+0000 7f041259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f04141901b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:21.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.899+0000 7f041259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f04141901b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34078/0 (socket says 172.21.15.99:34078) 2023-08-25T08:50:21.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.899+0000 7f041259c700 1 -- 172.21.15.99:0/105386621 learned_addr learned my addr 172.21.15.99:0/105386621 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:21.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.899+0000 7f041259c700 1 -- 172.21.15.99:0/105386621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0404009550 con 0x7f0414105020 2023-08-25T08:50:21.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.899+0000 7f041259c700 1 --2- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f04141901b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0404006af0 tx=0x7f0404011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:21.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.900+0000 7f04037fe700 1 -- 172.21.15.99:0/105386621 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0404003a40 con 0x7f0414105020 2023-08-25T08:50:21.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.900+0000 7f04037fe700 1 -- 172.21.15.99:0/105386621 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0404003ba0 con 0x7f0414105020 2023-08-25T08:50:21.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.900+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041418e800 con 0x7f0414105020 2023-08-25T08:50:21.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.900+0000 7f04037fe700 1 -- 172.21.15.99:0/105386621 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f040401a700 con 0x7f0414105020 2023-08-25T08:50:21.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.900+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041418ec20 con 0x7f0414105020 2023-08-25T08:50:21.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.901+0000 7f04037fe700 1 -- 172.21.15.99:0/105386621 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f0404024430 con 0x7f0414105020 2023-08-25T08:50:21.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.901+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0414061f50 con 0x7f0414105020 2023-08-25T08:50:21.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.901+0000 7f04037fe700 1 --2- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f03fc038110 0x7f03fc03a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:21.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.901+0000 7f04037fe700 1 -- 172.21.15.99:0/105386621 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f040404bf80 con 0x7f0414105020 2023-08-25T08:50:21.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.901+0000 7f0411d9b700 1 -- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f03fc038110 msgr2=0x7f03fc03a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3349539225 2023-08-25T08:50:21.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.902+0000 7f0411d9b700 1 --2- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f03fc038110 0x7f03fc03a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:50:21.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:20.910+0000 7f04037fe700 1 -- 172.21.15.99:0/105386621 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f04040246e0 con 0x7f0414105020 2023-08-25T08:50:21.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.103+0000 7f0411d9b700 1 -- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f03fc038110 msgr2=0x7f03fc03a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3349539225 2023-08-25T08:50:21.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.103+0000 7f0411d9b700 1 --2- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f03fc038110 0x7f03fc03a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T08:50:21.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.169+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0414109420 con 0x7f0414105020 2023-08-25T08:50:21.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.169+0000 7f04037fe700 1 -- 172.21.15.99:0/105386621 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f040400be80 con 0x7f0414105020 2023-08-25T08:50:21.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.171+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f03fc038110 msgr2=0x7f03fc03a5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:50:21.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.171+0000 7f0418ba0700 1 --2- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f03fc038110 0x7f03fc03a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:21.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.171+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 msgr2=0x7f04141901b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.171+0000 7f0418ba0700 1 --2- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f04141901b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0404006af0 tx=0x7f0404011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.171+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 shutdown_connections 2023-08-25T08:50:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.171+0000 7f0418ba0700 1 --2- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7f03fc038110 0x7f03fc03a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.172+0000 7f0418ba0700 1 --2- 172.21.15.99:0/105386621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414105020 0x7f04141901b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.172+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 >> 172.21.15.99:0/105386621 conn(0x7f04141005c0 msgr2=0x7f0414079910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.172+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 shutdown_connections 2023-08-25T08:50:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:21.172+0000 7f0418ba0700 1 -- 172.21.15.99:0/105386621 wait complete. 2023-08-25T08:50:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2023-08-25T08:50:21.561 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 5... 2023-08-25T08:50:21.811 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:21 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:21.540+0000 7f7902f51200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T08:50:21.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:21 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:21.672+0000 7f7902f51200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T08:50:22.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:21 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/105386621' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-08-25T08:50:22.561 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:22 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:22.222+0000 7f7902f51200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T08:50:23.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:23 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:23.053+0000 7f7902f51200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T08:50:23.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:23 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:23.158+0000 7f7902f51200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T08:50:24.452 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:24 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:24.113+0000 7f7902f51200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:50:24.453 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:24 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:24.232+0000 7f7902f51200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T08:50:24.809 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:24 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:24.452+0000 7f7902f51200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:50:25.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:24 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:24.809+0000 7f7902f51200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T08:50:25.562 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:25.081+0000 7f7902f51200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T08:50:25.562 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:25.270+0000 7f7902f51200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T08:50:26.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:25.611+0000 7f7902f51200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T08:50:26.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:26 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:26.516+0000 7f7902f51200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T08:50:26.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:26 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:26.686+0000 7f7902f51200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T08:50:27.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:26 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:26.929+0000 7f7902f51200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T08:50:27.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:27 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:27.071+0000 7f7902f51200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T08:50:27.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:27 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:27.343+0000 7f7902f51200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T08:50:28.811 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:28.394+0000 7f7902f51200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T08:50:28.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: Active manager daemon x restarted 2023-08-25T08:50:28.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: Activating manager daemon x 2023-08-25T08:50:28.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: osdmap e2: 0 total, 0 up, 0 in 2023-08-25T08:50:28.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: mgrmap e6: x(active, starting, since 0.00365342s) 2023-08-25T08:50:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:50:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T08:50:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T08:50:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T08:50:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T08:50:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: Manager daemon x is now available 2023-08-25T08:50:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:28.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:28.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:50:28.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:28 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:50:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:29 smithi099 ceph-mon[97641]: Found migration_current of "None". Setting to last migration. 2023-08-25T08:50:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:29 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:50:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:29 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T08:50:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:29 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T08:50:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:29 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:29 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:29 smithi099 ceph-mon[97641]: mgrmap e7: x(active, since 1.00716s) 2023-08-25T08:50:29.850 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2023-08-25T08:50:29.850 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-08-25T08:50:29.850 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2023-08-25T08:50:29.851 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2023-08-25T08:50:29.851 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.109+0000 7fd747954700 1 Processor -- start 2023-08-25T08:50:29.851 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.109+0000 7fd747954700 1 -- start start 2023-08-25T08:50:29.851 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.110+0000 7fd747954700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:29.851 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.110+0000 7fd747954700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74007a8a0 con 0x7fd74007be70 2023-08-25T08:50:29.851 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.110+0000 7fd7456f0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:29.851 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.110+0000 7fd7456f0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34092/0 (socket says 172.21.15.99:34092) 2023-08-25T08:50:29.852 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.110+0000 7fd7456f0700 1 -- 172.21.15.99:0/3651721918 learned_addr learned my addr 172.21.15.99:0/3651721918 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:29.852 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.110+0000 7fd7456f0700 1 -- 172.21.15.99:0/3651721918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74007a9e0 con 0x7fd74007be70 2023-08-25T08:50:29.852 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.111+0000 7fd7456f0700 1 --2- 172.21.15.99:0/3651721918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74007a2d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0098a0 tx=0x7fd73c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9973282248d5aa6b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:29.852 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.111+0000 7fd737fff700 1 -- 172.21.15.99:0/3651721918 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd73c004160 con 0x7fd74007be70 2023-08-25T08:50:29.852 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.111+0000 7fd737fff700 1 -- 172.21.15.99:0/3651721918 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd73c0042c0 con 0x7fd74007be70 2023-08-25T08:50:29.852 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.111+0000 7fd737fff700 1 -- 172.21.15.99:0/3651721918 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd73c011a00 con 0x7fd74007be70 2023-08-25T08:50:29.852 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.111+0000 7fd747954700 1 -- 172.21.15.99:0/3651721918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 msgr2=0x7fd74007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:29.853 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.111+0000 7fd747954700 1 --2- 172.21.15.99:0/3651721918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74007a2d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0098a0 tx=0x7fd73c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:29.853 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.112+0000 7fd747954700 1 -- 172.21.15.99:0/3651721918 shutdown_connections 2023-08-25T08:50:29.853 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.112+0000 7fd747954700 1 --2- 172.21.15.99:0/3651721918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74007a2d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:29.853 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.112+0000 7fd747954700 1 -- 172.21.15.99:0/3651721918 >> 172.21.15.99:0/3651721918 conn(0x7fd7401016e0 msgr2=0x7fd740103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:29.853 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.112+0000 7fd747954700 1 -- 172.21.15.99:0/3651721918 shutdown_connections 2023-08-25T08:50:29.853 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.112+0000 7fd747954700 1 -- 172.21.15.99:0/3651721918 wait complete. 2023-08-25T08:50:29.853 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.113+0000 7fd747954700 1 Processor -- start 2023-08-25T08:50:29.853 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.113+0000 7fd747954700 1 -- start start 2023-08-25T08:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.113+0000 7fd747954700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.113+0000 7fd747954700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74019c580 con 0x7fd74007be70 2023-08-25T08:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.114+0000 7fd7456f0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.114+0000 7fd7456f0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74019c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34100/0 (socket says 172.21.15.99:34100) 2023-08-25T08:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.114+0000 7fd7456f0700 1 -- 172.21.15.99:0/1411156816 learned_addr learned my addr 172.21.15.99:0/1411156816 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.114+0000 7fd7456f0700 1 -- 172.21.15.99:0/1411156816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c009550 con 0x7fd74007be70 2023-08-25T08:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.114+0000 7fd7456f0700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74019c040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009130 tx=0x7fd73c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:29.855 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.115+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd73c003b70 con 0x7fd74007be70 2023-08-25T08:50:29.855 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.115+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd73c003cd0 con 0x7fd74007be70 2023-08-25T08:50:29.855 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.115+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd73c0197d0 con 0x7fd74007be70 2023-08-25T08:50:29.855 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.115+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74019c780 con 0x7fd74007be70 2023-08-25T08:50:29.855 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.115+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74019cba0 con 0x7fd74007be70 2023-08-25T08:50:29.855 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.116+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fd73c01abb0 con 0x7fd74007be70 2023-08-25T08:50:29.855 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.117+0000 7fd7367fc700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:29.855 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.117+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 --> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd72c03ad30 con 0x7fd72c038160 2023-08-25T08:50:29.856 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.117+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd73c04c130 con 0x7fd74007be70 2023-08-25T08:50:29.856 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.117+0000 7fd744eef700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 msgr2=0x7fd72c03a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3349539225 2023-08-25T08:50:29.856 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.117+0000 7fd744eef700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:50:29.856 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.318+0000 7fd744eef700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 msgr2=0x7fd72c03a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3349539225 2023-08-25T08:50:29.856 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.318+0000 7fd744eef700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T08:50:29.856 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.719+0000 7fd744eef700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 msgr2=0x7fd72c03a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3349539225 2023-08-25T08:50:29.856 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:22.719+0000 7fd744eef700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-25T08:50:29.857 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:23.520+0000 7fd744eef700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 msgr2=0x7fd72c03a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3349539225 2023-08-25T08:50:29.857 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:23.520+0000 7fd744eef700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-25T08:50:29.857 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:25.122+0000 7fd744eef700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 msgr2=0x7fd72c03a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3349539225 2023-08-25T08:50:29.857 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:25.122+0000 7fd744eef700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-25T08:50:29.857 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:28.325+0000 7fd744eef700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 msgr2=0x7fd72c03a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3349539225 2023-08-25T08:50:29.857 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:28.325+0000 7fd744eef700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-25T08:50:29.857 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:28.398+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mgrmap(e 6) v1 ==== 44902+0+0 (secure 0 0 0) 0x7fd73c019d40 con 0x7fd74007be70 2023-08-25T08:50:29.857 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:28.398+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 msgr2=0x7fd72c03a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:50:29.858 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:28.398+0000 7fd7367fc700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:29.858 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.400+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 7) v1 ==== 45029+0+0 (secure 0 0 0) 0x7fd73c02d350 con 0x7fd74007be70 2023-08-25T08:50:29.858 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.401+0000 7fd7367fc700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fd72c03ba80 0x7fd72c03de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:29.858 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.401+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd72c03ad30 con 0x7fd72c03ba80 2023-08-25T08:50:29.858 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.402+0000 7fd744eef700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fd72c03ba80 0x7fd72c03de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:29.858 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.403+0000 7fd744eef700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fd72c03ba80 0x7fd72c03de70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd730003a10 tx=0x7fd7300092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:29.858 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.403+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mgr.14118 v2:172.21.15.99:6800/2336486699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fd72c03ad30 con 0x7fd72c03ba80 2023-08-25T08:50:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd74019d380 con 0x7fd72c03ba80 2023-08-25T08:50:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd7367fc700 1 -- 172.21.15.99:0/1411156816 <== mgr.14118 v2:172.21.15.99:6800/2336486699 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd74019d380 con 0x7fd72c03ba80 2023-08-25T08:50:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fd72c03ba80 msgr2=0x7fd72c03de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fd72c03ba80 0x7fd72c03de70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd730003a10 tx=0x7fd7300092b0 comp rx=0 tx=0).stop 2023-08-25T08:50:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 msgr2=0x7fd74019c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74019c040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009130 tx=0x7fd73c011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 shutdown_connections 2023-08-25T08:50:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fd72c03ba80 0x7fd72c03de70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:29.860 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd74007be70 0x7fd74019c040 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:29.860 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 --2- 172.21.15.99:0/1411156816 >> [v2:172.21.15.99:6800/3349539225,v1:172.21.15.99:6801/3349539225] conn(0x7fd72c038160 0x7fd72c03a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:29.860 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.412+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 >> 172.21.15.99:0/1411156816 conn(0x7fd7401016e0 msgr2=0x7fd740102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:29.860 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.413+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 shutdown_connections 2023-08-25T08:50:29.860 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:29.413+0000 7fd747954700 1 -- 172.21.15.99:0/1411156816 wait complete. 2023-08-25T08:50:29.860 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 5 is available 2023-08-25T08:50:29.860 INFO:teuthology.orchestra.run.smithi099.stdout:Setting orchestrator backend to cephadm... 2023-08-25T08:50:30.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:30 smithi099 ceph-mon[97641]: [25/Aug/2023:08:50:29] ENGINE Bus STARTING 2023-08-25T08:50:30.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:30 smithi099 ceph-mon[97641]: [25/Aug/2023:08:50:29] ENGINE Serving on https://172.21.15.99:7150 2023-08-25T08:50:30.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:30 smithi099 ceph-mon[97641]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-08-25T08:50:30.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:30 smithi099 ceph-mon[97641]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-08-25T08:50:30.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:30 smithi099 ceph-mon[97641]: [25/Aug/2023:08:50:29] ENGINE Serving on http://172.21.15.99:8765 2023-08-25T08:50:30.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:30 smithi099 ceph-mon[97641]: [25/Aug/2023:08:50:29] ENGINE Bus STARTED 2023-08-25T08:50:30.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:30 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:50:31.025 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.320+0000 7f52169bd700 1 Processor -- start 2023-08-25T08:50:31.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.321+0000 7f52169bd700 1 -- start start 2023-08-25T08:50:31.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.321+0000 7f52169bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:31.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.321+0000 7f52169bd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521007a8a0 con 0x7f521007be70 2023-08-25T08:50:31.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.321+0000 7f520ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:31.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.321+0000 7f520ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46824/0 (socket says 172.21.15.99:46824) 2023-08-25T08:50:31.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.321+0000 7f520ffff700 1 -- 172.21.15.99:0/941245565 learned_addr learned my addr 172.21.15.99:0/941245565 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:31.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.322+0000 7f520ffff700 1 -- 172.21.15.99:0/941245565 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521007a9e0 con 0x7f521007be70 2023-08-25T08:50:31.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.322+0000 7f520ffff700 1 --2- 172.21.15.99:0/941245565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521007a2d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f52000098a0 tx=0x7f5200009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a426cd334e3b1f4f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:31.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.322+0000 7f520effd700 1 -- 172.21.15.99:0/941245565 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5200004160 con 0x7f521007be70 2023-08-25T08:50:31.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.322+0000 7f520effd700 1 -- 172.21.15.99:0/941245565 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f52000042c0 con 0x7f521007be70 2023-08-25T08:50:31.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.322+0000 7f520effd700 1 -- 172.21.15.99:0/941245565 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5200011a00 con 0x7f521007be70 2023-08-25T08:50:31.033 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.323+0000 7f52169bd700 1 -- 172.21.15.99:0/941245565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 msgr2=0x7f521007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:31.033 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.323+0000 7f52169bd700 1 --2- 172.21.15.99:0/941245565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521007a2d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f52000098a0 tx=0x7f5200009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:31.034 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.323+0000 7f52169bd700 1 -- 172.21.15.99:0/941245565 shutdown_connections 2023-08-25T08:50:31.034 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.323+0000 7f52169bd700 1 --2- 172.21.15.99:0/941245565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521007a2d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:31.034 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.323+0000 7f52169bd700 1 -- 172.21.15.99:0/941245565 >> 172.21.15.99:0/941245565 conn(0x7f52101016e0 msgr2=0x7f5210103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:31.034 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.323+0000 7f52169bd700 1 -- 172.21.15.99:0/941245565 shutdown_connections 2023-08-25T08:50:31.034 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.323+0000 7f52169bd700 1 -- 172.21.15.99:0/941245565 wait complete. 2023-08-25T08:50:31.034 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.324+0000 7f52169bd700 1 Processor -- start 2023-08-25T08:50:31.034 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.324+0000 7f52169bd700 1 -- start start 2023-08-25T08:50:31.035 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.324+0000 7f52169bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:31.035 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.324+0000 7f52169bd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521019c5b0 con 0x7f521007be70 2023-08-25T08:50:31.035 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f520ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:31.035 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f520ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521019c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46840/0 (socket says 172.21.15.99:46840) 2023-08-25T08:50:31.035 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f520ffff700 1 -- 172.21.15.99:0/3497057896 learned_addr learned my addr 172.21.15.99:0/3497057896 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:31.035 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f520ffff700 1 -- 172.21.15.99:0/3497057896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5200009550 con 0x7f521007be70 2023-08-25T08:50:31.035 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f520ffff700 1 --2- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521019c070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5200009130 tx=0x7f5200011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:31.036 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f520d7fa700 1 -- 172.21.15.99:0/3497057896 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5200003b70 con 0x7f521007be70 2023-08-25T08:50:31.036 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f520d7fa700 1 -- 172.21.15.99:0/3497057896 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5200003cd0 con 0x7f521007be70 2023-08-25T08:50:31.036 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521019c7b0 con 0x7f521007be70 2023-08-25T08:50:31.036 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.325+0000 7f520d7fa700 1 -- 172.21.15.99:0/3497057896 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52000197d0 con 0x7f521007be70 2023-08-25T08:50:31.036 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.326+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521019cbd0 con 0x7f521007be70 2023-08-25T08:50:31.036 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.327+0000 7f520d7fa700 1 -- 172.21.15.99:0/3497057896 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 7) v1 ==== 45029+0+0 (secure 0 0 0) 0x7f520001abb0 con 0x7f521007be70 2023-08-25T08:50:31.036 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.327+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521010c510 con 0x7f521007be70 2023-08-25T08:50:31.036 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.327+0000 7f520d7fa700 1 --2- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f51f8038030 0x7f51f803a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:31.037 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.327+0000 7f520d7fa700 1 -- 172.21.15.99:0/3497057896 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f520004bf80 con 0x7f521007be70 2023-08-25T08:50:31.037 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.327+0000 7f520f7fe700 1 --2- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f51f8038030 0x7f51f803a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:31.037 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.328+0000 7f520f7fe700 1 --2- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f51f8038030 0x7f51f803a4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5204006fd0 tx=0x7f5204006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:31.037 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.333+0000 7f520d7fa700 1 -- 172.21.15.99:0/3497057896 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f520001a430 con 0x7f521007be70 2023-08-25T08:50:31.037 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.454+0000 7f520d7fa700 1 -- 172.21.15.99:0/3497057896 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f520002d350 con 0x7f521007be70 2023-08-25T08:50:31.037 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.546+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f521019f610 con 0x7f51f8038030 2023-08-25T08:50:31.037 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.552+0000 7f520d7fa700 1 -- 172.21.15.99:0/3497057896 <== mgr.14118 v2:172.21.15.99:6800/2336486699 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f521019f610 con 0x7f51f8038030 2023-08-25T08:50:31.038 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f51f8038030 msgr2=0x7f51f803a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:31.038 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 --2- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f51f8038030 0x7f51f803a4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5204006fd0 tx=0x7f5204006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:31.038 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 msgr2=0x7f521019c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:31.038 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 --2- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521019c070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5200009130 tx=0x7f5200011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:31.038 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 shutdown_connections 2023-08-25T08:50:31.038 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 --2- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f51f8038030 0x7f51f803a4f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:31.038 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 --2- 172.21.15.99:0/3497057896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521007be70 0x7f521019c070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:31.039 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 >> 172.21.15.99:0/3497057896 conn(0x7f52101016e0 msgr2=0x7f5210102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:31.039 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 shutdown_connections 2023-08-25T08:50:31.039 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:30.555+0000 7f52169bd700 1 -- 172.21.15.99:0/3497057896 wait complete. 2023-08-25T08:50:31.796 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:31 smithi099 ceph-mon[97641]: mgrmap e8: x(active, since 2s) 2023-08-25T08:50:31.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:31 smithi099 ceph-mon[97641]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:31.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:31 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:31.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:31 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:50:32.212 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout value unchanged 2023-08-25T08:50:32.213 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.564+0000 7f86c50f1700 1 Processor -- start 2023-08-25T08:50:32.213 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.564+0000 7f86c50f1700 1 -- start start 2023-08-25T08:50:32.213 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.564+0000 7f86c50f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:32.213 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.564+0000 7f86c50f1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c007a8c0 con 0x7f86c007be90 2023-08-25T08:50:32.213 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.564+0000 7f86bed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:32.213 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.565+0000 7f86bed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46846/0 (socket says 172.21.15.99:46846) 2023-08-25T08:50:32.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.565+0000 7f86bed9d700 1 -- 172.21.15.99:0/1475685351 learned_addr learned my addr 172.21.15.99:0/1475685351 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:32.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.565+0000 7f86bed9d700 1 -- 172.21.15.99:0/1475685351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c007aa00 con 0x7f86c007be90 2023-08-25T08:50:32.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.565+0000 7f86bed9d700 1 --2- 172.21.15.99:0/1475685351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c007a2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f86b00098a0 tx=0x7f86b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f590e1d21f78ed6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:32.214 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.565+0000 7f86bdd9b700 1 -- 172.21.15.99:0/1475685351 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b0004160 con 0x7f86c007be90 2023-08-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.565+0000 7f86bdd9b700 1 -- 172.21.15.99:0/1475685351 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86b00042c0 con 0x7f86c007be90 2023-08-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.565+0000 7f86bdd9b700 1 -- 172.21.15.99:0/1475685351 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b0011a00 con 0x7f86c007be90 2023-08-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.566+0000 7f86c50f1700 1 -- 172.21.15.99:0/1475685351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 msgr2=0x7f86c007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.566+0000 7f86c50f1700 1 --2- 172.21.15.99:0/1475685351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c007a2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f86b00098a0 tx=0x7f86b0009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.566+0000 7f86c50f1700 1 -- 172.21.15.99:0/1475685351 shutdown_connections 2023-08-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.566+0000 7f86c50f1700 1 --2- 172.21.15.99:0/1475685351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c007a2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:32.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.566+0000 7f86c50f1700 1 -- 172.21.15.99:0/1475685351 >> 172.21.15.99:0/1475685351 conn(0x7f86c01016e0 msgr2=0x7f86c0103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:32.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.566+0000 7f86c50f1700 1 -- 172.21.15.99:0/1475685351 shutdown_connections 2023-08-25T08:50:32.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.566+0000 7f86c50f1700 1 -- 172.21.15.99:0/1475685351 wait complete. 2023-08-25T08:50:32.216 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.567+0000 7f86c50f1700 1 Processor -- start 2023-08-25T08:50:32.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.567+0000 7f86c50f1700 1 -- start start 2023-08-25T08:50:32.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.567+0000 7f86c50f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:32.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.567+0000 7f86c50f1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c019c5f0 con 0x7f86c007be90 2023-08-25T08:50:32.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.568+0000 7f86bed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:32.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.568+0000 7f86bed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c019c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46860/0 (socket says 172.21.15.99:46860) 2023-08-25T08:50:32.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.568+0000 7f86bed9d700 1 -- 172.21.15.99:0/1266231639 learned_addr learned my addr 172.21.15.99:0/1266231639 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:32.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.568+0000 7f86bed9d700 1 -- 172.21.15.99:0/1266231639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0009550 con 0x7f86c007be90 2023-08-25T08:50:32.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.568+0000 7f86bed9d700 1 --2- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c019c0b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009130 tx=0x7f86b0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:32.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.568+0000 7f86a7fff700 1 -- 172.21.15.99:0/1266231639 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b0003ab0 con 0x7f86c007be90 2023-08-25T08:50:32.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.568+0000 7f86a7fff700 1 -- 172.21.15.99:0/1266231639 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86b0003c10 con 0x7f86c007be90 2023-08-25T08:50:32.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.568+0000 7f86a7fff700 1 -- 172.21.15.99:0/1266231639 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b00247d0 con 0x7f86c007be90 2023-08-25T08:50:32.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.569+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c019c7f0 con 0x7f86c007be90 2023-08-25T08:50:32.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.569+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c019cc10 con 0x7f86c007be90 2023-08-25T08:50:32.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.570+0000 7f86a7fff700 1 -- 172.21.15.99:0/1266231639 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f86b001abb0 con 0x7f86c007be90 2023-08-25T08:50:32.219 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.570+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c0196100 con 0x7f86c007be90 2023-08-25T08:50:32.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.570+0000 7f86a7fff700 1 --2- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f86a8038160 0x7f86a803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:32.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.570+0000 7f86a7fff700 1 -- 172.21.15.99:0/1266231639 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f86b004c0f0 con 0x7f86c007be90 2023-08-25T08:50:32.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.570+0000 7f86be59c700 1 --2- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f86a8038160 0x7f86a803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:32.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.571+0000 7f86be59c700 1 --2- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f86a8038160 0x7f86a803a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86b4006fd0 tx=0x7f86b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:32.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.575+0000 7f86a7fff700 1 -- 172.21.15.99:0/1266231639 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f86b007ee00 con 0x7f86c007be90 2023-08-25T08:50:32.220 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.772+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f86c002cd20 con 0x7f86a8038160 2023-08-25T08:50:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.773+0000 7f86a7fff700 1 -- 172.21.15.99:0/1266231639 <== mgr.14118 v2:172.21.15.99:6800/2336486699 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f86c002cd20 con 0x7f86a8038160 2023-08-25T08:50:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f86a8038160 msgr2=0x7f86a803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 --2- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f86a8038160 0x7f86a803a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86b4006fd0 tx=0x7f86b4006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 msgr2=0x7f86c019c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 --2- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c019c0b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009130 tx=0x7f86b0011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 shutdown_connections 2023-08-25T08:50:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 --2- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f86a8038160 0x7f86a803a620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 --2- 172.21.15.99:0/1266231639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c007be90 0x7f86c019c0b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:32.222 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 >> 172.21.15.99:0/1266231639 conn(0x7f86c01016e0 msgr2=0x7f86c0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:32.222 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 shutdown_connections 2023-08-25T08:50:32.222 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:31.775+0000 7f86c50f1700 1 -- 172.21.15.99:0/1266231639 wait complete. 2023-08-25T08:50:32.222 INFO:teuthology.orchestra.run.smithi099.stdout:Generating ssh key... 2023-08-25T08:50:32.463 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:32 smithi099 ceph-mon[97641]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:33.455 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: Generating public/private rsa key pair. 2023-08-25T08:50:33.456 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: Your identification has been saved in /tmp/tmp2it4ngh3/key. 2023-08-25T08:50:33.456 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: Your public key has been saved in /tmp/tmp2it4ngh3/key.pub. 2023-08-25T08:50:33.456 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: The key fingerprint is: 2023-08-25T08:50:33.456 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: SHA256:cJ26FuD0bskzktpgkoXATfMU0pVRZhlT83jsj2puqh8 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:50:33.456 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: The key's randomart image is: 2023-08-25T08:50:33.456 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: +---[RSA 3072]----+ 2023-08-25T08:50:33.457 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: | +.ooo+*+o | 2023-08-25T08:50:33.457 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: |. o =. .oo..= | 2023-08-25T08:50:33.457 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: |.. . .+ . o. + | 2023-08-25T08:50:33.457 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: | . . o = . o | 2023-08-25T08:50:33.457 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: | . . . S . | 2023-08-25T08:50:33.457 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: | o + + o | 2023-08-25T08:50:33.457 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: | o o o XE . . | 2023-08-25T08:50:33.457 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: | o + + o.o. | 2023-08-25T08:50:33.458 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: | . ..oo=o | 2023-08-25T08:50:33.458 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: +----[SHA256]-----+ 2023-08-25T08:50:33.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.709+0000 7fb44158c700 1 Processor -- start 2023-08-25T08:50:33.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.709+0000 7fb44158c700 1 -- start start 2023-08-25T08:50:33.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.709+0000 7fb44158c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:33.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.709+0000 7fb44158c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c07a8c0 con 0x7fb43c07be90 2023-08-25T08:50:33.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.710+0000 7fb43affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:33.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.710+0000 7fb43affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46868/0 (socket says 172.21.15.99:46868) 2023-08-25T08:50:33.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.710+0000 7fb43affd700 1 -- 172.21.15.99:0/3538684992 learned_addr learned my addr 172.21.15.99:0/3538684992 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:33.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.710+0000 7fb43affd700 1 -- 172.21.15.99:0/3538684992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c07aa00 con 0x7fb43c07be90 2023-08-25T08:50:33.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.710+0000 7fb43affd700 1 --2- 172.21.15.99:0/3538684992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c07a2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0098a0 tx=0x7fb42c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f86fcf790791e37 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:33.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb439ffb700 1 -- 172.21.15.99:0/3538684992 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb42c004160 con 0x7fb43c07be90 2023-08-25T08:50:33.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb439ffb700 1 -- 172.21.15.99:0/3538684992 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb42c0042c0 con 0x7fb43c07be90 2023-08-25T08:50:33.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb439ffb700 1 -- 172.21.15.99:0/3538684992 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb42c011a00 con 0x7fb43c07be90 2023-08-25T08:50:33.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb44158c700 1 -- 172.21.15.99:0/3538684992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 msgr2=0x7fb43c07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:33.590 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb44158c700 1 --2- 172.21.15.99:0/3538684992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c07a2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0098a0 tx=0x7fb42c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:33.590 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb44158c700 1 -- 172.21.15.99:0/3538684992 shutdown_connections 2023-08-25T08:50:33.590 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb44158c700 1 --2- 172.21.15.99:0/3538684992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c07a2f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:33.590 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb44158c700 1 -- 172.21.15.99:0/3538684992 >> 172.21.15.99:0/3538684992 conn(0x7fb43c1016e0 msgr2=0x7fb43c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:33.590 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb44158c700 1 -- 172.21.15.99:0/3538684992 shutdown_connections 2023-08-25T08:50:33.590 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.711+0000 7fb44158c700 1 -- 172.21.15.99:0/3538684992 wait complete. 2023-08-25T08:50:33.590 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.712+0000 7fb44158c700 1 Processor -- start 2023-08-25T08:50:33.591 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.712+0000 7fb44158c700 1 -- start start 2023-08-25T08:50:33.591 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.713+0000 7fb44158c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:33.591 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.713+0000 7fb44158c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c19c5f0 con 0x7fb43c07be90 2023-08-25T08:50:33.591 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.713+0000 7fb43affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:33.591 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.713+0000 7fb43affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c19c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46882/0 (socket says 172.21.15.99:46882) 2023-08-25T08:50:33.592 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.713+0000 7fb43affd700 1 -- 172.21.15.99:0/2561537280 learned_addr learned my addr 172.21.15.99:0/2561537280 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:33.592 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.713+0000 7fb43affd700 1 -- 172.21.15.99:0/2561537280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c009550 con 0x7fb43c07be90 2023-08-25T08:50:33.592 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.714+0000 7fb43affd700 1 --2- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c19c0b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009fb0 tx=0x7fb42c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:33.592 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.714+0000 7fb423fff700 1 -- 172.21.15.99:0/2561537280 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb42c003a40 con 0x7fb43c07be90 2023-08-25T08:50:33.592 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.714+0000 7fb423fff700 1 -- 172.21.15.99:0/2561537280 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb42c003ba0 con 0x7fb43c07be90 2023-08-25T08:50:33.592 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.714+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c19c7f0 con 0x7fb43c07be90 2023-08-25T08:50:33.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.714+0000 7fb423fff700 1 -- 172.21.15.99:0/2561537280 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb42c01a750 con 0x7fb43c07be90 2023-08-25T08:50:33.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.714+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c19cc10 con 0x7fb43c07be90 2023-08-25T08:50:33.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.715+0000 7fb423fff700 1 -- 172.21.15.99:0/2561537280 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fb42c024b60 con 0x7fb43c07be90 2023-08-25T08:50:33.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.715+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c196100 con 0x7fb43c07be90 2023-08-25T08:50:33.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.715+0000 7fb423fff700 1 --2- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb424038110 0x7fb42403a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:33.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.715+0000 7fb423fff700 1 -- 172.21.15.99:0/2561537280 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb42c04bfc0 con 0x7fb43c07be90 2023-08-25T08:50:33.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.716+0000 7fb43a7fc700 1 --2- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb424038110 0x7fb42403a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:33.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.716+0000 7fb43a7fc700 1 --2- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb424038110 0x7fb42403a5d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb430006fd0 tx=0x7fb430006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:33.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.720+0000 7fb423fff700 1 -- 172.21.15.99:0/2561537280 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb42c024430 con 0x7fb43c07be90 2023-08-25T08:50:33.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:32.916+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb43c02cd20 con 0x7fb424038110 2023-08-25T08:50:33.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.082+0000 7fb423fff700 1 -- 172.21.15.99:0/2561537280 <== mgr.14118 v2:172.21.15.99:6800/2336486699 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb43c02cd20 con 0x7fb424038110 2023-08-25T08:50:33.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb424038110 msgr2=0x7fb42403a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:33.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 --2- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb424038110 0x7fb42403a5d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb430006fd0 tx=0x7fb430006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:33.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 msgr2=0x7fb43c19c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:33.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 --2- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c19c0b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009fb0 tx=0x7fb42c011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:33.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 shutdown_connections 2023-08-25T08:50:33.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 --2- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb424038110 0x7fb42403a5d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:33.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 --2- 172.21.15.99:0/2561537280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb43c07be90 0x7fb43c19c0b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:33.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 >> 172.21.15.99:0/2561537280 conn(0x7fb43c1016e0 msgr2=0x7fb43c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:33.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 shutdown_connections 2023-08-25T08:50:33.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:33.084+0000 7fb44158c700 1 -- 172.21.15.99:0/2561537280 wait complete. 2023-08-25T08:50:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-mon[97641]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-mon[97641]: Generating ssh key... 2023-08-25T08:50:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:33 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:34.672 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9EUT+ijUfNGmrLF9rb81WnRaSi7HTCONvfh+nzI9qFlmVa4ESSP6QUC9dIDZjKTqfrlKTOGyyIV+HMwi7DH/Z4B+gvDirB8SuodrfM2CUhTMbUKmBMmPnjktZNKGItNpc+Xg9qDbVWk4XjfQ/p1buvVVjJYWi6c5N+qHHt6nk8acFZAxTn41p71swBUMe4BpKaSM2aS3acVyn3dF/XNl0aPcULtd6YJlZbvaFsR/rOIg+GlM8f6KLdfT55xSb6wmE/yLS2VQQ6Wy6SUJUqXEzV3jBmBSrTz+MmEzXgyBy48xrLQaseyyU1waT65ZUSJ25uhssBjYzyQkVwOF98jUd1FzFrz+CtdrAiu5k21yIJ76dJgF8dYBiqTxU/R0WJleVHpim8xC7WKN7+YNhG4ckh1ZYVsEc5lR0ywjM4v3fWQbp2TXDDFYypDuWg6A51Wy2XSj2w/jYhozN+Aunf3SXbIcJ4vDRrMvsmHXcoBHMXTWvNOPE5uR4e3cvZ6sJNYs= ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:50:34.672 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.093+0000 7f793acec700 1 Processor -- start 2023-08-25T08:50:34.673 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.093+0000 7f793acec700 1 -- start start 2023-08-25T08:50:34.673 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.093+0000 7f793acec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:34.673 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.093+0000 7f793acec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793407a8e0 con 0x7f793407beb0 2023-08-25T08:50:34.673 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.094+0000 7f7938a88700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:34.674 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.094+0000 7f7938a88700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46884/0 (socket says 172.21.15.99:46884) 2023-08-25T08:50:34.674 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.094+0000 7f7938a88700 1 -- 172.21.15.99:0/1321555599 learned_addr learned my addr 172.21.15.99:0/1321555599 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:34.674 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.094+0000 7f7938a88700 1 -- 172.21.15.99:0/1321555599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793407aa20 con 0x7f793407beb0 2023-08-25T08:50:34.674 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.094+0000 7f7938a88700 1 --2- 172.21.15.99:0/1321555599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793407a310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f79280098a0 tx=0x7f7928009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df779312c8f7854 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:34.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f79337fe700 1 -- 172.21.15.99:0/1321555599 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7928004160 con 0x7f793407beb0 2023-08-25T08:50:34.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f79337fe700 1 -- 172.21.15.99:0/1321555599 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79280042c0 con 0x7f793407beb0 2023-08-25T08:50:34.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f79337fe700 1 -- 172.21.15.99:0/1321555599 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7928011a00 con 0x7f793407beb0 2023-08-25T08:50:34.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f793acec700 1 -- 172.21.15.99:0/1321555599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 msgr2=0x7f793407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:34.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f793acec700 1 --2- 172.21.15.99:0/1321555599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793407a310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f79280098a0 tx=0x7f7928009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:34.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f793acec700 1 -- 172.21.15.99:0/1321555599 shutdown_connections 2023-08-25T08:50:34.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f793acec700 1 --2- 172.21.15.99:0/1321555599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793407a310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:34.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f793acec700 1 -- 172.21.15.99:0/1321555599 >> 172.21.15.99:0/1321555599 conn(0x7f79341016e0 msgr2=0x7f7934103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:34.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f793acec700 1 -- 172.21.15.99:0/1321555599 shutdown_connections 2023-08-25T08:50:34.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.095+0000 7f793acec700 1 -- 172.21.15.99:0/1321555599 wait complete. 2023-08-25T08:50:34.682 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.096+0000 7f793acec700 1 Processor -- start 2023-08-25T08:50:34.682 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.096+0000 7f793acec700 1 -- start start 2023-08-25T08:50:34.682 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.097+0000 7f793acec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:34.683 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.097+0000 7f793acec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793419c5d0 con 0x7f793407beb0 2023-08-25T08:50:34.683 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.097+0000 7f7938a88700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:34.683 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.097+0000 7f7938a88700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793419c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46886/0 (socket says 172.21.15.99:46886) 2023-08-25T08:50:34.683 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.097+0000 7f7938a88700 1 -- 172.21.15.99:0/1438458617 learned_addr learned my addr 172.21.15.99:0/1438458617 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:34.684 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.097+0000 7f7938a88700 1 -- 172.21.15.99:0/1438458617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928009550 con 0x7f793407beb0 2023-08-25T08:50:34.684 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.097+0000 7f7938a88700 1 --2- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793419c090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7928009130 tx=0x7f7928011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:34.684 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.098+0000 7f7931ffb700 1 -- 172.21.15.99:0/1438458617 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7928003ab0 con 0x7f793407beb0 2023-08-25T08:50:34.685 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.098+0000 7f7931ffb700 1 -- 172.21.15.99:0/1438458617 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7928003c10 con 0x7f793407beb0 2023-08-25T08:50:34.685 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.098+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793419c7d0 con 0x7f793407beb0 2023-08-25T08:50:34.685 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.098+0000 7f7931ffb700 1 -- 172.21.15.99:0/1438458617 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79280247d0 con 0x7f793407beb0 2023-08-25T08:50:34.685 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.098+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793419cbf0 con 0x7f793407beb0 2023-08-25T08:50:34.686 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.099+0000 7f7931ffb700 1 -- 172.21.15.99:0/1438458617 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f792801a430 con 0x7f793407beb0 2023-08-25T08:50:34.686 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.099+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7934196100 con 0x7f793407beb0 2023-08-25T08:50:34.686 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.099+0000 7f7931ffb700 1 --2- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f7924038160 0x7f792403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:34.687 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.099+0000 7f7931ffb700 1 -- 172.21.15.99:0/1438458617 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f792804c0c0 con 0x7f793407beb0 2023-08-25T08:50:34.687 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.100+0000 7f7933fff700 1 --2- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f7924038160 0x7f792403a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:34.687 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.100+0000 7f7933fff700 1 --2- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f7924038160 0x7f792403a620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7920006fd0 tx=0x7f7920006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:34.687 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.105+0000 7f7931ffb700 1 -- 172.21.15.99:0/1438458617 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f792807ee00 con 0x7f793407beb0 2023-08-25T08:50:34.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.303+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f793402cd20 con 0x7f7924038160 2023-08-25T08:50:34.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.304+0000 7f7931ffb700 1 -- 172.21.15.99:0/1438458617 <== mgr.14118 v2:172.21.15.99:6800/2336486699 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f793402cd20 con 0x7f7924038160 2023-08-25T08:50:34.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f7924038160 msgr2=0x7f792403a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:34.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 --2- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f7924038160 0x7f792403a620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7920006fd0 tx=0x7f7920006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:34.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 msgr2=0x7f793419c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:34.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 --2- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793419c090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7928009130 tx=0x7f7928011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:34.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 shutdown_connections 2023-08-25T08:50:34.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 --2- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f7924038160 0x7f792403a620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:34.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 --2- 172.21.15.99:0/1438458617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f793407beb0 0x7f793419c090 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:34.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 >> 172.21.15.99:0/1438458617 conn(0x7f79341016e0 msgr2=0x7f7934102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:34.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 shutdown_connections 2023-08-25T08:50:34.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:34.306+0000 7f793acec700 1 -- 172.21.15.99:0/1438458617 wait complete. 2023-08-25T08:50:34.692 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-08-25T08:50:34.692 INFO:teuthology.orchestra.run.smithi099.stdout:Adding key to root@localhost authorized_keys... 2023-08-25T08:50:34.693 INFO:teuthology.orchestra.run.smithi099.stdout:Adding host smithi099... 2023-08-25T08:50:35.422 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:35 smithi099 ceph-mon[97641]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:36.444 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:36 smithi099 ceph-mon[97641]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:37.561 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:37 smithi099 ceph-mon[97641]: Deploying cephadm binary to smithi099 2023-08-25T08:50:38.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Added host 'smithi099' with addr '172.21.15.99' 2023-08-25T08:50:38.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.180+0000 7f3831219700 1 Processor -- start 2023-08-25T08:50:38.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.181+0000 7f3831219700 1 -- start start 2023-08-25T08:50:38.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.181+0000 7f3831219700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c07a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.181+0000 7f3831219700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c07a880 con 0x7f382c07be50 2023-08-25T08:50:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.181+0000 7f382ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c07a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.181+0000 7f382ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c07a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46892/0 (socket says 172.21.15.99:46892) 2023-08-25T08:50:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.181+0000 7f382ad9d700 1 -- 172.21.15.99:0/303427730 learned_addr learned my addr 172.21.15.99:0/303427730 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.182+0000 7f382ad9d700 1 -- 172.21.15.99:0/303427730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c07a9c0 con 0x7f382c07be50 2023-08-25T08:50:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.182+0000 7f382ad9d700 1 --2- 172.21.15.99:0/303427730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c07a2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f381c0098a0 tx=0x7f381c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eab7ece7f931ae server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.182+0000 7f3829d9b700 1 -- 172.21.15.99:0/303427730 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f381c004160 con 0x7f382c07be50 2023-08-25T08:50:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.182+0000 7f3829d9b700 1 -- 172.21.15.99:0/303427730 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f381c0042c0 con 0x7f382c07be50 2023-08-25T08:50:38.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.182+0000 7f3829d9b700 1 -- 172.21.15.99:0/303427730 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f381c011a00 con 0x7f382c07be50 2023-08-25T08:50:38.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.183+0000 7f3831219700 1 -- 172.21.15.99:0/303427730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 msgr2=0x7f382c07a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:38.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.183+0000 7f3831219700 1 --2- 172.21.15.99:0/303427730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c07a2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f381c0098a0 tx=0x7f381c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:38.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.183+0000 7f3831219700 1 -- 172.21.15.99:0/303427730 shutdown_connections 2023-08-25T08:50:38.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.183+0000 7f3831219700 1 --2- 172.21.15.99:0/303427730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c07a2b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:38.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.183+0000 7f3831219700 1 -- 172.21.15.99:0/303427730 >> 172.21.15.99:0/303427730 conn(0x7f382c1016e0 msgr2=0x7f382c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:38.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.183+0000 7f3831219700 1 -- 172.21.15.99:0/303427730 shutdown_connections 2023-08-25T08:50:38.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.183+0000 7f3831219700 1 -- 172.21.15.99:0/303427730 wait complete. 2023-08-25T08:50:38.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.184+0000 7f3831219700 1 Processor -- start 2023-08-25T08:50:38.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.184+0000 7f3831219700 1 -- start start 2023-08-25T08:50:38.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.184+0000 7f3831219700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:38.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.184+0000 7f3831219700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c19c5c0 con 0x7f382c07be50 2023-08-25T08:50:38.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.185+0000 7f382ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:38.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.185+0000 7f382ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c19c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46894/0 (socket says 172.21.15.99:46894) 2023-08-25T08:50:38.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.185+0000 7f382ad9d700 1 -- 172.21.15.99:0/2386032670 learned_addr learned my addr 172.21.15.99:0/2386032670 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:38.961 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.185+0000 7f382ad9d700 1 -- 172.21.15.99:0/2386032670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c009550 con 0x7f382c07be50 2023-08-25T08:50:38.961 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.185+0000 7f382ad9d700 1 --2- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c19c080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f381c009130 tx=0x7f381c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:38.961 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.185+0000 7f3813fff700 1 -- 172.21.15.99:0/2386032670 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f381c003b70 con 0x7f382c07be50 2023-08-25T08:50:38.961 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.185+0000 7f3813fff700 1 -- 172.21.15.99:0/2386032670 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f381c003cd0 con 0x7f382c07be50 2023-08-25T08:50:38.961 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.185+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c19c7c0 con 0x7f382c07be50 2023-08-25T08:50:38.961 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.186+0000 7f3813fff700 1 -- 172.21.15.99:0/2386032670 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f381c0197d0 con 0x7f382c07be50 2023-08-25T08:50:38.961 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.186+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c19cbe0 con 0x7f382c07be50 2023-08-25T08:50:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.187+0000 7f3813fff700 1 -- 172.21.15.99:0/2386032670 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f381c01abb0 con 0x7f382c07be50 2023-08-25T08:50:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.187+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c1960f0 con 0x7f382c07be50 2023-08-25T08:50:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.187+0000 7f3813fff700 1 --2- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f3814038160 0x7f381403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.187+0000 7f3813fff700 1 -- 172.21.15.99:0/2386032670 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f381c04c200 con 0x7f382c07be50 2023-08-25T08:50:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.187+0000 7f382a59c700 1 --2- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f3814038160 0x7f381403a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.188+0000 7f382a59c700 1 --2- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f3814038160 0x7f381403a620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3820006fd0 tx=0x7f3820006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.192+0000 7f3813fff700 1 -- 172.21.15.99:0/2386032670 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f381c01a430 con 0x7f382c07be50 2023-08-25T08:50:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:35.395+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}) v1 -- 0x7f382c02cce0 con 0x7f3814038160 2023-08-25T08:50:38.963 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.558+0000 7f3813fff700 1 -- 172.21.15.99:0/2386032670 <== mgr.14118 v2:172.21.15.99:6800/2336486699 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f382c02cce0 con 0x7f3814038160 2023-08-25T08:50:38.963 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f3814038160 msgr2=0x7f381403a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:38.963 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 --2- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f3814038160 0x7f381403a620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3820006fd0 tx=0x7f3820006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:38.963 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 msgr2=0x7f382c19c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:38.963 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 --2- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c19c080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f381c009130 tx=0x7f381c011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:38.964 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 shutdown_connections 2023-08-25T08:50:38.964 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 --2- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f3814038160 0x7f381403a620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:38.964 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 --2- 172.21.15.99:0/2386032670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f382c07be50 0x7f382c19c080 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:38.964 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 >> 172.21.15.99:0/2386032670 conn(0x7f382c1016e0 msgr2=0x7f382c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:38.964 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 shutdown_connections 2023-08-25T08:50:38.964 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:38.561+0000 7f3831219700 1 -- 172.21.15.99:0/2386032670 wait complete. 2023-08-25T08:50:38.964 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mon service... 2023-08-25T08:50:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:39 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:39 smithi099 ceph-mon[97641]: Added host smithi099 2023-08-25T08:50:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:39 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:50:40.166 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-08-25T08:50:40.166 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.483+0000 7f1b24cc5700 1 Processor -- start 2023-08-25T08:50:40.166 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.483+0000 7f1b24cc5700 1 -- start start 2023-08-25T08:50:40.167 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.483+0000 7f1b24cc5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:40.167 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.483+0000 7f1b24cc5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2007a8e0 con 0x7f1b2007beb0 2023-08-25T08:50:40.167 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.484+0000 7f1b1e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:40.167 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.484+0000 7f1b1e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59992/0 (socket says 172.21.15.99:59992) 2023-08-25T08:50:40.167 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.484+0000 7f1b1e59c700 1 -- 172.21.15.99:0/3385161122 learned_addr learned my addr 172.21.15.99:0/3385161122 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:40.167 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.484+0000 7f1b1e59c700 1 -- 172.21.15.99:0/3385161122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2007aa20 con 0x7f1b2007beb0 2023-08-25T08:50:40.168 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.484+0000 7f1b1e59c700 1 --2- 172.21.15.99:0/3385161122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2007a310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b100098a0 tx=0x7f1b10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f4afcdf0b2f9b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:40.168 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.484+0000 7f1b1d59a700 1 -- 172.21.15.99:0/3385161122 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b10004160 con 0x7f1b2007beb0 2023-08-25T08:50:40.168 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.484+0000 7f1b1d59a700 1 -- 172.21.15.99:0/3385161122 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b100042c0 con 0x7f1b2007beb0 2023-08-25T08:50:40.168 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.484+0000 7f1b1d59a700 1 -- 172.21.15.99:0/3385161122 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b10011a00 con 0x7f1b2007beb0 2023-08-25T08:50:40.168 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.485+0000 7f1b24cc5700 1 -- 172.21.15.99:0/3385161122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 msgr2=0x7f1b2007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:40.169 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.485+0000 7f1b24cc5700 1 --2- 172.21.15.99:0/3385161122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2007a310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b100098a0 tx=0x7f1b10009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:40.169 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.485+0000 7f1b24cc5700 1 -- 172.21.15.99:0/3385161122 shutdown_connections 2023-08-25T08:50:40.169 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.485+0000 7f1b24cc5700 1 --2- 172.21.15.99:0/3385161122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2007a310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:40.169 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.485+0000 7f1b24cc5700 1 -- 172.21.15.99:0/3385161122 >> 172.21.15.99:0/3385161122 conn(0x7f1b201016e0 msgr2=0x7f1b20103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:40.169 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.485+0000 7f1b24cc5700 1 -- 172.21.15.99:0/3385161122 shutdown_connections 2023-08-25T08:50:40.169 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.485+0000 7f1b24cc5700 1 -- 172.21.15.99:0/3385161122 wait complete. 2023-08-25T08:50:40.169 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.486+0000 7f1b24cc5700 1 Processor -- start 2023-08-25T08:50:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.486+0000 7f1b24cc5700 1 -- start start 2023-08-25T08:50:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.487+0000 7f1b24cc5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2019c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.487+0000 7f1b24cc5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2019c5d0 con 0x7f1b2007beb0 2023-08-25T08:50:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.487+0000 7f1b1e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2019c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.487+0000 7f1b1e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2019c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60002/0 (socket says 172.21.15.99:60002) 2023-08-25T08:50:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.487+0000 7f1b1e59c700 1 -- 172.21.15.99:0/4207460865 learned_addr learned my addr 172.21.15.99:0/4207460865 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.487+0000 7f1b1e59c700 1 -- 172.21.15.99:0/4207460865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10009550 con 0x7f1b2007beb0 2023-08-25T08:50:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.488+0000 7f1b1e59c700 1 --2- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2019c090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009130 tx=0x7f1b10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:40.171 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.488+0000 7f1b0f7fe700 1 -- 172.21.15.99:0/4207460865 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b10003ab0 con 0x7f1b2007beb0 2023-08-25T08:50:40.171 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.488+0000 7f1b0f7fe700 1 -- 172.21.15.99:0/4207460865 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b10003c10 con 0x7f1b2007beb0 2023-08-25T08:50:40.171 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.488+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2019c7d0 con 0x7f1b2007beb0 2023-08-25T08:50:40.171 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.488+0000 7f1b0f7fe700 1 -- 172.21.15.99:0/4207460865 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b100247d0 con 0x7f1b2007beb0 2023-08-25T08:50:40.171 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.488+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2019cbf0 con 0x7f1b2007beb0 2023-08-25T08:50:40.171 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.489+0000 7f1b0f7fe700 1 -- 172.21.15.99:0/4207460865 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f1b1002d3f0 con 0x7f1b2007beb0 2023-08-25T08:50:40.171 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.489+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b20196100 con 0x7f1b2007beb0 2023-08-25T08:50:40.172 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.489+0000 7f1b0f7fe700 1 --2- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1b08038160 0x7f1b0803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:40.172 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.489+0000 7f1b0f7fe700 1 -- 172.21.15.99:0/4207460865 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1b1004c300 con 0x7f1b2007beb0 2023-08-25T08:50:40.172 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.490+0000 7f1b1dd9b700 1 --2- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1b08038160 0x7f1b0803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:40.172 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.490+0000 7f1b1dd9b700 1 --2- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1b08038160 0x7f1b0803a620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1b14006fd0 tx=0x7f1b14006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:40.172 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.495+0000 7f1b0f7fe700 1 -- 172.21.15.99:0/4207460865 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1b1001a610 con 0x7f1b2007beb0 2023-08-25T08:50:40.172 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.711+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1b2002cd20 con 0x7f1b08038160 2023-08-25T08:50:40.173 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.717+0000 7f1b0f7fe700 1 -- 172.21.15.99:0/4207460865 <== mgr.14118 v2:172.21.15.99:6800/2336486699 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1b2002cd20 con 0x7f1b08038160 2023-08-25T08:50:40.173 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1b08038160 msgr2=0x7f1b0803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:40.173 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 --2- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1b08038160 0x7f1b0803a620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1b14006fd0 tx=0x7f1b14006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:40.173 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 msgr2=0x7f1b2019c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:40.173 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 --2- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2019c090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009130 tx=0x7f1b10011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:40.174 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 shutdown_connections 2023-08-25T08:50:40.174 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 --2- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1b08038160 0x7f1b0803a620 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:40.174 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 --2- 172.21.15.99:0/4207460865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2007beb0 0x7f1b2019c090 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:40.174 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 >> 172.21.15.99:0/4207460865 conn(0x7f1b201016e0 msgr2=0x7f1b20102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:40.174 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 shutdown_connections 2023-08-25T08:50:40.174 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:39.720+0000 7f1b24cc5700 1 -- 172.21.15.99:0/4207460865 wait complete. 2023-08-25T08:50:40.174 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mgr service... 2023-08-25T08:50:40.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:40 smithi099 ceph-mon[97641]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:40.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:40 smithi099 ceph-mon[97641]: Saving service mon spec with placement count:5 2023-08-25T08:50:40.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:40 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:41.496 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-08-25T08:50:41.496 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.864+0000 7fb4a99b3700 1 Processor -- start 2023-08-25T08:50:41.497 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.865+0000 7fb4a99b3700 1 -- start start 2023-08-25T08:50:41.497 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.865+0000 7fb4a99b3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:41.497 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.865+0000 7fb4a99b3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a407a8e0 con 0x7fb4a407beb0 2023-08-25T08:50:41.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.865+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:41.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.865+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60018/0 (socket says 172.21.15.99:60018) 2023-08-25T08:50:41.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.865+0000 7fb4a2ffd700 1 -- 172.21.15.99:0/2508834141 learned_addr learned my addr 172.21.15.99:0/2508834141 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:41.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.866+0000 7fb4a2ffd700 1 -- 172.21.15.99:0/2508834141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a407aa20 con 0x7fb4a407beb0 2023-08-25T08:50:41.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.866+0000 7fb4a2ffd700 1 --2- 172.21.15.99:0/2508834141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a407a310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb4940098a0 tx=0x7fb494009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0102d3dabea4eb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:41.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.866+0000 7fb4a1ffb700 1 -- 172.21.15.99:0/2508834141 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb494004160 con 0x7fb4a407beb0 2023-08-25T08:50:41.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.866+0000 7fb4a1ffb700 1 -- 172.21.15.99:0/2508834141 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4940042c0 con 0x7fb4a407beb0 2023-08-25T08:50:41.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.866+0000 7fb4a1ffb700 1 -- 172.21.15.99:0/2508834141 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb494011a00 con 0x7fb4a407beb0 2023-08-25T08:50:41.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.867+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2508834141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 msgr2=0x7fb4a407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:41.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.867+0000 7fb4a99b3700 1 --2- 172.21.15.99:0/2508834141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a407a310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb4940098a0 tx=0x7fb494009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:41.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.867+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2508834141 shutdown_connections 2023-08-25T08:50:41.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.867+0000 7fb4a99b3700 1 --2- 172.21.15.99:0/2508834141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a407a310 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.867+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2508834141 >> 172.21.15.99:0/2508834141 conn(0x7fb4a41016e0 msgr2=0x7fb4a4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.867+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2508834141 shutdown_connections 2023-08-25T08:50:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.867+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2508834141 wait complete. 2023-08-25T08:50:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.868+0000 7fb4a99b3700 1 Processor -- start 2023-08-25T08:50:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.868+0000 7fb4a99b3700 1 -- start start 2023-08-25T08:50:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.869+0000 7fb4a99b3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.869+0000 7fb4a99b3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a419c5d0 con 0x7fb4a407beb0 2023-08-25T08:50:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.869+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:41.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.869+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a419c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60034/0 (socket says 172.21.15.99:60034) 2023-08-25T08:50:41.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.869+0000 7fb4a2ffd700 1 -- 172.21.15.99:0/2432123752 learned_addr learned my addr 172.21.15.99:0/2432123752 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:41.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.869+0000 7fb4a2ffd700 1 -- 172.21.15.99:0/2432123752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb494009550 con 0x7fb4a407beb0 2023-08-25T08:50:41.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.869+0000 7fb4a2ffd700 1 --2- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a419c090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb494009130 tx=0x7fb494011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:41.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.870+0000 7fb4a89b1700 1 -- 172.21.15.99:0/2432123752 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb494003ab0 con 0x7fb4a407beb0 2023-08-25T08:50:41.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.870+0000 7fb4a89b1700 1 -- 172.21.15.99:0/2432123752 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb494003c10 con 0x7fb4a407beb0 2023-08-25T08:50:41.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.870+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a419c7d0 con 0x7fb4a407beb0 2023-08-25T08:50:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.870+0000 7fb4a89b1700 1 -- 172.21.15.99:0/2432123752 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4940247d0 con 0x7fb4a407beb0 2023-08-25T08:50:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.870+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a419cbf0 con 0x7fb4a407beb0 2023-08-25T08:50:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.871+0000 7fb4a89b1700 1 -- 172.21.15.99:0/2432123752 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fb49402d3f0 con 0x7fb4a407beb0 2023-08-25T08:50:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.872+0000 7fb4a89b1700 1 --2- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb48c038160 0x7fb48c03a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.872+0000 7fb4a89b1700 1 -- 172.21.15.99:0/2432123752 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb49404d100 con 0x7fb4a407beb0 2023-08-25T08:50:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.872+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a4196100 con 0x7fb4a407beb0 2023-08-25T08:50:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.872+0000 7fb4a27fc700 1 --2- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb48c038160 0x7fb48c03a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.873+0000 7fb4a27fc700 1 --2- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb48c038160 0x7fb48c03a620 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb498006fd0 tx=0x7fb498006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:41.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:40.877+0000 7fb4a89b1700 1 -- 172.21.15.99:0/2432123752 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb494010970 con 0x7fb4a407beb0 2023-08-25T08:50:41.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.091+0000 7fb4a89b1700 1 -- 172.21.15.99:0/2432123752 <== mon.0 v2:172.21.15.99:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb494010970 con 0x7fb4a407beb0 2023-08-25T08:50:41.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.095+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb4a402cc80 con 0x7fb48c038160 2023-08-25T08:50:41.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.102+0000 7fb4a89b1700 1 -- 172.21.15.99:0/2432123752 <== mgr.14118 v2:172.21.15.99:6800/2336486699 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb4a402cc80 con 0x7fb48c038160 2023-08-25T08:50:41.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb48c038160 msgr2=0x7fb48c03a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:41.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 --2- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb48c038160 0x7fb48c03a620 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb498006fd0 tx=0x7fb498006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:41.505 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 msgr2=0x7fb4a419c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:41.505 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 --2- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a419c090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb494009130 tx=0x7fb494011ad0 comp rx=0 tx=0).stop 2023-08-25T08:50:41.505 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 shutdown_connections 2023-08-25T08:50:41.505 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 --2- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb48c038160 0x7fb48c03a620 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:41.505 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 --2- 172.21.15.99:0/2432123752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a407beb0 0x7fb4a419c090 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:41.505 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 >> 172.21.15.99:0/2432123752 conn(0x7fb4a41016e0 msgr2=0x7fb4a4102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:41.505 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 shutdown_connections 2023-08-25T08:50:41.506 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:41.105+0000 7fb4a99b3700 1 -- 172.21.15.99:0/2432123752 wait complete. 2023-08-25T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:42 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:42 smithi099 ceph-mon[97641]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:42 smithi099 ceph-mon[97641]: Saving service mgr spec with placement count:2 2023-08-25T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:42 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:42 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:42.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.076+0000 7f1e98226700 1 Processor -- start 2023-08-25T08:50:42.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.077+0000 7f1e98226700 1 -- start start 2023-08-25T08:50:42.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.077+0000 7f1e98226700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:42.873 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.077+0000 7f1e98226700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9007a8e0 con 0x7f1e9007beb0 2023-08-25T08:50:42.873 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.077+0000 7f1e95fc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:42.873 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.077+0000 7f1e95fc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60050/0 (socket says 172.21.15.99:60050) 2023-08-25T08:50:42.873 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.077+0000 7f1e95fc2700 1 -- 172.21.15.99:0/295504985 learned_addr learned my addr 172.21.15.99:0/295504985 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:42.873 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.078+0000 7f1e95fc2700 1 -- 172.21.15.99:0/295504985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9007aa20 con 0x7f1e9007beb0 2023-08-25T08:50:42.874 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.078+0000 7f1e95fc2700 1 --2- 172.21.15.99:0/295504985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9007a310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0098a0 tx=0x7f1e8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2385a24da77c20 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:42.874 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.078+0000 7f1e94fc0700 1 -- 172.21.15.99:0/295504985 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e8c004160 con 0x7f1e9007beb0 2023-08-25T08:50:42.874 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.078+0000 7f1e94fc0700 1 -- 172.21.15.99:0/295504985 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8c0042c0 con 0x7f1e9007beb0 2023-08-25T08:50:42.874 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.078+0000 7f1e94fc0700 1 -- 172.21.15.99:0/295504985 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e8c0119b0 con 0x7f1e9007beb0 2023-08-25T08:50:42.874 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.079+0000 7f1e98226700 1 -- 172.21.15.99:0/295504985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 msgr2=0x7f1e9007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:42.874 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.079+0000 7f1e98226700 1 --2- 172.21.15.99:0/295504985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9007a310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0098a0 tx=0x7f1e8c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:42.874 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.079+0000 7f1e98226700 1 -- 172.21.15.99:0/295504985 shutdown_connections 2023-08-25T08:50:42.874 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.079+0000 7f1e98226700 1 --2- 172.21.15.99:0/295504985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9007a310 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:42.875 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.079+0000 7f1e98226700 1 -- 172.21.15.99:0/295504985 >> 172.21.15.99:0/295504985 conn(0x7f1e901016e0 msgr2=0x7f1e90103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:42.875 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.079+0000 7f1e98226700 1 -- 172.21.15.99:0/295504985 shutdown_connections 2023-08-25T08:50:42.875 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.079+0000 7f1e98226700 1 -- 172.21.15.99:0/295504985 wait complete. 2023-08-25T08:50:42.875 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.080+0000 7f1e98226700 1 Processor -- start 2023-08-25T08:50:42.875 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.081+0000 7f1e98226700 1 -- start start 2023-08-25T08:50:42.875 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.081+0000 7f1e98226700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:42.875 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.081+0000 7f1e98226700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9019c5f0 con 0x7f1e9007beb0 2023-08-25T08:50:42.876 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.081+0000 7f1e95fc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:42.876 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.081+0000 7f1e95fc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9019c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60062/0 (socket says 172.21.15.99:60062) 2023-08-25T08:50:42.876 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.081+0000 7f1e95fc2700 1 -- 172.21.15.99:0/1863009784 learned_addr learned my addr 172.21.15.99:0/1863009784 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:42.876 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.082+0000 7f1e95fc2700 1 -- 172.21.15.99:0/1863009784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c009550 con 0x7f1e9007beb0 2023-08-25T08:50:42.876 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.082+0000 7f1e95fc2700 1 --2- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9019c0b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c003e60 tx=0x7f1e8c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:42.876 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.082+0000 7f1e86ffd700 1 -- 172.21.15.99:0/1863009784 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e8c003a60 con 0x7f1e9007beb0 2023-08-25T08:50:42.876 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.082+0000 7f1e86ffd700 1 -- 172.21.15.99:0/1863009784 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8c003bc0 con 0x7f1e9007beb0 2023-08-25T08:50:42.876 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.082+0000 7f1e86ffd700 1 -- 172.21.15.99:0/1863009784 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e8c0196c0 con 0x7f1e9007beb0 2023-08-25T08:50:42.877 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.082+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9019c7f0 con 0x7f1e9007beb0 2023-08-25T08:50:42.877 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.082+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9019cc10 con 0x7f1e9007beb0 2023-08-25T08:50:42.877 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.083+0000 7f1e86ffd700 1 -- 172.21.15.99:0/1863009784 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f1e8c01ab60 con 0x7f1e9007beb0 2023-08-25T08:50:42.877 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.083+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e90196120 con 0x7f1e9007beb0 2023-08-25T08:50:42.877 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.084+0000 7f1e86ffd700 1 --2- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1e7c038140 0x7f1e7c03a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:42.877 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.084+0000 7f1e86ffd700 1 -- 172.21.15.99:0/1863009784 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1e8c04c110 con 0x7f1e9007beb0 2023-08-25T08:50:42.877 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.084+0000 7f1e957c1700 1 --2- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1e7c038140 0x7f1e7c03a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:42.878 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.085+0000 7f1e957c1700 1 --2- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1e7c038140 0x7f1e7c03a600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e80006fd0 tx=0x7f1e80006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:42.878 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.088+0000 7f1e86ffd700 1 -- 172.21.15.99:0/1863009784 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1e8c018b00 con 0x7f1e9007beb0 2023-08-25T08:50:42.878 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.292+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f1e90061f50 con 0x7f1e9007beb0 2023-08-25T08:50:42.878 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.298+0000 7f1e86ffd700 1 -- 172.21.15.99:0/1863009784 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1e8c018b00 con 0x7f1e9007beb0 2023-08-25T08:50:42.878 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1e7c038140 msgr2=0x7f1e7c03a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:42.878 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 --2- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1e7c038140 0x7f1e7c03a600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e80006fd0 tx=0x7f1e80006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:42.878 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 msgr2=0x7f1e9019c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:42.878 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 --2- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9019c0b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c003e60 tx=0x7f1e8c011a30 comp rx=0 tx=0).stop 2023-08-25T08:50:42.879 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 shutdown_connections 2023-08-25T08:50:42.879 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 --2- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f1e7c038140 0x7f1e7c03a600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:42.879 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 --2- 172.21.15.99:0/1863009784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9007beb0 0x7f1e9019c0b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:42.879 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 >> 172.21.15.99:0/1863009784 conn(0x7f1e901016e0 msgr2=0x7f1e90102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:42.879 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 shutdown_connections 2023-08-25T08:50:42.879 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:42.300+0000 7f1e98226700 1 -- 172.21.15.99:0/1863009784 wait complete. 2023-08-25T08:50:43.561 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:43 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1863009784' entity='client.admin' 2023-08-25T08:50:44.050 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.461+0000 7f4531e82700 1 Processor -- start 2023-08-25T08:50:44.050 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.461+0000 7f4531e82700 1 -- start start 2023-08-25T08:50:44.050 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.461+0000 7f4531e82700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:44.050 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.461+0000 7f4531e82700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c07a8c0 con 0x7f452c07be90 2023-08-25T08:50:44.050 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.462+0000 7f452b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:44.051 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.462+0000 7f452b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60074/0 (socket says 172.21.15.99:60074) 2023-08-25T08:50:44.051 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.462+0000 7f452b7fe700 1 -- 172.21.15.99:0/4270484228 learned_addr learned my addr 172.21.15.99:0/4270484228 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:44.051 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.462+0000 7f452b7fe700 1 -- 172.21.15.99:0/4270484228 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c07aa00 con 0x7f452c07be90 2023-08-25T08:50:44.051 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.462+0000 7f452b7fe700 1 --2- 172.21.15.99:0/4270484228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c07a2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf783cd9400fe76 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:44.051 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.463+0000 7f452a7fc700 1 -- 172.21.15.99:0/4270484228 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f451c004160 con 0x7f452c07be90 2023-08-25T08:50:44.051 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.463+0000 7f452a7fc700 1 -- 172.21.15.99:0/4270484228 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451c0042c0 con 0x7f452c07be90 2023-08-25T08:50:44.052 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.463+0000 7f452a7fc700 1 -- 172.21.15.99:0/4270484228 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f451c0119b0 con 0x7f452c07be90 2023-08-25T08:50:44.052 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.463+0000 7f4531e82700 1 -- 172.21.15.99:0/4270484228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 msgr2=0x7f452c07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:44.052 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.463+0000 7f4531e82700 1 --2- 172.21.15.99:0/4270484228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c07a2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:44.052 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.464+0000 7f4531e82700 1 -- 172.21.15.99:0/4270484228 shutdown_connections 2023-08-25T08:50:44.052 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.464+0000 7f4531e82700 1 --2- 172.21.15.99:0/4270484228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c07a2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:44.052 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.464+0000 7f4531e82700 1 -- 172.21.15.99:0/4270484228 >> 172.21.15.99:0/4270484228 conn(0x7f452c1016e0 msgr2=0x7f452c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:44.052 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.464+0000 7f4531e82700 1 -- 172.21.15.99:0/4270484228 shutdown_connections 2023-08-25T08:50:44.053 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.464+0000 7f4531e82700 1 -- 172.21.15.99:0/4270484228 wait complete. 2023-08-25T08:50:44.053 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.465+0000 7f4531e82700 1 Processor -- start 2023-08-25T08:50:44.053 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.465+0000 7f4531e82700 1 -- start start 2023-08-25T08:50:44.053 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.465+0000 7f4531e82700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:44.053 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.465+0000 7f4531e82700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c19c610 con 0x7f452c07be90 2023-08-25T08:50:44.053 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.465+0000 7f452b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:44.053 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.465+0000 7f452b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c19c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60078/0 (socket says 172.21.15.99:60078) 2023-08-25T08:50:44.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.465+0000 7f452b7fe700 1 -- 172.21.15.99:0/1535642354 learned_addr learned my addr 172.21.15.99:0/1535642354 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:44.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.466+0000 7f452b7fe700 1 -- 172.21.15.99:0/1535642354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c009550 con 0x7f452c07be90 2023-08-25T08:50:44.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.466+0000 7f452b7fe700 1 --2- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c19c0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f451c003e60 tx=0x7f451c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:44.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.466+0000 7f4528ff9700 1 -- 172.21.15.99:0/1535642354 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f451c003a60 con 0x7f452c07be90 2023-08-25T08:50:44.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.466+0000 7f4528ff9700 1 -- 172.21.15.99:0/1535642354 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451c003bc0 con 0x7f452c07be90 2023-08-25T08:50:44.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.466+0000 7f4528ff9700 1 -- 172.21.15.99:0/1535642354 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f451c0196c0 con 0x7f452c07be90 2023-08-25T08:50:44.055 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.466+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c19c810 con 0x7f452c07be90 2023-08-25T08:50:44.055 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.466+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c19cc30 con 0x7f452c07be90 2023-08-25T08:50:44.055 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.467+0000 7f4528ff9700 1 -- 172.21.15.99:0/1535642354 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f451c01a430 con 0x7f452c07be90 2023-08-25T08:50:44.055 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.468+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c196120 con 0x7f452c07be90 2023-08-25T08:50:44.055 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.468+0000 7f4528ff9700 1 --2- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f4514038140 0x7f451403a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:44.055 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.468+0000 7f4528ff9700 1 -- 172.21.15.99:0/1535642354 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f451c04c1d0 con 0x7f452c07be90 2023-08-25T08:50:44.056 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.468+0000 7f452affd700 1 --2- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f4514038140 0x7f451403a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:44.056 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.469+0000 7f452affd700 1 --2- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f4514038140 0x7f451403a600 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4520006fd0 tx=0x7f4520006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:44.056 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.473+0000 7f4528ff9700 1 -- 172.21.15.99:0/1535642354 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f451c010380 con 0x7f452c07be90 2023-08-25T08:50:44.056 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.696+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f452c061f50 con 0x7f452c07be90 2023-08-25T08:50:44.057 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.702+0000 7f4528ff9700 1 -- 172.21.15.99:0/1535642354 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f451c010380 con 0x7f452c07be90 2023-08-25T08:50:44.057 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.704+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f4514038140 msgr2=0x7f451403a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:44.057 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.704+0000 7f4531e82700 1 --2- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f4514038140 0x7f451403a600 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4520006fd0 tx=0x7f4520006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:44.057 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.704+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 msgr2=0x7f452c19c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:44.057 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.704+0000 7f4531e82700 1 --2- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c19c0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f451c003e60 tx=0x7f451c011a30 comp rx=0 tx=0).stop 2023-08-25T08:50:44.057 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.704+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 shutdown_connections 2023-08-25T08:50:44.057 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.704+0000 7f4531e82700 1 --2- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f4514038140 0x7f451403a600 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:44.058 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.704+0000 7f4531e82700 1 --2- 172.21.15.99:0/1535642354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c07be90 0x7f452c19c0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:44.058 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.704+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 >> 172.21.15.99:0/1535642354 conn(0x7f452c1016e0 msgr2=0x7f452c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:44.058 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.705+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 shutdown_connections 2023-08-25T08:50:44.058 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:43.705+0000 7f4531e82700 1 -- 172.21.15.99:0/1535642354 wait complete. 2023-08-25T08:50:44.058 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling the dashboard module... 2023-08-25T08:50:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:44 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1535642354' entity='client.admin' 2023-08-25T08:50:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:44 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:46.023 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:45 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4197946036' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-08-25T08:50:46.023 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:45 smithi099 ceph-mon[97641]: from='mgr.14118 172.21.15.99:0/3969471178' entity='mgr.x' 2023-08-25T08:50:46.023 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:45 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: ignoring --setuser ceph since I am not root 2023-08-25T08:50:46.024 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:45 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: ignoring --setgroup ceph since I am not root 2023-08-25T08:50:46.024 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:46.022+0000 7fda64940200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T08:50:46.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.633+0000 7fb3ff74d700 1 Processor -- start 2023-08-25T08:50:46.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.633+0000 7fb3ff74d700 1 -- start start 2023-08-25T08:50:46.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.633+0000 7fb3ff74d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f807a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:46.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.633+0000 7fb3ff74d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f807a8a0 con 0x7fb3f807be70 2023-08-25T08:50:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.633+0000 7fb3fd4e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f807a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.633+0000 7fb3fd4e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f807a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60086/0 (socket says 172.21.15.99:60086) 2023-08-25T08:50:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.633+0000 7fb3fd4e9700 1 -- 172.21.15.99:0/4019242205 learned_addr learned my addr 172.21.15.99:0/4019242205 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.634+0000 7fb3fd4e9700 1 -- 172.21.15.99:0/4019242205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f807a9e0 con 0x7fb3f807be70 2023-08-25T08:50:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.634+0000 7fb3fd4e9700 1 --2- 172.21.15.99:0/4019242205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f807a2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40098a0 tx=0x7fb3f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a2a063971811d7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:46.144 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.634+0000 7fb3effff700 1 -- 172.21.15.99:0/4019242205 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3f4004160 con 0x7fb3f807be70 2023-08-25T08:50:46.144 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.634+0000 7fb3effff700 1 -- 172.21.15.99:0/4019242205 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3f40042c0 con 0x7fb3f807be70 2023-08-25T08:50:46.144 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.634+0000 7fb3effff700 1 -- 172.21.15.99:0/4019242205 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3f40119b0 con 0x7fb3f807be70 2023-08-25T08:50:46.144 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.635+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4019242205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 msgr2=0x7fb3f807a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:46.144 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.635+0000 7fb3ff74d700 1 --2- 172.21.15.99:0/4019242205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f807a2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40098a0 tx=0x7fb3f4009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:46.144 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.635+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4019242205 shutdown_connections 2023-08-25T08:50:46.144 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.635+0000 7fb3ff74d700 1 --2- 172.21.15.99:0/4019242205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f807a2d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:46.145 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.635+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4019242205 >> 172.21.15.99:0/4019242205 conn(0x7fb3f81016e0 msgr2=0x7fb3f8103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:46.145 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.635+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4019242205 shutdown_connections 2023-08-25T08:50:46.145 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.635+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4019242205 wait complete. 2023-08-25T08:50:46.145 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.636+0000 7fb3ff74d700 1 Processor -- start 2023-08-25T08:50:46.145 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.636+0000 7fb3ff74d700 1 -- start start 2023-08-25T08:50:46.145 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.637+0000 7fb3ff74d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f819c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:46.145 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.637+0000 7fb3ff74d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f819c5a0 con 0x7fb3f807be70 2023-08-25T08:50:46.145 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.637+0000 7fb3fd4e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f819c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:46.146 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.637+0000 7fb3fd4e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f819c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60098/0 (socket says 172.21.15.99:60098) 2023-08-25T08:50:46.146 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.637+0000 7fb3fd4e9700 1 -- 172.21.15.99:0/4197946036 learned_addr learned my addr 172.21.15.99:0/4197946036 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:46.146 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.637+0000 7fb3fd4e9700 1 -- 172.21.15.99:0/4197946036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009550 con 0x7fb3f807be70 2023-08-25T08:50:46.146 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.637+0000 7fb3fd4e9700 1 --2- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f819c060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4003f00 tx=0x7fb3f4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:46.146 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.638+0000 7fb3ee7fc700 1 -- 172.21.15.99:0/4197946036 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3f4003b20 con 0x7fb3f807be70 2023-08-25T08:50:46.146 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.638+0000 7fb3ee7fc700 1 -- 172.21.15.99:0/4197946036 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3f4003c80 con 0x7fb3f807be70 2023-08-25T08:50:46.146 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.638+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f819c7a0 con 0x7fb3f807be70 2023-08-25T08:50:46.147 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.638+0000 7fb3ee7fc700 1 -- 172.21.15.99:0/4197946036 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3f4019770 con 0x7fb3f807be70 2023-08-25T08:50:46.147 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.638+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f819cbc0 con 0x7fb3f807be70 2023-08-25T08:50:46.148 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.639+0000 7fb3ee7fc700 1 -- 172.21.15.99:0/4197946036 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fb3f401ab60 con 0x7fb3f807be70 2023-08-25T08:50:46.148 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.639+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f810c560 con 0x7fb3f807be70 2023-08-25T08:50:46.149 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.640+0000 7fb3ee7fc700 1 --2- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb3e4038160 0x7fb3e403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:46.149 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.640+0000 7fb3ee7fc700 1 -- 172.21.15.99:0/4197946036 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb3f404c230 con 0x7fb3f807be70 2023-08-25T08:50:46.149 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.640+0000 7fb3fcce8700 1 --2- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb3e4038160 0x7fb3e403a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:46.149 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.640+0000 7fb3fcce8700 1 --2- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb3e4038160 0x7fb3e403a620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006fd0 tx=0x7fb3e8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:46.149 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.645+0000 7fb3ee7fc700 1 -- 172.21.15.99:0/4197946036 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb3f401a430 con 0x7fb3f807be70 2023-08-25T08:50:46.149 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:44.908+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb3f810c7a0 con 0x7fb3f807be70 2023-08-25T08:50:46.149 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.712+0000 7fb3ee7fc700 1 -- 172.21.15.99:0/4197946036 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7fb3f402d350 con 0x7fb3f807be70 2023-08-25T08:50:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.714+0000 7fb3ee7fc700 1 -- 172.21.15.99:0/4197946036 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb3f40172b0 con 0x7fb3f807be70 2023-08-25T08:50:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb3e4038160 msgr2=0x7fb3e403a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 --2- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb3e4038160 0x7fb3e403a620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006fd0 tx=0x7fb3e8006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 msgr2=0x7fb3f819c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 --2- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f819c060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4003f00 tx=0x7fb3f4011a30 comp rx=0 tx=0).stop 2023-08-25T08:50:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 shutdown_connections 2023-08-25T08:50:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 --2- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7fb3e4038160 0x7fb3e403a620 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 --2- 172.21.15.99:0/4197946036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f807be70 0x7fb3f819c060 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 >> 172.21.15.99:0/4197946036 conn(0x7fb3f81016e0 msgr2=0x7fb3f8102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 shutdown_connections 2023-08-25T08:50:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:45.717+0000 7fb3ff74d700 1 -- 172.21.15.99:0/4197946036 wait complete. 2023-08-25T08:50:46.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:46.128+0000 7fda64940200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T08:50:46.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:46.261+0000 7fda64940200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T08:50:46.645 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:46.390+0000 7fda64940200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T08:50:46.921 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:46 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4197946036' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-08-25T08:50:46.921 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:46 smithi099 ceph-mon[97641]: mgrmap e9: x(active, since 17s) 2023-08-25T08:50:46.922 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:46.644+0000 7fda64940200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T08:50:47.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:47.139+0000 7fda64940200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T08:50:47.564 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:47.311+0000 7fda64940200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T08:50:47.564 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:47.438+0000 7fda64940200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T08:50:47.565 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:47.564+0000 7fda64940200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T08:50:47.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2023-08-25T08:50:47.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-08-25T08:50:47.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-25T08:50:47.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-08-25T08:50:47.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-08-25T08:50:47.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2023-08-25T08:50:47.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.708+0000 7f647ff8f700 1 Processor -- start 2023-08-25T08:50:47.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.708+0000 7f647ff8f700 1 -- start start 2023-08-25T08:50:47.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.708+0000 7f647ff8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647807a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:47.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.708+0000 7f647ff8f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647807a8a0 con 0x7f647807be70 2023-08-25T08:50:47.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.709+0000 7f647dd2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647807a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:47.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.709+0000 7f647dd2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647807a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60122/0 (socket says 172.21.15.99:60122) 2023-08-25T08:50:47.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.709+0000 7f647dd2b700 1 -- 172.21.15.99:0/2694224900 learned_addr learned my addr 172.21.15.99:0/2694224900 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:47.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.709+0000 7f647dd2b700 1 -- 172.21.15.99:0/2694224900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647807a9e0 con 0x7f647807be70 2023-08-25T08:50:47.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.709+0000 7f647dd2b700 1 --2- 172.21.15.99:0/2694224900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647807a2d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f64740098a0 tx=0x7f6474009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd684e99a06255fe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:47.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.709+0000 7f647cd29700 1 -- 172.21.15.99:0/2694224900 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6474004160 con 0x7f647807be70 2023-08-25T08:50:47.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647cd29700 1 -- 172.21.15.99:0/2694224900 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64740042c0 con 0x7f647807be70 2023-08-25T08:50:47.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647cd29700 1 -- 172.21.15.99:0/2694224900 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64740119b0 con 0x7f647807be70 2023-08-25T08:50:47.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647ff8f700 1 -- 172.21.15.99:0/2694224900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 msgr2=0x7f647807a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:47.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647ff8f700 1 --2- 172.21.15.99:0/2694224900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647807a2d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f64740098a0 tx=0x7f6474009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:47.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647ff8f700 1 -- 172.21.15.99:0/2694224900 shutdown_connections 2023-08-25T08:50:47.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647ff8f700 1 --2- 172.21.15.99:0/2694224900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647807a2d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:47.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647ff8f700 1 -- 172.21.15.99:0/2694224900 >> 172.21.15.99:0/2694224900 conn(0x7f64781016e0 msgr2=0x7f6478103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:47.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647ff8f700 1 -- 172.21.15.99:0/2694224900 shutdown_connections 2023-08-25T08:50:47.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.710+0000 7f647ff8f700 1 -- 172.21.15.99:0/2694224900 wait complete. 2023-08-25T08:50:47.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.711+0000 7f647ff8f700 1 Processor -- start 2023-08-25T08:50:47.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.712+0000 7f647ff8f700 1 -- start start 2023-08-25T08:50:47.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.712+0000 7f647ff8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647819c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:47.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.712+0000 7f647ff8f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647819c5a0 con 0x7f647807be70 2023-08-25T08:50:47.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.713+0000 7f647dd2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647819c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:47.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.713+0000 7f647dd2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647819c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60134/0 (socket says 172.21.15.99:60134) 2023-08-25T08:50:47.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.713+0000 7f647dd2b700 1 -- 172.21.15.99:0/2831935295 learned_addr learned my addr 172.21.15.99:0/2831935295 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:47.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.713+0000 7f647dd2b700 1 -- 172.21.15.99:0/2831935295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474009550 con 0x7f647807be70 2023-08-25T08:50:47.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.713+0000 7f647dd2b700 1 --2- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647819c060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6474003f00 tx=0x7f6474011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:47.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.714+0000 7f646effd700 1 -- 172.21.15.99:0/2831935295 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6474003b20 con 0x7f647807be70 2023-08-25T08:50:47.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.714+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647819c7a0 con 0x7f647807be70 2023-08-25T08:50:47.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.714+0000 7f646effd700 1 -- 172.21.15.99:0/2831935295 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6474003c80 con 0x7f647807be70 2023-08-25T08:50:47.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.714+0000 7f646effd700 1 -- 172.21.15.99:0/2831935295 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6474019770 con 0x7f647807be70 2023-08-25T08:50:47.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.714+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647819cbc0 con 0x7f647807be70 2023-08-25T08:50:47.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.715+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6478061f50 con 0x7f647807be70 2023-08-25T08:50:47.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.716+0000 7f646effd700 1 -- 172.21.15.99:0/2831935295 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f647401a430 con 0x7f647807be70 2023-08-25T08:50:47.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.716+0000 7f646effd700 1 --2- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f6464038160 0x7f646403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:47.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.716+0000 7f646effd700 1 -- 172.21.15.99:0/2831935295 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f647404c230 con 0x7f647807be70 2023-08-25T08:50:47.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.716+0000 7f647d52a700 1 -- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f6464038160 msgr2=0x7f646403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2336486699 2023-08-25T08:50:47.749 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.716+0000 7f647d52a700 1 --2- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f6464038160 0x7f646403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:50:47.749 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.721+0000 7f646effd700 1 -- 172.21.15.99:0/2831935295 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f647402d9a0 con 0x7f647807be70 2023-08-25T08:50:47.749 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.917+0000 7f647d52a700 1 -- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f6464038160 msgr2=0x7f646403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2336486699 2023-08-25T08:50:47.750 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:46.917+0000 7f647d52a700 1 --2- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f6464038160 0x7f646403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T08:50:47.750 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.018+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f647810c560 con 0x7f647807be70 2023-08-25T08:50:47.750 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.018+0000 7f646effd700 1 -- 172.21.15.99:0/2831935295 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f647402d350 con 0x7f647807be70 2023-08-25T08:50:47.750 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f6464038160 msgr2=0x7f646403a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:50:47.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 --2- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f6464038160 0x7f646403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:47.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 msgr2=0x7f647819c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:47.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 --2- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647819c060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6474003f00 tx=0x7f6474011a30 comp rx=0 tx=0).stop 2023-08-25T08:50:47.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 shutdown_connections 2023-08-25T08:50:47.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 --2- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f6464038160 0x7f646403a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:47.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 --2- 172.21.15.99:0/2831935295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647807be70 0x7f647819c060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:47.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 >> 172.21.15.99:0/2831935295 conn(0x7f64781016e0 msgr2=0x7f6478102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:47.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 shutdown_connections 2023-08-25T08:50:47.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:47.021+0000 7f647ff8f700 1 -- 172.21.15.99:0/2831935295 wait complete. 2023-08-25T08:50:47.753 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2023-08-25T08:50:47.753 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 9... 2023-08-25T08:50:47.964 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:47 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2831935295' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-08-25T08:50:48.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:48.163+0000 7fda64940200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T08:50:49.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:49 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:49.011+0000 7fda64940200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T08:50:49.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:49 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:49.115+0000 7fda64940200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T08:50:50.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:50 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:50.055+0000 7fda64940200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:50:50.313 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:50 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:50.172+0000 7fda64940200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T08:50:50.749 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:50 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:50.392+0000 7fda64940200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:50:50.749 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:50 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:50.748+0000 7fda64940200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T08:50:51.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:51.023+0000 7fda64940200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T08:50:51.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:51.214+0000 7fda64940200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T08:50:51.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:51.553+0000 7fda64940200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T08:50:52.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:52 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:52.466+0000 7fda64940200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T08:50:52.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:52 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:52.637+0000 7fda64940200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T08:50:53.296 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:52 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:52.882+0000 7fda64940200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T08:50:53.296 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:53 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:53.024+0000 7fda64940200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T08:50:53.296 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:53 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:53.295+0000 7fda64940200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T08:50:54.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:50:54.348+0000 7fda64940200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T08:50:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: Active manager daemon x restarted 2023-08-25T08:50:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: Activating manager daemon x 2023-08-25T08:50:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: osdmap e3: 0 total, 0 up, 0 in 2023-08-25T08:50:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: mgrmap e10: x(active, starting, since 0.00900631s) 2023-08-25T08:50:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:50:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T08:50:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T08:50:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T08:50:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T08:50:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: Manager daemon x is now available 2023-08-25T08:50:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:50:55.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:50:55.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T08:50:55.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T08:50:55.738 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2023-08-25T08:50:55.739 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-08-25T08:50:55.739 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2023-08-25T08:50:55.739 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2023-08-25T08:50:55.739 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.263+0000 7f8daa8c6700 1 Processor -- start 2023-08-25T08:50:55.739 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.263+0000 7f8daa8c6700 1 -- start start 2023-08-25T08:50:55.739 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.263+0000 7f8daa8c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da4106540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:55.739 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.263+0000 7f8daa8c6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da4106b10 con 0x7f8da4106120 2023-08-25T08:50:55.740 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.264+0000 7f8da3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da4106540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:55.740 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.264+0000 7f8da3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da4106540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37102/0 (socket says 172.21.15.99:37102) 2023-08-25T08:50:55.740 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.264+0000 7f8da3fff700 1 -- 172.21.15.99:0/2158416417 learned_addr learned my addr 172.21.15.99:0/2158416417 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:55.740 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.265+0000 7f8da3fff700 1 -- 172.21.15.99:0/2158416417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4107320 con 0x7f8da4106120 2023-08-25T08:50:55.740 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.265+0000 7f8da3fff700 1 --2- 172.21.15.99:0/2158416417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da4106540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d940098a0 tx=0x7f8d94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537b6ca4ca4732ed server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:55.740 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.265+0000 7f8da2ffd700 1 -- 172.21.15.99:0/2158416417 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d94004160 con 0x7f8da4106120 2023-08-25T08:50:55.740 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.265+0000 7f8da2ffd700 1 -- 172.21.15.99:0/2158416417 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d940042c0 con 0x7f8da4106120 2023-08-25T08:50:55.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.265+0000 7f8da2ffd700 1 -- 172.21.15.99:0/2158416417 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d940119b0 con 0x7f8da4106120 2023-08-25T08:50:55.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.266+0000 7f8daa8c6700 1 -- 172.21.15.99:0/2158416417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 msgr2=0x7f8da4106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:55.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.266+0000 7f8daa8c6700 1 --2- 172.21.15.99:0/2158416417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da4106540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d940098a0 tx=0x7f8d94009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:55.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.266+0000 7f8daa8c6700 1 -- 172.21.15.99:0/2158416417 shutdown_connections 2023-08-25T08:50:55.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.266+0000 7f8daa8c6700 1 --2- 172.21.15.99:0/2158416417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da4106540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:55.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.266+0000 7f8daa8c6700 1 -- 172.21.15.99:0/2158416417 >> 172.21.15.99:0/2158416417 conn(0x7f8da41016e0 msgr2=0x7f8da4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:55.741 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.266+0000 7f8daa8c6700 1 -- 172.21.15.99:0/2158416417 shutdown_connections 2023-08-25T08:50:55.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.266+0000 7f8daa8c6700 1 -- 172.21.15.99:0/2158416417 wait complete. 2023-08-25T08:50:55.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.267+0000 7f8daa8c6700 1 Processor -- start 2023-08-25T08:50:55.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.268+0000 7f8daa8c6700 1 -- start start 2023-08-25T08:50:55.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.268+0000 7f8daa8c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:55.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.268+0000 7f8daa8c6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da419c610 con 0x7f8da4106120 2023-08-25T08:50:55.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.268+0000 7f8da3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:55.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.268+0000 7f8da3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da419c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37118/0 (socket says 172.21.15.99:37118) 2023-08-25T08:50:55.742 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.268+0000 7f8da3fff700 1 -- 172.21.15.99:0/1855972116 learned_addr learned my addr 172.21.15.99:0/1855972116 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:55.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.269+0000 7f8da3fff700 1 -- 172.21.15.99:0/1855972116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d94009550 con 0x7f8da4106120 2023-08-25T08:50:55.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.269+0000 7f8da3fff700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da419c0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d94003e60 tx=0x7f8d94011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:55.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.269+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d94003a60 con 0x7f8da4106120 2023-08-25T08:50:55.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.269+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d94003bc0 con 0x7f8da4106120 2023-08-25T08:50:55.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.269+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da419c810 con 0x7f8da4106120 2023-08-25T08:50:55.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.269+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d940196c0 con 0x7f8da4106120 2023-08-25T08:50:55.743 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.269+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da419cc30 con 0x7f8da4106120 2023-08-25T08:50:55.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.271+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f8d9401ab60 con 0x7f8da4106120 2023-08-25T08:50:55.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.271+0000 7f8da17fa700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 0x7f8d8c03a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:55.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.271+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 --> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d8c03ad60 con 0x7f8d8c038190 2023-08-25T08:50:55.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.271+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8d9404c250 con 0x7f8da4106120 2023-08-25T08:50:55.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.271+0000 7f8da37fe700 1 -- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 msgr2=0x7f8d8c03a650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2336486699 2023-08-25T08:50:55.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.271+0000 7f8da37fe700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 0x7f8d8c03a650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:50:55.744 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.472+0000 7f8da37fe700 1 -- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 msgr2=0x7f8d8c03a650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2336486699 2023-08-25T08:50:55.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.472+0000 7f8da37fe700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 0x7f8d8c03a650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T08:50:55.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.873+0000 7f8da37fe700 1 -- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 msgr2=0x7f8d8c03a650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2336486699 2023-08-25T08:50:55.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:48.873+0000 7f8da37fe700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 0x7f8d8c03a650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-25T08:50:55.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:49.674+0000 7f8da37fe700 1 -- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 msgr2=0x7f8d8c03a650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2336486699 2023-08-25T08:50:55.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:49.674+0000 7f8da37fe700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 0x7f8d8c03a650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-25T08:50:55.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:51.275+0000 7f8da37fe700 1 -- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 msgr2=0x7f8d8c03a650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2336486699 2023-08-25T08:50:55.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:51.275+0000 7f8da37fe700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 0x7f8d8c03a650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-25T08:50:55.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:54.358+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mgrmap(e 10) v1 ==== 44915+0+0 (secure 0 0 0) 0x7f8d94019b10 con 0x7f8da4106120 2023-08-25T08:50:55.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:54.358+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 msgr2=0x7f8d8c03a650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:50:55.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:54.358+0000 7f8da17fa700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 0x7f8d8c03a650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:55.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.358+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 11) v1 ==== 45042+0+0 (secure 0 0 0) 0x7f8d9404d5a0 con 0x7f8da4106120 2023-08-25T08:50:55.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.358+0000 7f8da17fa700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8d8c03bc80 0x7f8d8c03e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:55.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.359+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d8c03ad60 con 0x7f8d8c03bc80 2023-08-25T08:50:55.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.359+0000 7f8da37fe700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8d8c03bc80 0x7f8d8c03e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:55.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.360+0000 7f8da37fe700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8d8c03bc80 0x7f8d8c03e070 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8d98003a10 tx=0x7f8d98009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.361+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f8d8c03ad60 con 0x7f8d8c03bc80 2023-08-25T08:50:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.370+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8da419d410 con 0x7f8d8c03bc80 2023-08-25T08:50:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.370+0000 7f8da17fa700 1 -- 172.21.15.99:0/1855972116 <== mgr.14150 v2:172.21.15.99:6800/4068336524 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f8da419d410 con 0x7f8d8c03bc80 2023-08-25T08:50:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8d8c03bc80 msgr2=0x7f8d8c03e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8d8c03bc80 0x7f8d8c03e070 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8d98003a10 tx=0x7f8d98009270 comp rx=0 tx=0).stop 2023-08-25T08:50:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 msgr2=0x7f8da419c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da419c0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d94003e60 tx=0x7f8d94011a30 comp rx=0 tx=0).stop 2023-08-25T08:50:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 shutdown_connections 2023-08-25T08:50:55.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8d8c03bc80 0x7f8d8c03e070 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:55.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da4106120 0x7f8da419c0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:55.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 --2- 172.21.15.99:0/1855972116 >> [v2:172.21.15.99:6800/2336486699,v1:172.21.15.99:6801/2336486699] conn(0x7f8d8c038190 0x7f8d8c03a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:55.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 >> 172.21.15.99:0/1855972116 conn(0x7f8da41016e0 msgr2=0x7f8da4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:55.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 shutdown_connections 2023-08-25T08:50:55.748 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:55.371+0000 7f8daa8c6700 1 -- 172.21.15.99:0/1855972116 wait complete. 2023-08-25T08:50:55.748 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 9 is available 2023-08-25T08:50:55.749 INFO:teuthology.orchestra.run.smithi099.stdout:Generating a dashboard self-signed certificate... 2023-08-25T08:50:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:56 smithi099 ceph-mon[97641]: mgrmap e11: x(active, since 1.01042s) 2023-08-25T08:50:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:56 smithi099 ceph-mon[97641]: [25/Aug/2023:08:50:55] ENGINE Bus STARTING 2023-08-25T08:50:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:56 smithi099 ceph-mon[97641]: [25/Aug/2023:08:50:55] ENGINE Serving on http://172.21.15.99:8765 2023-08-25T08:50:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:56 smithi099 ceph-mon[97641]: [25/Aug/2023:08:50:55] ENGINE Serving on https://172.21.15.99:7150 2023-08-25T08:50:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:56 smithi099 ceph-mon[97641]: [25/Aug/2023:08:50:55] ENGINE Bus STARTED 2023-08-25T08:50:57.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-08-25T08:50:57.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.505+0000 7f522a503700 1 Processor -- start 2023-08-25T08:50:57.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.505+0000 7f522a503700 1 -- start start 2023-08-25T08:50:57.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.505+0000 7f522a503700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f5224108730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:57.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.505+0000 7f522a503700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224108d00 con 0x7f5224108310 2023-08-25T08:50:57.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.506+0000 7f5223fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f5224108730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:57.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.506+0000 7f5223fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f5224108730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37210/0 (socket says 172.21.15.99:37210) 2023-08-25T08:50:57.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.506+0000 7f5223fff700 1 -- 172.21.15.99:0/2807848434 learned_addr learned my addr 172.21.15.99:0/2807848434 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:57.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.506+0000 7f5223fff700 1 -- 172.21.15.99:0/2807848434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224109510 con 0x7f5224108310 2023-08-25T08:50:57.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.506+0000 7f5223fff700 1 --2- 172.21.15.99:0/2807848434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f5224108730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f52140098a0 tx=0x7f5214009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f134281b699cf0c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:57.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f5222ffd700 1 -- 172.21.15.99:0/2807848434 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5214004160 con 0x7f5224108310 2023-08-25T08:50:57.320 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f5222ffd700 1 -- 172.21.15.99:0/2807848434 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52140042c0 con 0x7f5224108310 2023-08-25T08:50:57.320 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f5222ffd700 1 -- 172.21.15.99:0/2807848434 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52140119b0 con 0x7f5224108310 2023-08-25T08:50:57.323 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f522a503700 1 -- 172.21.15.99:0/2807848434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 msgr2=0x7f5224108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:57.323 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f522a503700 1 --2- 172.21.15.99:0/2807848434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f5224108730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f52140098a0 tx=0x7f5214009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:57.324 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f522a503700 1 -- 172.21.15.99:0/2807848434 shutdown_connections 2023-08-25T08:50:57.324 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f522a503700 1 --2- 172.21.15.99:0/2807848434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f5224108730 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:57.324 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f522a503700 1 -- 172.21.15.99:0/2807848434 >> 172.21.15.99:0/2807848434 conn(0x7f522407b6e0 msgr2=0x7f522407bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:57.324 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.507+0000 7f522a503700 1 -- 172.21.15.99:0/2807848434 shutdown_connections 2023-08-25T08:50:57.324 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.508+0000 7f522a503700 1 -- 172.21.15.99:0/2807848434 wait complete. 2023-08-25T08:50:57.324 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.509+0000 7f522a503700 1 Processor -- start 2023-08-25T08:50:57.325 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.509+0000 7f522a503700 1 -- start start 2023-08-25T08:50:57.325 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.510+0000 7f522a503700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f522419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:57.325 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.510+0000 7f522a503700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522419c570 con 0x7f5224108310 2023-08-25T08:50:57.325 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.510+0000 7f5223fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f522419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:57.325 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.510+0000 7f5223fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f522419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37226/0 (socket says 172.21.15.99:37226) 2023-08-25T08:50:57.325 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.510+0000 7f5223fff700 1 -- 172.21.15.99:0/720976724 learned_addr learned my addr 172.21.15.99:0/720976724 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:57.326 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.510+0000 7f5223fff700 1 -- 172.21.15.99:0/720976724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214009550 con 0x7f5224108310 2023-08-25T08:50:57.326 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.510+0000 7f5223fff700 1 --2- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f522419c030 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5214000c00 tx=0x7f5214011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:57.326 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.511+0000 7f52217fa700 1 -- 172.21.15.99:0/720976724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52140039f0 con 0x7f5224108310 2023-08-25T08:50:57.326 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.511+0000 7f52217fa700 1 -- 172.21.15.99:0/720976724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5214003b50 con 0x7f5224108310 2023-08-25T08:50:57.327 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.511+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522419c770 con 0x7f5224108310 2023-08-25T08:50:57.327 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.511+0000 7f52217fa700 1 -- 172.21.15.99:0/720976724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f521401a780 con 0x7f5224108310 2023-08-25T08:50:57.327 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.511+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522419cb90 con 0x7f5224108310 2023-08-25T08:50:57.327 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.512+0000 7f52217fa700 1 -- 172.21.15.99:0/720976724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f5214024440 con 0x7f5224108310 2023-08-25T08:50:57.327 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.512+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5224061f50 con 0x7f5224108310 2023-08-25T08:50:57.327 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.512+0000 7f52217fa700 1 --2- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f520c038160 0x7f520c03a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:57.327 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.512+0000 7f52217fa700 1 -- 172.21.15.99:0/720976724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f521404c220 con 0x7f5224108310 2023-08-25T08:50:57.328 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.512+0000 7f52237fe700 1 --2- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f520c038160 0x7f520c03a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:57.328 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.513+0000 7f52237fe700 1 --2- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f520c038160 0x7f520c03a620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5218006fd0 tx=0x7f5218006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:57.328 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.517+0000 7f52217fa700 1 -- 172.21.15.99:0/720976724 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f52140189a0 con 0x7f5224108310 2023-08-25T08:50:57.328 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.734+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f5224106ce0 con 0x7f520c038160 2023-08-25T08:50:57.328 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.884+0000 7f52217fa700 1 -- 172.21.15.99:0/720976724 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f5224106ce0 con 0x7f520c038160 2023-08-25T08:50:57.328 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f520c038160 msgr2=0x7f520c03a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:57.328 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 --2- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f520c038160 0x7f520c03a620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5218006fd0 tx=0x7f5218006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:57.329 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 msgr2=0x7f522419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:57.329 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 --2- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f522419c030 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5214000c00 tx=0x7f5214011a30 comp rx=0 tx=0).stop 2023-08-25T08:50:57.329 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 shutdown_connections 2023-08-25T08:50:57.329 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 --2- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f520c038160 0x7f520c03a620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:57.329 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 --2- 172.21.15.99:0/720976724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5224108310 0x7f522419c030 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:57.329 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 >> 172.21.15.99:0/720976724 conn(0x7f522407b6e0 msgr2=0x7f5224106080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:57.329 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 shutdown_connections 2023-08-25T08:50:57.330 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:56.887+0000 7f522a503700 1 -- 172.21.15.99:0/720976724 wait complete. 2023-08-25T08:50:57.330 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial admin user... 2023-08-25T08:50:57.609 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:57 smithi099 ceph-mon[97641]: mgrmap e12: x(active, since 2s) 2023-08-25T08:50:57.609 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:50:57.610 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:57 smithi099 ceph-mon[97641]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:57.610 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:50:57.610 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PAa0tOhT9hnhnvCtpr06.eb8hz/.VGIH8DR8.MsYO0xMkfKbUEa8m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1692953458, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-08-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.842+0000 7f80983e6700 1 Processor -- start 2023-08-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.842+0000 7f80983e6700 1 -- start start 2023-08-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.842+0000 7f80983e6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.842+0000 7f80983e6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809007a8c0 con 0x7f809007be90 2023-08-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.842+0000 7f8096182700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:58.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.843+0000 7f8096182700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51024/0 (socket says 172.21.15.99:51024) 2023-08-25T08:50:58.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.843+0000 7f8096182700 1 -- 172.21.15.99:0/1184723575 learned_addr learned my addr 172.21.15.99:0/1184723575 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:58.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.843+0000 7f8096182700 1 -- 172.21.15.99:0/1184723575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809007aa00 con 0x7f809007be90 2023-08-25T08:50:58.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.843+0000 7f8096182700 1 --2- 172.21.15.99:0/1184723575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809007a2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f808c0098a0 tx=0x7f808c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fde8abc37540956 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:58.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.843+0000 7f8095180700 1 -- 172.21.15.99:0/1184723575 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f808c004160 con 0x7f809007be90 2023-08-25T08:50:58.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.843+0000 7f8095180700 1 -- 172.21.15.99:0/1184723575 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f808c0042c0 con 0x7f809007be90 2023-08-25T08:50:58.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.843+0000 7f8095180700 1 -- 172.21.15.99:0/1184723575 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f808c0119b0 con 0x7f809007be90 2023-08-25T08:50:58.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.844+0000 7f80983e6700 1 -- 172.21.15.99:0/1184723575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 msgr2=0x7f809007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:58.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.844+0000 7f80983e6700 1 --2- 172.21.15.99:0/1184723575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809007a2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f808c0098a0 tx=0x7f808c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:50:58.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.844+0000 7f80983e6700 1 -- 172.21.15.99:0/1184723575 shutdown_connections 2023-08-25T08:50:58.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.844+0000 7f80983e6700 1 --2- 172.21.15.99:0/1184723575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809007a2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:58.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.844+0000 7f80983e6700 1 -- 172.21.15.99:0/1184723575 >> 172.21.15.99:0/1184723575 conn(0x7f80901016e0 msgr2=0x7f8090103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:58.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.844+0000 7f80983e6700 1 -- 172.21.15.99:0/1184723575 shutdown_connections 2023-08-25T08:50:58.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.844+0000 7f80983e6700 1 -- 172.21.15.99:0/1184723575 wait complete. 2023-08-25T08:50:58.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.845+0000 7f80983e6700 1 Processor -- start 2023-08-25T08:50:58.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.845+0000 7f80983e6700 1 -- start start 2023-08-25T08:50:58.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.845+0000 7f80983e6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:58.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.845+0000 7f80983e6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809019c610 con 0x7f809007be90 2023-08-25T08:50:58.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.846+0000 7f8096182700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:58.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.846+0000 7f8096182700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809019c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51032/0 (socket says 172.21.15.99:51032) 2023-08-25T08:50:58.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.846+0000 7f8096182700 1 -- 172.21.15.99:0/621437231 learned_addr learned my addr 172.21.15.99:0/621437231 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:50:58.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.846+0000 7f8096182700 1 -- 172.21.15.99:0/621437231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c009550 con 0x7f809007be90 2023-08-25T08:50:58.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.846+0000 7f8096182700 1 --2- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809019c0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f808c003e60 tx=0x7f808c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:58.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.846+0000 7f80877fe700 1 -- 172.21.15.99:0/621437231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f808c003a60 con 0x7f809007be90 2023-08-25T08:50:58.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.846+0000 7f80877fe700 1 -- 172.21.15.99:0/621437231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f808c003bc0 con 0x7f809007be90 2023-08-25T08:50:58.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.846+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809019c810 con 0x7f809007be90 2023-08-25T08:50:58.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.847+0000 7f80877fe700 1 -- 172.21.15.99:0/621437231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f808c0196c0 con 0x7f809007be90 2023-08-25T08:50:58.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.847+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809019cc30 con 0x7f809007be90 2023-08-25T08:50:58.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.848+0000 7f80877fe700 1 -- 172.21.15.99:0/621437231 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f808c01ab60 con 0x7f809007be90 2023-08-25T08:50:58.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.848+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8090196120 con 0x7f809007be90 2023-08-25T08:50:58.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.848+0000 7f80877fe700 1 --2- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f807c038190 0x7f807c03a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:50:58.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.848+0000 7f80877fe700 1 -- 172.21.15.99:0/621437231 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f808c04c330 con 0x7f809007be90 2023-08-25T08:50:58.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.848+0000 7f8095981700 1 --2- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f807c038190 0x7f807c03a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:50:58.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.849+0000 7f8095981700 1 --2- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f807c038190 0x7f807c03a650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8080006fd0 tx=0x7f8080006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:50:58.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:57.853+0000 7f80877fe700 1 -- 172.21.15.99:0/621437231 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f808c010790 con 0x7f809007be90 2023-08-25T08:50:58.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.077+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8090061d70 con 0x7f807c038190 2023-08-25T08:50:58.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.308+0000 7f80877fe700 1 -- 172.21.15.99:0/621437231 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8090061d70 con 0x7f807c038190 2023-08-25T08:50:58.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f807c038190 msgr2=0x7f807c03a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:58.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 --2- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f807c038190 0x7f807c03a650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8080006fd0 tx=0x7f8080006e40 comp rx=0 tx=0).stop 2023-08-25T08:50:58.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 msgr2=0x7f809019c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:50:58.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 --2- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809019c0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f808c003e60 tx=0x7f808c011a30 comp rx=0 tx=0).stop 2023-08-25T08:50:58.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 shutdown_connections 2023-08-25T08:50:58.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 --2- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f807c038190 0x7f807c03a650 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:58.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 --2- 172.21.15.99:0/621437231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f809007be90 0x7f809019c0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:50:58.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 >> 172.21.15.99:0/621437231 conn(0x7f80901016e0 msgr2=0x7f8090102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:50:58.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 shutdown_connections 2023-08-25T08:50:58.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:58.312+0000 7f80983e6700 1 -- 172.21.15.99:0/621437231 wait complete. 2023-08-25T08:50:58.817 INFO:teuthology.orchestra.run.smithi099.stdout:Fetching dashboard port number... 2023-08-25T08:50:59.050 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:50:59.050 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:58 smithi099 ceph-mon[97641]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:50:59.050 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:00.195 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 8443 2023-08-25T08:51:00.195 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.347+0000 7fe16c168700 1 Processor -- start 2023-08-25T08:51:00.196 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.348+0000 7fe16c168700 1 -- start start 2023-08-25T08:51:00.196 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.348+0000 7fe16c168700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:00.196 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.348+0000 7fe16c168700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16407a8a0 con 0x7fe16407be70 2023-08-25T08:51:00.196 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.348+0000 7fe169f04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:00.196 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.348+0000 7fe169f04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51038/0 (socket says 172.21.15.99:51038) 2023-08-25T08:51:00.196 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.348+0000 7fe169f04700 1 -- 172.21.15.99:0/1082128652 learned_addr learned my addr 172.21.15.99:0/1082128652 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:00.196 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.349+0000 7fe169f04700 1 -- 172.21.15.99:0/1082128652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16407a9e0 con 0x7fe16407be70 2023-08-25T08:51:00.197 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.349+0000 7fe169f04700 1 --2- 172.21.15.99:0/1082128652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16407a2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1600098a0 tx=0x7fe160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a730a5f538ff1081 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:00.197 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.349+0000 7fe168f02700 1 -- 172.21.15.99:0/1082128652 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe160004160 con 0x7fe16407be70 2023-08-25T08:51:00.197 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.349+0000 7fe168f02700 1 -- 172.21.15.99:0/1082128652 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1600042c0 con 0x7fe16407be70 2023-08-25T08:51:00.197 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.349+0000 7fe168f02700 1 -- 172.21.15.99:0/1082128652 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1600119b0 con 0x7fe16407be70 2023-08-25T08:51:00.197 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.350+0000 7fe16c168700 1 -- 172.21.15.99:0/1082128652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 msgr2=0x7fe16407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:00.197 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.350+0000 7fe16c168700 1 --2- 172.21.15.99:0/1082128652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16407a2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1600098a0 tx=0x7fe160009bb0 comp rx=0 tx=0).stop 2023-08-25T08:51:00.197 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.350+0000 7fe16c168700 1 -- 172.21.15.99:0/1082128652 shutdown_connections 2023-08-25T08:51:00.198 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.350+0000 7fe16c168700 1 --2- 172.21.15.99:0/1082128652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16407a2d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:00.198 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.350+0000 7fe16c168700 1 -- 172.21.15.99:0/1082128652 >> 172.21.15.99:0/1082128652 conn(0x7fe1641016e0 msgr2=0x7fe164103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:00.198 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.350+0000 7fe16c168700 1 -- 172.21.15.99:0/1082128652 shutdown_connections 2023-08-25T08:51:00.198 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.350+0000 7fe16c168700 1 -- 172.21.15.99:0/1082128652 wait complete. 2023-08-25T08:51:00.198 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.351+0000 7fe16c168700 1 Processor -- start 2023-08-25T08:51:00.198 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.351+0000 7fe16c168700 1 -- start start 2023-08-25T08:51:00.198 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.351+0000 7fe16c168700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16419c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:00.198 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.351+0000 7fe16c168700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16419c5a0 con 0x7fe16407be70 2023-08-25T08:51:00.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.352+0000 7fe169f04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16419c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:00.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.352+0000 7fe169f04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16419c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51044/0 (socket says 172.21.15.99:51044) 2023-08-25T08:51:00.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.352+0000 7fe169f04700 1 -- 172.21.15.99:0/2089309129 learned_addr learned my addr 172.21.15.99:0/2089309129 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:00.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.352+0000 7fe169f04700 1 -- 172.21.15.99:0/2089309129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160009550 con 0x7fe16407be70 2023-08-25T08:51:00.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.352+0000 7fe169f04700 1 --2- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16419c060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe160003f00 tx=0x7fe160011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:00.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.353+0000 7fe15affd700 1 -- 172.21.15.99:0/2089309129 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe160003b20 con 0x7fe16407be70 2023-08-25T08:51:00.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.353+0000 7fe15affd700 1 -- 172.21.15.99:0/2089309129 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe160003c80 con 0x7fe16407be70 2023-08-25T08:51:00.199 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.353+0000 7fe15affd700 1 -- 172.21.15.99:0/2089309129 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe160019770 con 0x7fe16407be70 2023-08-25T08:51:00.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.353+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16419c7a0 con 0x7fe16407be70 2023-08-25T08:51:00.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.353+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16419cbc0 con 0x7fe16407be70 2023-08-25T08:51:00.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.354+0000 7fe15affd700 1 -- 172.21.15.99:0/2089309129 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7fe16001ab60 con 0x7fe16407be70 2023-08-25T08:51:00.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.354+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16410c560 con 0x7fe16407be70 2023-08-25T08:51:00.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.354+0000 7fe15affd700 1 --2- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe1500381b0 0x7fe15003a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:00.200 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.354+0000 7fe15affd700 1 -- 172.21.15.99:0/2089309129 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe16004c3d0 con 0x7fe16407be70 2023-08-25T08:51:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.355+0000 7fe169703700 1 --2- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe1500381b0 0x7fe15003a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.355+0000 7fe169703700 1 --2- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe1500381b0 0x7fe15003a670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe154006fd0 tx=0x7fe154006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.360+0000 7fe15affd700 1 -- 172.21.15.99:0/2089309129 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fe16001a430 con 0x7fe16407be70 2023-08-25T08:51:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.603+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fe16410c7a0 con 0x7fe16407be70 2023-08-25T08:51:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.604+0000 7fe15affd700 1 -- 172.21.15.99:0/2089309129 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fe1600172b0 con 0x7fe16407be70 2023-08-25T08:51:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.606+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe1500381b0 msgr2=0x7fe15003a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.606+0000 7fe16c168700 1 --2- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe1500381b0 0x7fe15003a670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe154006fd0 tx=0x7fe154006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.606+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 msgr2=0x7fe16419c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:00.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.606+0000 7fe16c168700 1 --2- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16419c060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe160003f00 tx=0x7fe160011a30 comp rx=0 tx=0).stop 2023-08-25T08:51:00.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.606+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 shutdown_connections 2023-08-25T08:51:00.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.606+0000 7fe16c168700 1 --2- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe1500381b0 0x7fe15003a670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:00.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.606+0000 7fe16c168700 1 --2- 172.21.15.99:0/2089309129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe16407be70 0x7fe16419c060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:00.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.606+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 >> 172.21.15.99:0/2089309129 conn(0x7fe1641016e0 msgr2=0x7fe164102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:00.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.607+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 shutdown_connections 2023-08-25T08:51:00.202 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:50:59.607+0000 7fe16c168700 1 -- 172.21.15.99:0/2089309129 wait complete. 2023-08-25T08:51:00.213 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-25T08:51:00.213 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2023-08-25T08:51:00.229 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-25T08:51:00.229 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2023-08-25T08:51:00.229 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2023-08-25T08:51:00.230 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-08-25T08:51:00.231 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph Dashboard is now available at: 2023-08-25T08:51:00.232 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:00.232 INFO:teuthology.orchestra.run.smithi099.stdout: URL: https://smithi099.front.sepia.ceph.com:8443/ 2023-08-25T08:51:00.232 INFO:teuthology.orchestra.run.smithi099.stdout: User: admin 2023-08-25T08:51:00.232 INFO:teuthology.orchestra.run.smithi099.stdout: Password: bk199mg13w 2023-08-25T08:51:00.232 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:00.233 INFO:teuthology.orchestra.run.smithi099.stdout:Saving cluster configuration to /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config directory 2023-08-25T08:51:00.233 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling autotune for osd_memory_target 2023-08-25T08:51:00.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:50:59 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2089309129' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-08-25T08:51:01.414 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.759+0000 7fe259388700 1 Processor -- start 2023-08-25T08:51:01.414 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.759+0000 7fe259388700 1 -- start start 2023-08-25T08:51:01.415 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.759+0000 7fe259388700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:01.415 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.759+0000 7fe259388700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254105a10 con 0x7fe254105020 2023-08-25T08:51:01.415 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.760+0000 7fe252ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:01.415 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.760+0000 7fe252ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51050/0 (socket says 172.21.15.99:51050) 2023-08-25T08:51:01.415 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.760+0000 7fe252ffd700 1 -- 172.21.15.99:0/3241571194 learned_addr learned my addr 172.21.15.99:0/3241571194 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:01.415 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.760+0000 7fe252ffd700 1 -- 172.21.15.99:0/3241571194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254106220 con 0x7fe254105020 2023-08-25T08:51:01.415 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.760+0000 7fe252ffd700 1 --2- 172.21.15.99:0/3241571194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254105440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2440098a0 tx=0x7fe244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76f563d1f5381b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:01.416 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe251ffb700 1 -- 172.21.15.99:0/3241571194 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe244004160 con 0x7fe254105020 2023-08-25T08:51:01.416 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe251ffb700 1 -- 172.21.15.99:0/3241571194 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2440042c0 con 0x7fe254105020 2023-08-25T08:51:01.416 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe251ffb700 1 -- 172.21.15.99:0/3241571194 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2440119b0 con 0x7fe254105020 2023-08-25T08:51:01.416 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe259388700 1 -- 172.21.15.99:0/3241571194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 msgr2=0x7fe254105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:01.416 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe259388700 1 --2- 172.21.15.99:0/3241571194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254105440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2440098a0 tx=0x7fe244009bb0 comp rx=0 tx=0).stop 2023-08-25T08:51:01.416 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe259388700 1 -- 172.21.15.99:0/3241571194 shutdown_connections 2023-08-25T08:51:01.416 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe259388700 1 --2- 172.21.15.99:0/3241571194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254105440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe259388700 1 -- 172.21.15.99:0/3241571194 >> 172.21.15.99:0/3241571194 conn(0x7fe2541005c0 msgr2=0x7fe254102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe259388700 1 -- 172.21.15.99:0/3241571194 shutdown_connections 2023-08-25T08:51:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.761+0000 7fe259388700 1 -- 172.21.15.99:0/3241571194 wait complete. 2023-08-25T08:51:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.763+0000 7fe259388700 1 Processor -- start 2023-08-25T08:51:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.763+0000 7fe259388700 1 -- start start 2023-08-25T08:51:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.764+0000 7fe259388700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.764+0000 7fe259388700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25419a370 con 0x7fe254105020 2023-08-25T08:51:01.417 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.764+0000 7fe252ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.764+0000 7fe252ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51060/0 (socket says 172.21.15.99:51060) 2023-08-25T08:51:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.764+0000 7fe252ffd700 1 -- 172.21.15.99:0/2563435338 learned_addr learned my addr 172.21.15.99:0/2563435338 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.764+0000 7fe252ffd700 1 -- 172.21.15.99:0/2563435338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe244009550 con 0x7fe254105020 2023-08-25T08:51:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.764+0000 7fe252ffd700 1 --2- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254199e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe244003e60 tx=0x7fe244011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.765+0000 7fe23bfff700 1 -- 172.21.15.99:0/2563435338 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe244003970 con 0x7fe254105020 2023-08-25T08:51:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.765+0000 7fe23bfff700 1 -- 172.21.15.99:0/2563435338 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe244003ad0 con 0x7fe254105020 2023-08-25T08:51:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.765+0000 7fe23bfff700 1 -- 172.21.15.99:0/2563435338 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2440195e0 con 0x7fe254105020 2023-08-25T08:51:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.765+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25419a570 con 0x7fe254105020 2023-08-25T08:51:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.765+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25419a990 con 0x7fe254105020 2023-08-25T08:51:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.766+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe254109450 con 0x7fe254105020 2023-08-25T08:51:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.766+0000 7fe23bfff700 1 -- 172.21.15.99:0/2563435338 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7fe2440198a0 con 0x7fe254105020 2023-08-25T08:51:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.766+0000 7fe23bfff700 1 --2- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe23c038110 0x7fe23c03a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.766+0000 7fe23bfff700 1 -- 172.21.15.99:0/2563435338 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe24404c2b0 con 0x7fe254105020 2023-08-25T08:51:01.419 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.766+0000 7fe2527fc700 1 --2- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe23c038110 0x7fe23c03a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:01.420 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.767+0000 7fe2527fc700 1 --2- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe23c038110 0x7fe23c03a5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe248006fd0 tx=0x7fe248006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:01.420 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.771+0000 7fe23bfff700 1 -- 172.21.15.99:0/2563435338 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fe244003c40 con 0x7fe254105020 2023-08-25T08:51:01.420 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.964+0000 7fe23bfff700 1 -- 172.21.15.99:0/2563435338 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fe24401a430 con 0x7fe254105020 2023-08-25T08:51:01.420 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.971+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fe25419d3d0 con 0x7fe254105020 2023-08-25T08:51:01.420 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.971+0000 7fe23bfff700 1 -- 172.21.15.99:0/2563435338 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fe25419d3d0 con 0x7fe254105020 2023-08-25T08:51:01.420 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe23c038110 msgr2=0x7fe23c03a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:01.420 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 --2- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe23c038110 0x7fe23c03a5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe248006fd0 tx=0x7fe248006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 msgr2=0x7fe254199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 --2- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254199e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe244003e60 tx=0x7fe244011aa0 comp rx=0 tx=0).stop 2023-08-25T08:51:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 shutdown_connections 2023-08-25T08:51:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 --2- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe23c038110 0x7fe23c03a5d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 --2- 172.21.15.99:0/2563435338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe254105020 0x7fe254199e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 >> 172.21.15.99:0/2563435338 conn(0x7fe2541005c0 msgr2=0x7fe254079910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 shutdown_connections 2023-08-25T08:51:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:00.974+0000 7fe259388700 1 -- 172.21.15.99:0/2563435338 wait complete. 2023-08-25T08:51:02.300 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:01 smithi099 ceph-mon[97641]: mgrmap e13: x(active, since 6s) 2023-08-25T08:51:02.788 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.958+0000 7f9721aac700 1 Processor -- start 2023-08-25T08:51:02.788 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.958+0000 7f9721aac700 1 -- start start 2023-08-25T08:51:02.788 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.958+0000 7f9721aac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c07a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:02.788 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.958+0000 7f9721aac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c07a900 con 0x7f971c07bed0 2023-08-25T08:51:02.789 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.959+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c07a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:02.789 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.959+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c07a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51074/0 (socket says 172.21.15.99:51074) 2023-08-25T08:51:02.789 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.959+0000 7f971b7fe700 1 -- 172.21.15.99:0/2424885819 learned_addr learned my addr 172.21.15.99:0/2424885819 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:02.789 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.959+0000 7f971b7fe700 1 -- 172.21.15.99:0/2424885819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c07aa40 con 0x7f971c07bed0 2023-08-25T08:51:02.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.959+0000 7f971b7fe700 1 --2- 172.21.15.99:0/2424885819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c07a330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f970c0098a0 tx=0x7f970c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316cc788f390927 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:02.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f971a7fc700 1 -- 172.21.15.99:0/2424885819 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f970c004160 con 0x7f971c07bed0 2023-08-25T08:51:02.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f971a7fc700 1 -- 172.21.15.99:0/2424885819 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f970c0042c0 con 0x7f971c07bed0 2023-08-25T08:51:02.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f971a7fc700 1 -- 172.21.15.99:0/2424885819 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f970c0119b0 con 0x7f971c07bed0 2023-08-25T08:51:02.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f9721aac700 1 -- 172.21.15.99:0/2424885819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 msgr2=0x7f971c07a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:02.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f9721aac700 1 --2- 172.21.15.99:0/2424885819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c07a330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f970c0098a0 tx=0x7f970c009bb0 comp rx=0 tx=0).stop 2023-08-25T08:51:02.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f9721aac700 1 -- 172.21.15.99:0/2424885819 shutdown_connections 2023-08-25T08:51:02.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f9721aac700 1 --2- 172.21.15.99:0/2424885819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c07a330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:02.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f9721aac700 1 -- 172.21.15.99:0/2424885819 >> 172.21.15.99:0/2424885819 conn(0x7f971c1016e0 msgr2=0x7f971c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:02.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.960+0000 7f9721aac700 1 -- 172.21.15.99:0/2424885819 shutdown_connections 2023-08-25T08:51:02.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.961+0000 7f9721aac700 1 -- 172.21.15.99:0/2424885819 wait complete. 2023-08-25T08:51:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.961+0000 7f9721aac700 1 Processor -- start 2023-08-25T08:51:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.962+0000 7f9721aac700 1 -- start start 2023-08-25T08:51:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.962+0000 7f9721aac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.962+0000 7f9721aac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c19c610 con 0x7f971c07bed0 2023-08-25T08:51:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.962+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.962+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c19c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51086/0 (socket says 172.21.15.99:51086) 2023-08-25T08:51:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.962+0000 7f971b7fe700 1 -- 172.21.15.99:0/2138053637 learned_addr learned my addr 172.21.15.99:0/2138053637 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.963+0000 7f971b7fe700 1 -- 172.21.15.99:0/2138053637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c009550 con 0x7f971c07bed0 2023-08-25T08:51:02.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.963+0000 7f971b7fe700 1 --2- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c19c0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f970c003e60 tx=0x7f970c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:02.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.965+0000 7f9718ff9700 1 -- 172.21.15.99:0/2138053637 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f970c003a60 con 0x7f971c07bed0 2023-08-25T08:51:02.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.965+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c19c810 con 0x7f971c07bed0 2023-08-25T08:51:02.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.965+0000 7f9718ff9700 1 -- 172.21.15.99:0/2138053637 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f970c003bc0 con 0x7f971c07bed0 2023-08-25T08:51:02.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.965+0000 7f9718ff9700 1 -- 172.21.15.99:0/2138053637 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f970c0196c0 con 0x7f971c07bed0 2023-08-25T08:51:02.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.965+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c19cc30 con 0x7f971c07bed0 2023-08-25T08:51:02.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.967+0000 7f9718ff9700 1 -- 172.21.15.99:0/2138053637 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f970c019980 con 0x7f971c07bed0 2023-08-25T08:51:02.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.967+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c196120 con 0x7f971c07bed0 2023-08-25T08:51:02.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.967+0000 7f9718ff9700 1 --2- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f97040381e0 0x7f970403a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:02.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.967+0000 7f9718ff9700 1 -- 172.21.15.99:0/2138053637 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f970c04c590 con 0x7f971c07bed0 2023-08-25T08:51:02.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.967+0000 7f971affd700 1 --2- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f97040381e0 0x7f970403a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:02.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.968+0000 7f971affd700 1 --2- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f97040381e0 0x7f970403a6a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9710006fd0 tx=0x7f9710006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:02.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:01.972+0000 7f9718ff9700 1 -- 172.21.15.99:0/2138053637 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f970c023370 con 0x7f971c07bed0 2023-08-25T08:51:02.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.273+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f971c061f50 con 0x7f971c07bed0 2023-08-25T08:51:02.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.277+0000 7f9718ff9700 1 -- 172.21.15.99:0/2138053637 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f970c018350 con 0x7f971c07bed0 2023-08-25T08:51:02.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.279+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f97040381e0 msgr2=0x7f970403a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:02.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.279+0000 7f9721aac700 1 --2- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f97040381e0 0x7f970403a6a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9710006fd0 tx=0x7f9710006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:02.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.279+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 msgr2=0x7f971c19c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:02.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.279+0000 7f9721aac700 1 --2- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c19c0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f970c003e60 tx=0x7f970c011a30 comp rx=0 tx=0).stop 2023-08-25T08:51:02.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.279+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 shutdown_connections 2023-08-25T08:51:02.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.279+0000 7f9721aac700 1 --2- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f97040381e0 0x7f970403a6a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:02.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.279+0000 7f9721aac700 1 --2- 172.21.15.99:0/2138053637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f971c07bed0 0x7f971c19c0d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:02.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.279+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 >> 172.21.15.99:0/2138053637 conn(0x7f971c1016e0 msgr2=0x7f971c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:02.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.280+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 shutdown_connections 2023-08-25T08:51:02.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2023-08-25T08:51:02.280+0000 7f9721aac700 1 -- 172.21.15.99:0/2138053637 wait complete. 2023-08-25T08:51:02.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-08-25T08:51:02.799 INFO:teuthology.orchestra.run.smithi099.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-08-25T08:51:02.799 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:02.799 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:02.799 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:02.800 INFO:teuthology.orchestra.run.smithi099.stdout:Or, if you are only running a single cluster on this host: 2023-08-25T08:51:02.800 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:02.800 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-08-25T08:51:02.800 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:02.800 INFO:teuthology.orchestra.run.smithi099.stdout:Please consider enabling telemetry to help improve Ceph: 2023-08-25T08:51:02.800 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:02.800 INFO:teuthology.orchestra.run.smithi099.stdout: ceph telemetry on 2023-08-25T08:51:02.801 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:02.801 INFO:teuthology.orchestra.run.smithi099.stdout:For more information see: 2023-08-25T08:51:02.801 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:02.801 INFO:teuthology.orchestra.run.smithi099.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-08-25T08:51:02.801 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:02.801 INFO:teuthology.orchestra.run.smithi099.stdout:Bootstrap complete. 2023-08-25T08:51:02.853 INFO:tasks.cephadm:Fetching config... 2023-08-25T08:51:02.853 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:51:02.854 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-08-25T08:51:02.900 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-08-25T08:51:02.900 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:51:02.900 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-08-25T08:51:02.988 INFO:tasks.cephadm:Fetching mon keyring... 2023-08-25T08:51:02.988 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:51:02.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-08-25T08:51:03.096 INFO:tasks.cephadm:Fetching pub ssh key... 2023-08-25T08:51:03.096 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:51:03.096 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-08-25T08:51:03.179 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-08-25T08:51:03.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9EUT+ijUfNGmrLF9rb81WnRaSi7HTCONvfh+nzI9qFlmVa4ESSP6QUC9dIDZjKTqfrlKTOGyyIV+HMwi7DH/Z4B+gvDirB8SuodrfM2CUhTMbUKmBMmPnjktZNKGItNpc+Xg9qDbVWk4XjfQ/p1buvVVjJYWi6c5N+qHHt6nk8acFZAxTn41p71swBUMe4BpKaSM2aS3acVyn3dF/XNl0aPcULtd6YJlZbvaFsR/rOIg+GlM8f6KLdfT55xSb6wmE/yLS2VQQ6Wy6SUJUqXEzV3jBmBSrTz+MmEzXgyBy48xrLQaseyyU1waT65ZUSJ25uhssBjYzyQkVwOF98jUd1FzFrz+CtdrAiu5k21yIJ76dJgF8dYBiqTxU/R0WJleVHpim8xC7WKN7+YNhG4ckh1ZYVsEc5lR0ywjM4v3fWQbp2TXDDFYypDuWg6A51Wy2XSj2w/jYhozN+Aunf3SXbIcJ4vDRrMvsmHXcoBHMXTWvNOPE5uR4e3cvZ6sJNYs= ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-25T08:51:03.317 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9EUT+ijUfNGmrLF9rb81WnRaSi7HTCONvfh+nzI9qFlmVa4ESSP6QUC9dIDZjKTqfrlKTOGyyIV+HMwi7DH/Z4B+gvDirB8SuodrfM2CUhTMbUKmBMmPnjktZNKGItNpc+Xg9qDbVWk4XjfQ/p1buvVVjJYWi6c5N+qHHt6nk8acFZAxTn41p71swBUMe4BpKaSM2aS3acVyn3dF/XNl0aPcULtd6YJlZbvaFsR/rOIg+GlM8f6KLdfT55xSb6wmE/yLS2VQQ6Wy6SUJUqXEzV3jBmBSrTz+MmEzXgyBy48xrLQaseyyU1waT65ZUSJ25uhssBjYzyQkVwOF98jUd1FzFrz+CtdrAiu5k21yIJ76dJgF8dYBiqTxU/R0WJleVHpim8xC7WKN7+YNhG4ckh1ZYVsEc5lR0ywjM4v3fWQbp2TXDDFYypDuWg6A51Wy2XSj2w/jYhozN+Aunf3SXbIcJ4vDRrMvsmHXcoBHMXTWvNOPE5uR4e3cvZ6sJNYs= ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:03.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:03 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2138053637' entity='client.admin' 2023-08-25T08:51:03.348 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9EUT+ijUfNGmrLF9rb81WnRaSi7HTCONvfh+nzI9qFlmVa4ESSP6QUC9dIDZjKTqfrlKTOGyyIV+HMwi7DH/Z4B+gvDirB8SuodrfM2CUhTMbUKmBMmPnjktZNKGItNpc+Xg9qDbVWk4XjfQ/p1buvVVjJYWi6c5N+qHHt6nk8acFZAxTn41p71swBUMe4BpKaSM2aS3acVyn3dF/XNl0aPcULtd6YJlZbvaFsR/rOIg+GlM8f6KLdfT55xSb6wmE/yLS2VQQ6Wy6SUJUqXEzV3jBmBSrTz+MmEzXgyBy48xrLQaseyyU1waT65ZUSJ25uhssBjYzyQkVwOF98jUd1FzFrz+CtdrAiu5k21yIJ76dJgF8dYBiqTxU/R0WJleVHpim8xC7WKN7+YNhG4ckh1ZYVsEc5lR0ywjM4v3fWQbp2TXDDFYypDuWg6A51Wy2XSj2w/jYhozN+Aunf3SXbIcJ4vDRrMvsmHXcoBHMXTWvNOPE5uR4e3cvZ6sJNYs= ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-25T08:51:03.395 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9EUT+ijUfNGmrLF9rb81WnRaSi7HTCONvfh+nzI9qFlmVa4ESSP6QUC9dIDZjKTqfrlKTOGyyIV+HMwi7DH/Z4B+gvDirB8SuodrfM2CUhTMbUKmBMmPnjktZNKGItNpc+Xg9qDbVWk4XjfQ/p1buvVVjJYWi6c5N+qHHt6nk8acFZAxTn41p71swBUMe4BpKaSM2aS3acVyn3dF/XNl0aPcULtd6YJlZbvaFsR/rOIg+GlM8f6KLdfT55xSb6wmE/yLS2VQQ6Wy6SUJUqXEzV3jBmBSrTz+MmEzXgyBy48xrLQaseyyU1waT65ZUSJ25uhssBjYzyQkVwOF98jUd1FzFrz+CtdrAiu5k21yIJ76dJgF8dYBiqTxU/R0WJleVHpim8xC7WKN7+YNhG4ckh1ZYVsEc5lR0ywjM4v3fWQbp2TXDDFYypDuWg6A51Wy2XSj2w/jYhozN+Aunf3SXbIcJ4vDRrMvsmHXcoBHMXTWvNOPE5uR4e3cvZ6sJNYs= ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:03.408 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9EUT+ijUfNGmrLF9rb81WnRaSi7HTCONvfh+nzI9qFlmVa4ESSP6QUC9dIDZjKTqfrlKTOGyyIV+HMwi7DH/Z4B+gvDirB8SuodrfM2CUhTMbUKmBMmPnjktZNKGItNpc+Xg9qDbVWk4XjfQ/p1buvVVjJYWi6c5N+qHHt6nk8acFZAxTn41p71swBUMe4BpKaSM2aS3acVyn3dF/XNl0aPcULtd6YJlZbvaFsR/rOIg+GlM8f6KLdfT55xSb6wmE/yLS2VQQ6Wy6SUJUqXEzV3jBmBSrTz+MmEzXgyBy48xrLQaseyyU1waT65ZUSJ25uhssBjYzyQkVwOF98jUd1FzFrz+CtdrAiu5k21yIJ76dJgF8dYBiqTxU/R0WJleVHpim8xC7WKN7+YNhG4ckh1ZYVsEc5lR0ywjM4v3fWQbp2TXDDFYypDuWg6A51Wy2XSj2w/jYhozN+Aunf3SXbIcJ4vDRrMvsmHXcoBHMXTWvNOPE5uR4e3cvZ6sJNYs= ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-25T08:51:03.454 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9EUT+ijUfNGmrLF9rb81WnRaSi7HTCONvfh+nzI9qFlmVa4ESSP6QUC9dIDZjKTqfrlKTOGyyIV+HMwi7DH/Z4B+gvDirB8SuodrfM2CUhTMbUKmBMmPnjktZNKGItNpc+Xg9qDbVWk4XjfQ/p1buvVVjJYWi6c5N+qHHt6nk8acFZAxTn41p71swBUMe4BpKaSM2aS3acVyn3dF/XNl0aPcULtd6YJlZbvaFsR/rOIg+GlM8f6KLdfT55xSb6wmE/yLS2VQQ6Wy6SUJUqXEzV3jBmBSrTz+MmEzXgyBy48xrLQaseyyU1waT65ZUSJ25uhssBjYzyQkVwOF98jUd1FzFrz+CtdrAiu5k21yIJ76dJgF8dYBiqTxU/R0WJleVHpim8xC7WKN7+YNhG4ckh1ZYVsEc5lR0ywjM4v3fWQbp2TXDDFYypDuWg6A51Wy2XSj2w/jYhozN+Aunf3SXbIcJ4vDRrMvsmHXcoBHMXTWvNOPE5uR4e3cvZ6sJNYs= ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:03.466 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-08-25T08:51:04.416 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:05.786 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:05.786 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:05.787 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:51:05.787 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:05.787 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:05.787 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:06.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.297+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3134824294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 msgr2=0x7f6fbc100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:06.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.297+0000 7f6fc11ed700 1 --2- 172.21.15.99:0/3134824294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 0x7f6fbc100850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009910 tx=0x7f6fac009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:06.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.297+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3134824294 shutdown_connections 2023-08-25T08:51:06.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.297+0000 7f6fc11ed700 1 --2- 172.21.15.99:0/3134824294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 0x7f6fbc100850 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.297+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3134824294 >> 172.21.15.99:0/3134824294 conn(0x7f6fbc0fb9b0 msgr2=0x7f6fbc0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.297+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3134824294 shutdown_connections 2023-08-25T08:51:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.297+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3134824294 wait complete. 2023-08-25T08:51:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.298+0000 7f6fc11ed700 1 Processor -- start 2023-08-25T08:51:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.298+0000 7f6fc11ed700 1 -- start start 2023-08-25T08:51:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.299+0000 7f6fc11ed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 0x7f6fbc0746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.299+0000 7f6fc11ed700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc074c00 con 0x7f6fbc100430 2023-08-25T08:51:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.299+0000 7f6fbad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 0x7f6fbc0746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.299+0000 7f6fbad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 0x7f6fbc0746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51098/0 (socket says 172.21.15.99:51098) 2023-08-25T08:51:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.299+0000 7f6fbad9d700 1 -- 172.21.15.99:0/3629956778 learned_addr learned my addr 172.21.15.99:0/3629956778 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.299+0000 7f6fbad9d700 1 -- 172.21.15.99:0/3629956778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0095f0 con 0x7f6fbc100430 2023-08-25T08:51:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.300+0000 7f6fbad9d700 1 --2- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 0x7f6fbc0746c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6fac006010 tx=0x7f6fac0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.300+0000 7f6fa3fff700 1 -- 172.21.15.99:0/3629956778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fac01d070 con 0x7f6fbc100430 2023-08-25T08:51:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.300+0000 7f6fa3fff700 1 -- 172.21.15.99:0/3629956778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fac0044e0 con 0x7f6fbc100430 2023-08-25T08:51:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.300+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc072d10 con 0x7f6fbc100430 2023-08-25T08:51:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.300+0000 7f6fa3fff700 1 -- 172.21.15.99:0/3629956778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fac0039b0 con 0x7f6fbc100430 2023-08-25T08:51:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.300+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc073150 con 0x7f6fbc100430 2023-08-25T08:51:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.301+0000 7f6fa3fff700 1 -- 172.21.15.99:0/3629956778 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f6fac003c70 con 0x7f6fbc100430 2023-08-25T08:51:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.301+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc061f50 con 0x7f6fbc100430 2023-08-25T08:51:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.302+0000 7f6fa3fff700 1 --2- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fa40381b0 0x7f6fa403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.302+0000 7f6fa3fff700 1 -- 172.21.15.99:0/3629956778 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6fac04c360 con 0x7f6fbc100430 2023-08-25T08:51:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.302+0000 7f6fba59c700 1 --2- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fa40381b0 0x7f6fa403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:06.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.303+0000 7f6fba59c700 1 --2- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fa40381b0 0x7f6fa403a670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0006fd0 tx=0x7f6fb0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.307+0000 7f6fa3fff700 1 -- 172.21.15.99:0/3629956778 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f6fac02a950 con 0x7f6fbc100430 2023-08-25T08:51:06.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.512+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f6fbc106180 con 0x7f6fbc100430 2023-08-25T08:51:06.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.516+0000 7f6fa3fff700 1 -- 172.21.15.99:0/3629956778 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6fac02a360 con 0x7f6fbc100430 2023-08-25T08:51:06.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.518+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fa40381b0 msgr2=0x7f6fa403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:06.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.518+0000 7f6fc11ed700 1 --2- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fa40381b0 0x7f6fa403a670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0006fd0 tx=0x7f6fb0006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:06.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.518+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 msgr2=0x7f6fbc0746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:06.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.518+0000 7f6fc11ed700 1 --2- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 0x7f6fbc0746c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6fac006010 tx=0x7f6fac0057c0 comp rx=0 tx=0).stop 2023-08-25T08:51:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.519+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 shutdown_connections 2023-08-25T08:51:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.519+0000 7f6fc11ed700 1 --2- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fa40381b0 0x7f6fa403a670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.519+0000 7f6fc11ed700 1 --2- 172.21.15.99:0/3629956778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fbc100430 0x7f6fbc0746c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.519+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 >> 172.21.15.99:0/3629956778 conn(0x7f6fbc0fb9b0 msgr2=0x7f6fbc0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.519+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 shutdown_connections 2023-08-25T08:51:06.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:06.519+0000 7f6fc11ed700 1 -- 172.21.15.99:0/3629956778 wait complete. 2023-08-25T08:51:07.001 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-08-25T08:51:07.002 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-08-25T08:51:07.717 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:07.811 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:07 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3629956778' entity='client.admin' 2023-08-25T08:51:07.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:07.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.085+0000 7fc4097a9700 1 -- 172.21.15.99:0/3838244279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 msgr2=0x7fc404100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.085+0000 7fc4097a9700 1 --2- 172.21.15.99:0/3838244279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 0x7fc404100850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009910 tx=0x7fc3f4009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.085+0000 7fc4097a9700 1 -- 172.21.15.99:0/3838244279 shutdown_connections 2023-08-25T08:51:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.085+0000 7fc4097a9700 1 --2- 172.21.15.99:0/3838244279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 0x7fc404100850 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.085+0000 7fc4097a9700 1 -- 172.21.15.99:0/3838244279 >> 172.21.15.99:0/3838244279 conn(0x7fc4040fb9b0 msgr2=0x7fc4040fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.085+0000 7fc4097a9700 1 -- 172.21.15.99:0/3838244279 shutdown_connections 2023-08-25T08:51:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.085+0000 7fc4097a9700 1 -- 172.21.15.99:0/3838244279 wait complete. 2023-08-25T08:51:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.086+0000 7fc4097a9700 1 Processor -- start 2023-08-25T08:51:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.086+0000 7fc4097a9700 1 -- start start 2023-08-25T08:51:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.087+0000 7fc4097a9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 0x7fc404195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.087+0000 7fc4097a9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc404195a80 con 0x7fc404100430 2023-08-25T08:51:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.087+0000 7fc402ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 0x7fc404195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.087+0000 7fc402ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 0x7fc404195540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53100/0 (socket says 172.21.15.99:53100) 2023-08-25T08:51:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.087+0000 7fc402ffd700 1 -- 172.21.15.99:0/3087204390 learned_addr learned my addr 172.21.15.99:0/3087204390 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.087+0000 7fc402ffd700 1 -- 172.21.15.99:0/3087204390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f40095f0 con 0x7fc404100430 2023-08-25T08:51:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.087+0000 7fc402ffd700 1 --2- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 0x7fc404195540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4004f40 tx=0x7fc3f40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.087+0000 7fc3ebfff700 1 -- 172.21.15.99:0/3087204390 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3f401d070 con 0x7fc404100430 2023-08-25T08:51:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.088+0000 7fc3ebfff700 1 -- 172.21.15.99:0/3087204390 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3f400bc10 con 0x7fc404100430 2023-08-25T08:51:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.088+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc404195c80 con 0x7fc404100430 2023-08-25T08:51:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.088+0000 7fc3ebfff700 1 -- 172.21.15.99:0/3087204390 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3f4003a70 con 0x7fc404100430 2023-08-25T08:51:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.088+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc404196120 con 0x7fc404100430 2023-08-25T08:51:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.089+0000 7fc3ebfff700 1 -- 172.21.15.99:0/3087204390 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fc3f4003ce0 con 0x7fc404100430 2023-08-25T08:51:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.089+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40418ef40 con 0x7fc404100430 2023-08-25T08:51:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.089+0000 7fc3ebfff700 1 --2- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3ec038200 0x7fc3ec03a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.089+0000 7fc3ebfff700 1 -- 172.21.15.99:0/3087204390 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc3f404bdf0 con 0x7fc404100430 2023-08-25T08:51:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.089+0000 7fc4027fc700 1 --2- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3ec038200 0x7fc3ec03a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.090+0000 7fc4027fc700 1 --2- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3ec038200 0x7fc3ec03a6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8006fd0 tx=0x7fc3f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.094+0000 7fc3ebfff700 1 -- 172.21.15.99:0/3087204390 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc3f402abf0 con 0x7fc404100430 2023-08-25T08:51:09.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.291+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fc40402cd20 con 0x7fc3ec038200 2023-08-25T08:51:09.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.296+0000 7fc3ebfff700 1 -- 172.21.15.99:0/3087204390 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc40402cd20 con 0x7fc3ec038200 2023-08-25T08:51:09.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.298+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3ec038200 msgr2=0x7fc3ec03a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:09.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.298+0000 7fc4097a9700 1 --2- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3ec038200 0x7fc3ec03a6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8006fd0 tx=0x7fc3f8006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:09.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.298+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 msgr2=0x7fc404195540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:09.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.298+0000 7fc4097a9700 1 --2- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 0x7fc404195540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4004f40 tx=0x7fc3f40057c0 comp rx=0 tx=0).stop 2023-08-25T08:51:09.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.298+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 shutdown_connections 2023-08-25T08:51:09.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.298+0000 7fc4097a9700 1 --2- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3ec038200 0x7fc3ec03a6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:09.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.298+0000 7fc4097a9700 1 --2- 172.21.15.99:0/3087204390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc404100430 0x7fc404195540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:09.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.298+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 >> 172.21.15.99:0/3087204390 conn(0x7fc4040fb9b0 msgr2=0x7fc4040fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:09.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.299+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 shutdown_connections 2023-08-25T08:51:09.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:09.299+0000 7fc4097a9700 1 -- 172.21.15.99:0/3087204390 wait complete. 2023-08-25T08:51:09.691 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2023-08-25T08:51:09.691 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:51:09.692 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2023-08-25T08:51:09.713 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:51:09.713 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:09.773 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2023-08-25T08:51:09.774 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch host add smithi157 2023-08-25T08:51:10.515 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:10.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:10 smithi099 ceph-mon[97641]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:10.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:10.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:10.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:10.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:10.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:10 smithi099 ceph-mon[97641]: Updating smithi099:/etc/ceph/ceph.conf 2023-08-25T08:51:10.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:10 smithi099 ceph-mon[97641]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:11.403 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:11 smithi099 ceph-mon[97641]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:11.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:11 smithi099 ceph-mon[97641]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:51:12.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.042+0000 7fb64e5ce700 1 -- 172.21.15.99:0/4118724453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 msgr2=0x7fb6481009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:12.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.042+0000 7fb64e5ce700 1 --2- 172.21.15.99:0/4118724453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 0x7fb6481009f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb638009910 tx=0x7fb638009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:12.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.042+0000 7fb64e5ce700 1 -- 172.21.15.99:0/4118724453 shutdown_connections 2023-08-25T08:51:12.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.042+0000 7fb64e5ce700 1 --2- 172.21.15.99:0/4118724453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 0x7fb6481009f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:12.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.042+0000 7fb64e5ce700 1 -- 172.21.15.99:0/4118724453 >> 172.21.15.99:0/4118724453 conn(0x7fb6480fbee0 msgr2=0x7fb6480fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:12.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.042+0000 7fb64e5ce700 1 -- 172.21.15.99:0/4118724453 shutdown_connections 2023-08-25T08:51:12.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.042+0000 7fb64e5ce700 1 -- 172.21.15.99:0/4118724453 wait complete. 2023-08-25T08:51:12.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.043+0000 7fb64e5ce700 1 Processor -- start 2023-08-25T08:51:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.043+0000 7fb64e5ce700 1 -- start start 2023-08-25T08:51:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.044+0000 7fb64e5ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 0x7fb64819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.044+0000 7fb64e5ce700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64819de90 con 0x7fb648100610 2023-08-25T08:51:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.044+0000 7fb647fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 0x7fb64819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.044+0000 7fb647fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 0x7fb64819d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53108/0 (socket says 172.21.15.99:53108) 2023-08-25T08:51:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.044+0000 7fb647fff700 1 -- 172.21.15.99:0/637339670 learned_addr learned my addr 172.21.15.99:0/637339670 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.044+0000 7fb647fff700 1 -- 172.21.15.99:0/637339670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380095f0 con 0x7fb648100610 2023-08-25T08:51:12.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.045+0000 7fb647fff700 1 --2- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 0x7fb64819d950 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb638006010 tx=0x7fb638004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:12.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.045+0000 7fb6457fa700 1 -- 172.21.15.99:0/637339670 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb63801d070 con 0x7fb648100610 2023-08-25T08:51:12.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.045+0000 7fb6457fa700 1 -- 172.21.15.99:0/637339670 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63800bc10 con 0x7fb648100610 2023-08-25T08:51:12.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.045+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64819e090 con 0x7fb648100610 2023-08-25T08:51:12.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.045+0000 7fb6457fa700 1 -- 172.21.15.99:0/637339670 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb638003910 con 0x7fb648100610 2023-08-25T08:51:12.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.045+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb648197940 con 0x7fb648100610 2023-08-25T08:51:12.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.046+0000 7fb6457fa700 1 -- 172.21.15.99:0/637339670 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fb63800bd80 con 0x7fb648100610 2023-08-25T08:51:12.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.046+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64819e4b0 con 0x7fb648100610 2023-08-25T08:51:12.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.046+0000 7fb6457fa700 1 --2- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb630038200 0x7fb63003a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:12.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.046+0000 7fb6457fa700 1 -- 172.21.15.99:0/637339670 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb63804bdf0 con 0x7fb648100610 2023-08-25T08:51:12.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.047+0000 7fb6477fe700 1 --2- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb630038200 0x7fb63003a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:12.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.047+0000 7fb6477fe700 1 --2- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb630038200 0x7fb63003a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb63c006fd0 tx=0x7fb63c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:12.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.051+0000 7fb6457fa700 1 -- 172.21.15.99:0/637339670 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb638029360 con 0x7fb648100610 2023-08-25T08:51:12.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:12.254+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) v1 -- 0x7fb64810a700 con 0x7fb630038200 2023-08-25T08:51:12.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:12 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:12.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:12 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:12.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:12 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:13.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:13 smithi099 ceph-mon[97641]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:13.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:13 smithi099 ceph-mon[97641]: Deploying cephadm binary to smithi157 2023-08-25T08:51:14.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.576+0000 7fb6457fa700 1 -- 172.21.15.99:0/637339670 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb64810a700 con 0x7fb630038200 2023-08-25T08:51:14.577 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi157' with addr '172.21.15.157' 2023-08-25T08:51:14.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb630038200 msgr2=0x7fb63003a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:14.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 --2- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb630038200 0x7fb63003a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb63c006fd0 tx=0x7fb63c006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 msgr2=0x7fb64819d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 --2- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 0x7fb64819d950 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb638006010 tx=0x7fb638004ff0 comp rx=0 tx=0).stop 2023-08-25T08:51:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 shutdown_connections 2023-08-25T08:51:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 --2- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb630038200 0x7fb63003a6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 --2- 172.21.15.99:0/637339670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb648100610 0x7fb64819d950 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 >> 172.21.15.99:0/637339670 conn(0x7fb6480fbee0 msgr2=0x7fb648103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:14.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 shutdown_connections 2023-08-25T08:51:14.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:14.579+0000 7fb64e5ce700 1 -- 172.21.15.99:0/637339670 wait complete. 2023-08-25T08:51:15.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch host ls --format=json 2023-08-25T08:51:15.727 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:15.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:15 smithi099 ceph-mon[97641]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:15.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:15.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:15 smithi099 ceph-mon[97641]: Added host smithi157 2023-08-25T08:51:15.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:15.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:16.930 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:16 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.105+0000 7f5a803c5700 1 -- 172.21.15.99:0/2593454433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 msgr2=0x7f5a78106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.105+0000 7f5a803c5700 1 --2- 172.21.15.99:0/2593454433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 0x7f5a78106820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009910 tx=0x7f5a74009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.105+0000 7f5a803c5700 1 -- 172.21.15.99:0/2593454433 shutdown_connections 2023-08-25T08:51:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.105+0000 7f5a803c5700 1 --2- 172.21.15.99:0/2593454433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 0x7f5a78106820 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.105+0000 7f5a803c5700 1 -- 172.21.15.99:0/2593454433 >> 172.21.15.99:0/2593454433 conn(0x7f5a78074e20 msgr2=0x7f5a78075230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:17.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.105+0000 7f5a803c5700 1 -- 172.21.15.99:0/2593454433 shutdown_connections 2023-08-25T08:51:17.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.105+0000 7f5a803c5700 1 -- 172.21.15.99:0/2593454433 wait complete. 2023-08-25T08:51:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.106+0000 7f5a803c5700 1 Processor -- start 2023-08-25T08:51:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.106+0000 7f5a803c5700 1 -- start start 2023-08-25T08:51:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.107+0000 7f5a803c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 0x7f5a7819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.107+0000 7f5a803c5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7819dd70 con 0x7f5a78106440 2023-08-25T08:51:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.107+0000 7f5a7e161700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 0x7f5a7819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.107+0000 7f5a7e161700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 0x7f5a7819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35400/0 (socket says 172.21.15.99:35400) 2023-08-25T08:51:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.107+0000 7f5a7e161700 1 -- 172.21.15.99:0/1920672988 learned_addr learned my addr 172.21.15.99:0/1920672988 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.107+0000 7f5a7e161700 1 -- 172.21.15.99:0/1920672988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740095f0 con 0x7f5a78106440 2023-08-25T08:51:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.107+0000 7f5a7e161700 1 --2- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 0x7f5a7819d830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a74006010 tx=0x7f5a74004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.108+0000 7f5a6f7fe700 1 -- 172.21.15.99:0/1920672988 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a7401d070 con 0x7f5a78106440 2023-08-25T08:51:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.108+0000 7f5a6f7fe700 1 -- 172.21.15.99:0/1920672988 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a7400bbb0 con 0x7f5a78106440 2023-08-25T08:51:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.108+0000 7f5a6f7fe700 1 -- 172.21.15.99:0/1920672988 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a74003930 con 0x7f5a78106440 2023-08-25T08:51:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.108+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7819dfd0 con 0x7f5a78106440 2023-08-25T08:51:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.108+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a781978b0 con 0x7f5a78106440 2023-08-25T08:51:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.109+0000 7f5a6f7fe700 1 -- 172.21.15.99:0/1920672988 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f5a7400bd20 con 0x7f5a78106440 2023-08-25T08:51:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.109+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a78061f50 con 0x7f5a78106440 2023-08-25T08:51:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.109+0000 7f5a6f7fe700 1 --2- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5a64038200 0x7f5a6403a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.109+0000 7f5a6f7fe700 1 -- 172.21.15.99:0/1920672988 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5a7404be00 con 0x7f5a78106440 2023-08-25T08:51:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.109+0000 7f5a7d960700 1 --2- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5a64038200 0x7f5a6403a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.110+0000 7f5a7d960700 1 --2- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5a64038200 0x7f5a6403a6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a68006fd0 tx=0x7f5a68006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:17.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.114+0000 7f5a6f7fe700 1 -- 172.21.15.99:0/1920672988 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5a74029360 con 0x7f5a78106440 2023-08-25T08:51:17.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.314+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a7810a710 con 0x7f5a64038200 2023-08-25T08:51:17.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.315+0000 7f5a6f7fe700 1 -- 172.21.15.99:0/1920672988 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f5a7810a710 con 0x7f5a64038200 2023-08-25T08:51:17.316 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:17.317 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2023-08-25T08:51:17.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5a64038200 msgr2=0x7f5a6403a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:17.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 --2- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5a64038200 0x7f5a6403a6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a68006fd0 tx=0x7f5a68006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:17.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 msgr2=0x7f5a7819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:17.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 --2- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 0x7f5a7819d830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a74006010 tx=0x7f5a74004ff0 comp rx=0 tx=0).stop 2023-08-25T08:51:17.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 shutdown_connections 2023-08-25T08:51:17.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 --2- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5a64038200 0x7f5a6403a6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:17.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 --2- 172.21.15.99:0/1920672988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a78106440 0x7f5a7819d830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:17.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 >> 172.21.15.99:0/1920672988 conn(0x7f5a78074e20 msgr2=0x7f5a781021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:17.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 shutdown_connections 2023-08-25T08:51:17.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:17.318+0000 7f5a803c5700 1 -- 172.21.15.99:0/1920672988 wait complete. 2023-08-25T08:51:17.804 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2023-08-25T08:51:17.804 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:51:17.804 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2023-08-25T08:51:17.825 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:51:17.825 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:17.884 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2023-08-25T08:51:17.884 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch host add smithi181 2023-08-25T08:51:17.946 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:17 smithi099 ceph-mon[97641]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:17.946 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:18.600 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:18.985 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:18 smithi099 ceph-mon[97641]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:51:20.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.032+0000 7f9273879700 1 -- 172.21.15.99:0/167932078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 msgr2=0x7f926c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:20.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.032+0000 7f9273879700 1 --2- 172.21.15.99:0/167932078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 0x7f926c1009f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9268009910 tx=0x7f9268009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:20.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.032+0000 7f9273879700 1 -- 172.21.15.99:0/167932078 shutdown_connections 2023-08-25T08:51:20.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.032+0000 7f9273879700 1 --2- 172.21.15.99:0/167932078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 0x7f926c1009f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:20.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.032+0000 7f9273879700 1 -- 172.21.15.99:0/167932078 >> 172.21.15.99:0/167932078 conn(0x7f926c0fbee0 msgr2=0x7f926c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:20.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.032+0000 7f9273879700 1 -- 172.21.15.99:0/167932078 shutdown_connections 2023-08-25T08:51:20.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.032+0000 7f9273879700 1 -- 172.21.15.99:0/167932078 wait complete. 2023-08-25T08:51:20.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.033+0000 7f9273879700 1 Processor -- start 2023-08-25T08:51:20.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.033+0000 7f9273879700 1 -- start start 2023-08-25T08:51:20.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.034+0000 7f9273879700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 0x7f926c19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:20.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.034+0000 7f9273879700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c19bb60 con 0x7f926c100610 2023-08-25T08:51:20.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.034+0000 7f9271615700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 0x7f926c19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:20.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.034+0000 7f9271615700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 0x7f926c19b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35406/0 (socket says 172.21.15.99:35406) 2023-08-25T08:51:20.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.034+0000 7f9271615700 1 -- 172.21.15.99:0/2288190905 learned_addr learned my addr 172.21.15.99:0/2288190905 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:20.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.035+0000 7f9271615700 1 -- 172.21.15.99:0/2288190905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92680095f0 con 0x7f926c100610 2023-08-25T08:51:20.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.035+0000 7f9271615700 1 --2- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 0x7f926c19b620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9268006010 tx=0x7f9268004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:20.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.035+0000 7f92627fc700 1 -- 172.21.15.99:0/2288190905 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f926801d070 con 0x7f926c100610 2023-08-25T08:51:20.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.035+0000 7f92627fc700 1 -- 172.21.15.99:0/2288190905 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f926800bc10 con 0x7f926c100610 2023-08-25T08:51:20.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.035+0000 7f92627fc700 1 -- 172.21.15.99:0/2288190905 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9268003910 con 0x7f926c100610 2023-08-25T08:51:20.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.035+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c19bd60 con 0x7f926c100610 2023-08-25T08:51:20.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.035+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c19c180 con 0x7f926c100610 2023-08-25T08:51:20.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.036+0000 7f92627fc700 1 -- 172.21.15.99:0/2288190905 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f926800bd80 con 0x7f926c100610 2023-08-25T08:51:20.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.036+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c061f50 con 0x7f926c100610 2023-08-25T08:51:20.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.037+0000 7f92627fc700 1 --2- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9258038200 0x7f925803a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:20.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.037+0000 7f92627fc700 1 -- 172.21.15.99:0/2288190905 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f926804bd00 con 0x7f926c100610 2023-08-25T08:51:20.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.037+0000 7f9270e14700 1 --2- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9258038200 0x7f925803a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:20.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.037+0000 7f9270e14700 1 --2- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9258038200 0x7f925803a6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f925c006fd0 tx=0x7f925c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:20.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.042+0000 7f92627fc700 1 -- 172.21.15.99:0/2288190905 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9268029360 con 0x7f926c100610 2023-08-25T08:51:20.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:19 smithi099 ceph-mon[97641]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:20.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:20.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:20.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:20.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:20.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:51:20.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:20.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:20.285+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) v1 -- 0x7f926c0008d0 con 0x7f9258038200 2023-08-25T08:51:21.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:20 smithi099 ceph-mon[97641]: Updating smithi157:/etc/ceph/ceph.conf 2023-08-25T08:51:21.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:20 smithi099 ceph-mon[97641]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:21.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:21.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:21.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:22.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:21 smithi099 ceph-mon[97641]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:22.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:21 smithi099 ceph-mon[97641]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:51:22.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:21 smithi099 ceph-mon[97641]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:22.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:21 smithi099 ceph-mon[97641]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:22.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:21 smithi099 ceph-mon[97641]: Deploying cephadm binary to smithi181 2023-08-25T08:51:22.603 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi181' with addr '172.21.15.181' 2023-08-25T08:51:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.603+0000 7f92627fc700 1 -- 172.21.15.99:0/2288190905 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f926c0008d0 con 0x7f9258038200 2023-08-25T08:51:22.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9258038200 msgr2=0x7f925803a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:22.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 --2- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9258038200 0x7f925803a6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f925c006fd0 tx=0x7f925c006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:22.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 msgr2=0x7f926c19b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:22.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 --2- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 0x7f926c19b620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9268006010 tx=0x7f9268004ff0 comp rx=0 tx=0).stop 2023-08-25T08:51:22.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 shutdown_connections 2023-08-25T08:51:22.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 --2- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9258038200 0x7f925803a6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:22.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 --2- 172.21.15.99:0/2288190905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f926c100610 0x7f926c19b620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:22.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 >> 172.21.15.99:0/2288190905 conn(0x7f926c0fbee0 msgr2=0x7f926c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:22.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 shutdown_connections 2023-08-25T08:51:22.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:22.606+0000 7f9273879700 1 -- 172.21.15.99:0/2288190905 wait complete. 2023-08-25T08:51:23.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch host ls --format=json 2023-08-25T08:51:23.896 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:24.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:23 smithi099 ceph-mon[97641]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:24.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:24.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:23 smithi099 ceph-mon[97641]: Added host smithi181 2023-08-25T08:51:24.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:24.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:24.719 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:24 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.354+0000 7f2e12d25700 1 -- 172.21.15.99:0/2135713444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 msgr2=0x7f2e0c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:25.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.354+0000 7f2e12d25700 1 --2- 172.21.15.99:0/2135713444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 0x7f2e0c1009f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009910 tx=0x7f2e00009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:25.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.354+0000 7f2e12d25700 1 -- 172.21.15.99:0/2135713444 shutdown_connections 2023-08-25T08:51:25.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.354+0000 7f2e12d25700 1 --2- 172.21.15.99:0/2135713444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 0x7f2e0c1009f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:25.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.354+0000 7f2e12d25700 1 -- 172.21.15.99:0/2135713444 >> 172.21.15.99:0/2135713444 conn(0x7f2e0c0fbee0 msgr2=0x7f2e0c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:25.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.354+0000 7f2e12d25700 1 -- 172.21.15.99:0/2135713444 shutdown_connections 2023-08-25T08:51:25.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.354+0000 7f2e12d25700 1 -- 172.21.15.99:0/2135713444 wait complete. 2023-08-25T08:51:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.355+0000 7f2e12d25700 1 Processor -- start 2023-08-25T08:51:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.356+0000 7f2e12d25700 1 -- start start 2023-08-25T08:51:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.356+0000 7f2e12d25700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 0x7f2e0c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.356+0000 7f2e12d25700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c19de90 con 0x7f2e0c100610 2023-08-25T08:51:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.356+0000 7f2e10ac1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 0x7f2e0c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.356+0000 7f2e10ac1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 0x7f2e0c19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35426/0 (socket says 172.21.15.99:35426) 2023-08-25T08:51:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.356+0000 7f2e10ac1700 1 -- 172.21.15.99:0/1461677879 learned_addr learned my addr 172.21.15.99:0/1461677879 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.357+0000 7f2e10ac1700 1 -- 172.21.15.99:0/1461677879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e000095f0 con 0x7f2e0c100610 2023-08-25T08:51:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.357+0000 7f2e10ac1700 1 --2- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 0x7f2e0c19d950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e00006010 tx=0x7f2e00004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.357+0000 7f2e09ffb700 1 -- 172.21.15.99:0/1461677879 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e0001d070 con 0x7f2e0c100610 2023-08-25T08:51:25.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.357+0000 7f2e09ffb700 1 -- 172.21.15.99:0/1461677879 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e0000bc10 con 0x7f2e0c100610 2023-08-25T08:51:25.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.357+0000 7f2e09ffb700 1 -- 172.21.15.99:0/1461677879 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e00003910 con 0x7f2e0c100610 2023-08-25T08:51:25.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.357+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c19e090 con 0x7f2e0c100610 2023-08-25T08:51:25.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.357+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c197940 con 0x7f2e0c100610 2023-08-25T08:51:25.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.358+0000 7f2e09ffb700 1 -- 172.21.15.99:0/1461677879 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f2e0000bd80 con 0x7f2e0c100610 2023-08-25T08:51:25.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.358+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0c19e4b0 con 0x7f2e0c100610 2023-08-25T08:51:25.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.359+0000 7f2e09ffb700 1 --2- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2dfc038200 0x7f2dfc03a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:25.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.359+0000 7f2e09ffb700 1 -- 172.21.15.99:0/1461677879 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2e0004bdc0 con 0x7f2e0c100610 2023-08-25T08:51:25.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.359+0000 7f2e0bfff700 1 --2- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2dfc038200 0x7f2dfc03a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:25.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.359+0000 7f2e0bfff700 1 --2- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2dfc038200 0x7f2dfc03a6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2df8006fd0 tx=0x7f2df8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:25.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.364+0000 7f2e09ffb700 1 -- 172.21.15.99:0/1461677879 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f2e00029360 con 0x7f2e0c100610 2023-08-25T08:51:25.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.565+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e0c10a770 con 0x7f2dfc038200 2023-08-25T08:51:25.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.566+0000 7f2e09ffb700 1 -- 172.21.15.99:0/1461677879 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f2e0c10a770 con 0x7f2dfc038200 2023-08-25T08:51:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:51:25.567 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2023-08-25T08:51:25.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2dfc038200 msgr2=0x7f2dfc03a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:25.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 --2- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2dfc038200 0x7f2dfc03a6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2df8006fd0 tx=0x7f2df8006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:25.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 msgr2=0x7f2e0c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:25.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 --2- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 0x7f2e0c19d950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e00006010 tx=0x7f2e00004ff0 comp rx=0 tx=0).stop 2023-08-25T08:51:25.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 shutdown_connections 2023-08-25T08:51:25.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 --2- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2dfc038200 0x7f2dfc03a6c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:25.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 --2- 172.21.15.99:0/1461677879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c100610 0x7f2e0c19d950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:25.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 >> 172.21.15.99:0/1461677879 conn(0x7f2e0c0fbee0 msgr2=0x7f2e0c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:25.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 shutdown_connections 2023-08-25T08:51:25.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:25.569+0000 7f2e12d25700 1 -- 172.21.15.99:0/1461677879 wait complete. 2023-08-25T08:51:26.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:25 smithi099 ceph-mon[97641]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:26.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:25 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:26.062 INFO:tasks.cephadm:Setting crush tunables to default 2023-08-25T08:51:26.063 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd crush tunables default 2023-08-25T08:51:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:27.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:26 smithi099 ceph-mon[97641]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:51:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:27 smithi099 ceph-mon[97641]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:51:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:28.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.181+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/4095540585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 msgr2=0x7f4f9c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:28.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.181+0000 7f4fa1dbe700 1 --2- 172.21.15.99:0/4095540585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 0x7f4f9c108a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009910 tx=0x7f4f8c009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:28.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.181+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/4095540585 shutdown_connections 2023-08-25T08:51:28.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.181+0000 7f4fa1dbe700 1 --2- 172.21.15.99:0/4095540585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 0x7f4f9c108a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:28.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.181+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/4095540585 >> 172.21.15.99:0/4095540585 conn(0x7f4f9c0fe040 msgr2=0x7f4f9c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:28.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.181+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/4095540585 shutdown_connections 2023-08-25T08:51:28.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.181+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/4095540585 wait complete. 2023-08-25T08:51:28.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.182+0000 7f4fa1dbe700 1 Processor -- start 2023-08-25T08:51:28.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.182+0000 7f4fa1dbe700 1 -- start start 2023-08-25T08:51:28.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.183+0000 7f4fa1dbe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 0x7f4f9c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:28.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.183+0000 7f4fa1dbe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c197e90 con 0x7f4f9c108660 2023-08-25T08:51:28.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.183+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 0x7f4f9c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:28.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.183+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 0x7f4f9c197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38526/0 (socket says 172.21.15.99:38526) 2023-08-25T08:51:28.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.183+0000 7f4f9b7fe700 1 -- 172.21.15.99:0/2581435798 learned_addr learned my addr 172.21.15.99:0/2581435798 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:28.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.183+0000 7f4f9b7fe700 1 -- 172.21.15.99:0/2581435798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c0095f0 con 0x7f4f9c108660 2023-08-25T08:51:28.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.183+0000 7f4f9b7fe700 1 --2- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 0x7f4f9c197950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006010 tx=0x7f4f8c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:28.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.184+0000 7f4f98ff9700 1 -- 172.21.15.99:0/2581435798 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f8c01d070 con 0x7f4f9c108660 2023-08-25T08:51:28.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.184+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c1980f0 con 0x7f4f9c108660 2023-08-25T08:51:28.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.184+0000 7f4f98ff9700 1 -- 172.21.15.99:0/2581435798 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8c00bc10 con 0x7f4f9c108660 2023-08-25T08:51:28.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.184+0000 7f4f98ff9700 1 -- 172.21.15.99:0/2581435798 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f8c003910 con 0x7f4f9c108660 2023-08-25T08:51:28.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.184+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c1985f0 con 0x7f4f9c108660 2023-08-25T08:51:28.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.185+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c061f50 con 0x7f4f9c108660 2023-08-25T08:51:28.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.185+0000 7f4f98ff9700 1 -- 172.21.15.99:0/2581435798 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f4f8c00bd80 con 0x7f4f9c108660 2023-08-25T08:51:28.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.185+0000 7f4f98ff9700 1 --2- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f840381b0 0x7f4f8403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:28.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.185+0000 7f4f98ff9700 1 -- 172.21.15.99:0/2581435798 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f8c04bc80 con 0x7f4f9c108660 2023-08-25T08:51:28.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.186+0000 7f4f9affd700 1 --2- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f840381b0 0x7f4f8403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:28.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.186+0000 7f4f9affd700 1 --2- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f840381b0 0x7f4f8403a670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4f90006fd0 tx=0x7f4f90006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:28.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.190+0000 7f4f98ff9700 1 -- 172.21.15.99:0/2581435798 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4f8c029360 con 0x7f4f9c108660 2023-08-25T08:51:28.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.403+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4f9c1985f0 con 0x7f4f9c108660 2023-08-25T08:51:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.686+0000 7f4f98ff9700 1 -- 172.21.15.99:0/2581435798 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4f9c1985f0 con 0x7f4f9c108660 2023-08-25T08:51:28.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f840381b0 msgr2=0x7f4f8403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:28.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 --2- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f840381b0 0x7f4f8403a670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4f90006fd0 tx=0x7f4f90006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 msgr2=0x7f4f9c197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 --2- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 0x7f4f9c197950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006010 tx=0x7f4f8c004ff0 comp rx=0 tx=0).stop 2023-08-25T08:51:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 shutdown_connections 2023-08-25T08:51:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 --2- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f840381b0 0x7f4f8403a670 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 --2- 172.21.15.99:0/2581435798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f9c108660 0x7f4f9c197950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 >> 172.21.15.99:0/2581435798 conn(0x7f4f9c0fe040 msgr2=0x7f4f9c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 shutdown_connections 2023-08-25T08:51:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:28.689+0000 7f4fa1dbe700 1 -- 172.21.15.99:0/2581435798 wait complete. 2023-08-25T08:51:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2023-08-25T08:51:29.049 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:28 smithi099 ceph-mon[97641]: Updating smithi181:/etc/ceph/ceph.conf 2023-08-25T08:51:29.050 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:28 smithi099 ceph-mon[97641]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:29.050 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:28 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2581435798' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-25T08:51:29.050 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:28 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:29.051 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:28 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:29.051 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:28 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:29.065 INFO:tasks.cephadm:Adding mon.a on smithi099 2023-08-25T08:51:29.066 INFO:tasks.cephadm:Adding mon.b on smithi157 2023-08-25T08:51:29.066 INFO:tasks.cephadm:Adding mon.c on smithi181 2023-08-25T08:51:29.066 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch apply mon '3;smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c' 2023-08-25T08:51:29.643 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:30.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:29 smithi099 ceph-mon[97641]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:30.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:29 smithi099 ceph-mon[97641]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:51:30.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:29 smithi099 ceph-mon[97641]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:30.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:29 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2581435798' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-25T08:51:30.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:29 smithi099 ceph-mon[97641]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:30.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.824+0000 7fee5e940700 1 -- 172.21.15.181:0/3362227593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 msgr2=0x7fee58105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:30.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.824+0000 7fee5e940700 1 --2- 172.21.15.181:0/3362227593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 0x7fee58105710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009910 tx=0x7fee4c009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:30.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.824+0000 7fee5e940700 1 -- 172.21.15.181:0/3362227593 shutdown_connections 2023-08-25T08:51:30.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.824+0000 7fee5e940700 1 --2- 172.21.15.181:0/3362227593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 0x7fee58105710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:30.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.824+0000 7fee5e940700 1 -- 172.21.15.181:0/3362227593 >> 172.21.15.181:0/3362227593 conn(0x7fee58074e20 msgr2=0x7fee58075230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:30.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.824+0000 7fee5e940700 1 -- 172.21.15.181:0/3362227593 shutdown_connections 2023-08-25T08:51:30.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.824+0000 7fee5e940700 1 -- 172.21.15.181:0/3362227593 wait complete. 2023-08-25T08:51:30.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.825+0000 7fee5e940700 1 Processor -- start 2023-08-25T08:51:30.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.825+0000 7fee5e940700 1 -- start start 2023-08-25T08:51:30.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.825+0000 7fee5e940700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 0x7fee581966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:30.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.825+0000 7fee5e940700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58196bf0 con 0x7fee58105330 2023-08-25T08:51:30.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee57fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 0x7fee581966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:30.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee57fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 0x7fee581966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.181:49190/0 (socket says 172.21.15.181:49190) 2023-08-25T08:51:30.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee57fff700 1 -- 172.21.15.181:0/1362647477 learned_addr learned my addr 172.21.15.181:0/1362647477 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:51:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee57fff700 1 -- 172.21.15.181:0/1362647477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0095f0 con 0x7fee58105330 2023-08-25T08:51:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee57fff700 1 --2- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 0x7fee581966b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006010 tx=0x7fee4c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee557fa700 1 -- 172.21.15.181:0/1362647477 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4c01d070 con 0x7fee58105330 2023-08-25T08:51:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee58196e50 con 0x7fee58105330 2023-08-25T08:51:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee557fa700 1 -- 172.21.15.181:0/1362647477 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee4c00bbb0 con 0x7fee58105330 2023-08-25T08:51:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee557fa700 1 -- 172.21.15.181:0/1362647477 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4c003930 con 0x7fee58105330 2023-08-25T08:51:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.826+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581972d0 con 0x7fee58105330 2023-08-25T08:51:30.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.827+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee58197510 con 0x7fee58105330 2023-08-25T08:51:30.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.827+0000 7fee557fa700 1 -- 172.21.15.181:0/1362647477 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fee4c00bd20 con 0x7fee58105330 2023-08-25T08:51:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.827+0000 7fee557fa700 1 --2- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee400381b0 0x7fee4003a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.828+0000 7fee557fa700 1 -- 172.21.15.181:0/1362647477 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee4c04be10 con 0x7fee58105330 2023-08-25T08:51:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.828+0000 7fee577fe700 1 --2- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee400381b0 0x7fee4003a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.828+0000 7fee577fe700 1 --2- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee400381b0 0x7fee4003a670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee48006fd0 tx=0x7fee48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:30.833+0000 7fee557fa700 1 -- 172.21.15.181:0/1362647477 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fee4c029360 con 0x7fee58105330 2023-08-25T08:51:31.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.029+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fee581095a0 con 0x7fee400381b0 2023-08-25T08:51:31.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.036+0000 7fee557fa700 1 -- 172.21.15.181:0/1362647477 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fee581095a0 con 0x7fee400381b0 2023-08-25T08:51:31.038 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mon update... 2023-08-25T08:51:31.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee400381b0 msgr2=0x7fee4003a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:31.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 --2- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee400381b0 0x7fee4003a670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee48006fd0 tx=0x7fee48006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:31.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 msgr2=0x7fee581966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:31.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 --2- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 0x7fee581966b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006010 tx=0x7fee4c004ff0 comp rx=0 tx=0).stop 2023-08-25T08:51:31.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 shutdown_connections 2023-08-25T08:51:31.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 --2- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee400381b0 0x7fee4003a670 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 --2- 172.21.15.181:0/1362647477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee58105330 0x7fee581966b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 >> 172.21.15.181:0/1362647477 conn(0x7fee58074e20 msgr2=0x7fee580fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 shutdown_connections 2023-08-25T08:51:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:31.038+0000 7fee5e940700 1 -- 172.21.15.181:0/1362647477 wait complete. 2023-08-25T08:51:31.422 DEBUG:teuthology.orchestra.run.smithi157:mon.b> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.b.service 2023-08-25T08:51:31.426 DEBUG:teuthology.orchestra.run.smithi181:mon.c> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.c.service 2023-08-25T08:51:31.429 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-25T08:51:31.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph mon dump -f json 2023-08-25T08:51:31.456 INFO:journalctl@ceph.mon.c.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:51:31.462 INFO:journalctl@ceph.mon.b.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:51:32.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:31 smithi099 ceph-mon[97641]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:32.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:32.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:32.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:32.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:32.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:32.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:32.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:32.313 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:33.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:32 smithi099 ceph-mon[97641]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:33.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:32 smithi099 ceph-mon[97641]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c;count:3 2023-08-25T08:51:33.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:32 smithi099 ceph-mon[97641]: Deploying daemon mon.c on smithi181 2023-08-25T08:51:33.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.872+0000 7f20aa409700 1 -- 172.21.15.181:0/3927241581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 msgr2=0x7f20a4105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.872+0000 7f20aa409700 1 --2- 172.21.15.181:0/3927241581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 0x7f20a4105710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2094009910 tx=0x7f2094009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.872+0000 7f20aa409700 1 -- 172.21.15.181:0/3927241581 shutdown_connections 2023-08-25T08:51:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.872+0000 7f20aa409700 1 --2- 172.21.15.181:0/3927241581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 0x7f20a4105710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.872+0000 7f20aa409700 1 -- 172.21.15.181:0/3927241581 >> 172.21.15.181:0/3927241581 conn(0x7f20a4074e20 msgr2=0x7f20a4075230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.872+0000 7f20aa409700 1 -- 172.21.15.181:0/3927241581 shutdown_connections 2023-08-25T08:51:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.872+0000 7f20aa409700 1 -- 172.21.15.181:0/3927241581 wait complete. 2023-08-25T08:51:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.873+0000 7f20aa409700 1 Processor -- start 2023-08-25T08:51:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.873+0000 7f20aa409700 1 -- start start 2023-08-25T08:51:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.873+0000 7f20aa409700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 0x7f20a419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.873+0000 7f20aa409700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a419bb70 con 0x7f20a4105330 2023-08-25T08:51:33.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.873+0000 7f20a3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 0x7f20a419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:33.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.873+0000 7f20a3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 0x7f20a419b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.181:49196/0 (socket says 172.21.15.181:49196) 2023-08-25T08:51:33.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.873+0000 7f20a3fff700 1 -- 172.21.15.181:0/1750536009 learned_addr learned my addr 172.21.15.181:0/1750536009 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:51:33.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.874+0000 7f20a3fff700 1 -- 172.21.15.181:0/1750536009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940095f0 con 0x7f20a4105330 2023-08-25T08:51:33.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.874+0000 7f20a3fff700 1 --2- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 0x7f20a419b630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2094004f40 tx=0x7f2094005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:33.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.874+0000 7f20a17fa700 1 -- 172.21.15.181:0/1750536009 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f209401d070 con 0x7f20a4105330 2023-08-25T08:51:33.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.874+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a419bdd0 con 0x7f20a4105330 2023-08-25T08:51:33.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.874+0000 7f20a17fa700 1 -- 172.21.15.181:0/1750536009 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2094004370 con 0x7f20a4105330 2023-08-25T08:51:33.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.874+0000 7f20a17fa700 1 -- 172.21.15.181:0/1750536009 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2094003970 con 0x7f20a4105330 2023-08-25T08:51:33.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.874+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a41956b0 con 0x7f20a4105330 2023-08-25T08:51:33.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.875+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a4061f50 con 0x7f20a4105330 2023-08-25T08:51:33.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.875+0000 7f20a17fa700 1 -- 172.21.15.181:0/1750536009 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f2094003c30 con 0x7f20a4105330 2023-08-25T08:51:33.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.875+0000 7f20a17fa700 1 --2- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f208c0381b0 0x7f208c03a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:33.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.875+0000 7f20a17fa700 1 -- 172.21.15.181:0/1750536009 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f209404bfb0 con 0x7f20a4105330 2023-08-25T08:51:33.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.876+0000 7f20a37fe700 1 --2- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f208c0381b0 0x7f208c03a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:33.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.876+0000 7f20a37fe700 1 --2- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f208c0381b0 0x7f208c03a670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2098006fd0 tx=0x7f2098006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:33.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:33.881+0000 7f20a17fa700 1 -- 172.21.15.181:0/1750536009 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f209407ee00 con 0x7f20a4105330 2023-08-25T08:51:34.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:33 smithi099 ceph-mon[97641]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.171+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f20a4196bc0 con 0x7f20a4105330 2023-08-25T08:51:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.172+0000 7f20a17fa700 1 -- 172.21.15.181:0/1750536009 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f2094026030 con 0x7f20a4105330 2023-08-25T08:51:34.173 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:51:34.174 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","modified":"2023-08-25T08:49:48.573313Z","created":"2023-08-25T08:49:48.573313Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-08-25T08:51:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.175+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f208c0381b0 msgr2=0x7f208c03a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.175+0000 7f20aa409700 1 --2- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f208c0381b0 0x7f208c03a670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2098006fd0 tx=0x7f2098006e40 comp rx=0 tx=0).stop 2023-08-25T08:51:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.175+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 msgr2=0x7f20a419b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.175+0000 7f20aa409700 1 --2- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 0x7f20a419b630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2094004f40 tx=0x7f2094005020 comp rx=0 tx=0).stop 2023-08-25T08:51:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.176+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 shutdown_connections 2023-08-25T08:51:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.176+0000 7f20aa409700 1 --2- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f208c0381b0 0x7f208c03a670 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.176+0000 7f20aa409700 1 --2- 172.21.15.181:0/1750536009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a4105330 0x7f20a419b630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.176+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 >> 172.21.15.181:0/1750536009 conn(0x7f20a4074e20 msgr2=0x7f20a4102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.176+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 shutdown_connections 2023-08-25T08:51:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:34.176+0000 7f20aa409700 1 -- 172.21.15.181:0/1750536009 wait complete. 2023-08-25T08:51:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2023-08-25T08:51:34.877 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 podman[131254]: 2023-08-25T08:51:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: set uid:gid to 167:167 (ceph:ceph) 2023-08-25T08:51:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev), process ceph-mon, pid 7 2023-08-25T08:51:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 bash[131254]: 8c6cb3773acd6b263aa7a22e35d6ff0b997c8d3fcb080b0a2fc2c3bca7c25935 2023-08-25T08:51:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pidfile_write: ignore empty --pid-file 2023-08-25T08:51:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: load: jerasure load: lrc 2023-08-25T08:51:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: RocksDB version: 7.9.2 2023-08-25T08:51:35.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Git sha 0 2023-08-25T08:51:35.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Compile date 2023-08-24 22:08:31 2023-08-25T08:51:35.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: DB SUMMARY 2023-08-25T08:51:35.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: DB Session ID: JMMEP1NVMDHGFBVK2DJQ 2023-08-25T08:51:35.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: CURRENT file: CURRENT 2023-08-25T08:51:35.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: IDENTITY file: IDENTITY 2023-08-25T08:51:35.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-08-25T08:51:35.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-08-25T08:51:35.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-08-25T08:51:35.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.error_if_exists: 0 2023-08-25T08:51:35.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.create_if_missing: 0 2023-08-25T08:51:35.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.paranoid_checks: 1 2023-08-25T08:51:35.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-25T08:51:35.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-25T08:51:35.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-25T08:51:35.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.env: 0x55ff93054da0 2023-08-25T08:51:35.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.fs: PosixFileSystem 2023-08-25T08:51:35.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.info_log: 0x55ff93e49440 2023-08-25T08:51:35.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_file_opening_threads: 16 2023-08-25T08:51:35.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.statistics: (nil) 2023-08-25T08:51:35.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.use_fsync: 0 2023-08-25T08:51:35.247 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_log_file_size: 0 2023-08-25T08:51:35.247 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-25T08:51:35.247 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-25T08:51:35.247 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.keep_log_file_num: 1000 2023-08-25T08:51:35.247 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.recycle_log_file_num: 0 2023-08-25T08:51:35.247 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.allow_fallocate: 1 2023-08-25T08:51:35.247 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.allow_mmap_reads: 0 2023-08-25T08:51:35.247 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.allow_mmap_writes: 0 2023-08-25T08:51:35.248 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.use_direct_reads: 0 2023-08-25T08:51:35.248 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-25T08:51:35.248 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.create_missing_column_families: 0 2023-08-25T08:51:35.248 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.db_log_dir: 2023-08-25T08:51:35.248 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.wal_dir: 2023-08-25T08:51:35.248 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-25T08:51:35.248 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-25T08:51:35.249 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-25T08:51:35.249 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-25T08:51:35.249 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-25T08:51:35.249 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-25T08:51:35.249 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.advise_random_on_open: 1 2023-08-25T08:51:35.249 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.db_write_buffer_size: 0 2023-08-25T08:51:35.249 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.write_buffer_manager: 0x55ff93e034a0 2023-08-25T08:51:35.250 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-25T08:51:35.250 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-25T08:51:35.250 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-25T08:51:35.250 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.rate_limiter: (nil) 2023-08-25T08:51:35.250 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-25T08:51:35.250 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.wal_recovery_mode: 2 2023-08-25T08:51:35.250 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.enable_thread_tracking: 0 2023-08-25T08:51:35.250 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.enable_pipelined_write: 0 2023-08-25T08:51:35.251 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.unordered_write: 0 2023-08-25T08:51:35.251 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-25T08:51:35.251 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-25T08:51:35.251 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-25T08:51:35.251 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-25T08:51:35.251 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.row_cache: None 2023-08-25T08:51:35.251 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.wal_filter: None 2023-08-25T08:51:35.251 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-25T08:51:35.252 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.allow_ingest_behind: 0 2023-08-25T08:51:35.252 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.two_write_queues: 0 2023-08-25T08:51:35.252 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.manual_wal_flush: 0 2023-08-25T08:51:35.252 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.wal_compression: 0 2023-08-25T08:51:35.252 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.atomic_flush: 0 2023-08-25T08:51:35.252 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-25T08:51:35.252 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-25T08:51:35.253 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-25T08:51:35.253 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.log_readahead_size: 0 2023-08-25T08:51:35.253 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-25T08:51:35.253 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.best_efforts_recovery: 0 2023-08-25T08:51:35.253 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-25T08:51:35.253 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-25T08:51:35.253 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.allow_data_in_errors: 0 2023-08-25T08:51:35.255 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.db_host_id: __hostname__ 2023-08-25T08:51:35.255 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-25T08:51:35.255 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_background_jobs: 2 2023-08-25T08:51:35.255 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_background_compactions: -1 2023-08-25T08:51:35.255 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_subcompactions: 1 2023-08-25T08:51:35.255 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-25T08:51:35.256 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-25T08:51:35.256 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-25T08:51:35.256 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_total_wal_size: 0 2023-08-25T08:51:35.256 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-25T08:51:35.256 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-25T08:51:35.256 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-25T08:51:35.256 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-25T08:51:35.257 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_open_files: -1 2023-08-25T08:51:35.257 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bytes_per_sync: 0 2023-08-25T08:51:35.257 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-25T08:51:35.257 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-25T08:51:35.257 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_readahead_size: 0 2023-08-25T08:51:35.257 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_background_flushes: -1 2023-08-25T08:51:35.257 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Compression algorithms supported: 2023-08-25T08:51:35.257 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-25T08:51:35.258 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: kZSTD supported: 0 2023-08-25T08:51:35.258 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: kXpressCompression supported: 0 2023-08-25T08:51:35.258 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: kLZ4HCCompression supported: 1 2023-08-25T08:51:35.258 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: kZlibCompression supported: 1 2023-08-25T08:51:35.258 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: kSnappyCompression supported: 1 2023-08-25T08:51:35.258 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: kLZ4Compression supported: 1 2023-08-25T08:51:35.258 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: kBZip2Compression supported: 0 2023-08-25T08:51:35.259 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-25T08:51:35.259 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-25T08:51:35.259 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-08-25T08:51:35.259 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-25T08:51:35.259 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-25T08:51:35.259 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.merge_operator: 2023-08-25T08:51:35.259 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_filter: None 2023-08-25T08:51:35.260 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_filter_factory: None 2023-08-25T08:51:35.260 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.sst_partitioner_factory: None 2023-08-25T08:51:35.260 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-25T08:51:35.260 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-25T08:51:35.260 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ff93e49540) 2023-08-25T08:51:35.260 INFO:journalctl@ceph.mon.c.smithi181.stdout: cache_index_and_filter_blocks: 1 2023-08-25T08:51:35.260 INFO:journalctl@ceph.mon.c.smithi181.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-25T08:51:35.261 INFO:journalctl@ceph.mon.c.smithi181.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-25T08:51:35.261 INFO:journalctl@ceph.mon.c.smithi181.stdout: pin_top_level_index_and_filter: 1 2023-08-25T08:51:35.261 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_type: 0 2023-08-25T08:51:35.261 INFO:journalctl@ceph.mon.c.smithi181.stdout: data_block_index_type: 0 2023-08-25T08:51:35.261 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_shortening: 1 2023-08-25T08:51:35.261 INFO:journalctl@ceph.mon.c.smithi181.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-25T08:51:35.261 INFO:journalctl@ceph.mon.c.smithi181.stdout: checksum: 4 2023-08-25T08:51:35.262 INFO:journalctl@ceph.mon.c.smithi181.stdout: no_block_cache: 0 2023-08-25T08:51:35.262 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache: 0x55ff93ea11f0 2023-08-25T08:51:35.262 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_name: BinnedLRUCache 2023-08-25T08:51:35.262 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_options: 2023-08-25T08:51:35.262 INFO:journalctl@ceph.mon.c.smithi181.stdout: capacity : 536870912 2023-08-25T08:51:35.262 INFO:journalctl@ceph.mon.c.smithi181.stdout: num_shard_bits : 4 2023-08-25T08:51:35.262 INFO:journalctl@ceph.mon.c.smithi181.stdout: strict_capacity_limit : 0 2023-08-25T08:51:35.263 INFO:journalctl@ceph.mon.c.smithi181.stdout: high_pri_pool_ratio: 0.000 2023-08-25T08:51:35.263 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_compressed: (nil) 2023-08-25T08:51:35.263 INFO:journalctl@ceph.mon.c.smithi181.stdout: persistent_cache: (nil) 2023-08-25T08:51:35.263 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_size: 4096 2023-08-25T08:51:35.263 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_size_deviation: 10 2023-08-25T08:51:35.263 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_restart_interval: 16 2023-08-25T08:51:35.263 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_block_restart_interval: 1 2023-08-25T08:51:35.263 INFO:journalctl@ceph.mon.c.smithi181.stdout: metadata_block_size: 4096 2023-08-25T08:51:35.264 INFO:journalctl@ceph.mon.c.smithi181.stdout: partition_filters: 0 2023-08-25T08:51:35.264 INFO:journalctl@ceph.mon.c.smithi181.stdout: use_delta_encoding: 1 2023-08-25T08:51:35.264 INFO:journalctl@ceph.mon.c.smithi181.stdout: filter_policy: bloomfilter 2023-08-25T08:51:35.264 INFO:journalctl@ceph.mon.c.smithi181.stdout: whole_key_filtering: 1 2023-08-25T08:51:35.264 INFO:journalctl@ceph.mon.c.smithi181.stdout: verify_compression: 0 2023-08-25T08:51:35.264 INFO:journalctl@ceph.mon.c.smithi181.stdout: read_amp_bytes_per_bit: 0 2023-08-25T08:51:35.264 INFO:journalctl@ceph.mon.c.smithi181.stdout: format_version: 5 2023-08-25T08:51:35.264 INFO:journalctl@ceph.mon.c.smithi181.stdout: enable_index_compression: 1 2023-08-25T08:51:35.265 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_align: 0 2023-08-25T08:51:35.265 INFO:journalctl@ceph.mon.c.smithi181.stdout: max_auto_readahead_size: 262144 2023-08-25T08:51:35.265 INFO:journalctl@ceph.mon.c.smithi181.stdout: prepopulate_block_cache: 0 2023-08-25T08:51:35.265 INFO:journalctl@ceph.mon.c.smithi181.stdout: initial_auto_readahead_size: 8192 2023-08-25T08:51:35.265 INFO:journalctl@ceph.mon.c.smithi181.stdout: num_file_reads_for_auto_readahead: 2 2023-08-25T08:51:35.265 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.write_buffer_size: 33554432 2023-08-25T08:51:35.265 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_write_buffer_number: 2 2023-08-25T08:51:35.266 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression: NoCompression 2023-08-25T08:51:35.266 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression: Disabled 2023-08-25T08:51:35.266 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.prefix_extractor: nullptr 2023-08-25T08:51:35.266 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-25T08:51:35.266 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.num_levels: 7 2023-08-25T08:51:35.266 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-25T08:51:35.266 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-25T08:51:35.266 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-25T08:51:35.267 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-25T08:51:35.267 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-25T08:51:35.267 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-25T08:51:35.267 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-25T08:51:35.267 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-25T08:51:35.267 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-25T08:51:35.267 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-25T08:51:35.268 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-25T08:51:35.268 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-25T08:51:35.268 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-25T08:51:35.268 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.level: 32767 2023-08-25T08:51:35.268 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.strategy: 0 2023-08-25T08:51:35.268 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-25T08:51:35.268 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-25T08:51:35.268 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-25T08:51:35.269 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-25T08:51:35.269 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.enabled: false 2023-08-25T08:51:35.269 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-25T08:51:35.269 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-25T08:51:35.269 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-25T08:51:35.269 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-25T08:51:35.269 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.target_file_size_base: 67108864 2023-08-25T08:51:35.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-25T08:51:35.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-25T08:51:35.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-25T08:51:35.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-25T08:51:35.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-25T08:51:35.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-25T08:51:35.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-25T08:51:35.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-25T08:51:35.271 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-25T08:51:35.271 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-25T08:51:35.271 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-25T08:51:35.271 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-25T08:51:35.271 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-25T08:51:35.272 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-25T08:51:35.272 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.arena_block_size: 1048576 2023-08-25T08:51:35.272 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-25T08:51:35.272 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-25T08:51:35.272 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.disable_auto_compactions: 0 2023-08-25T08:51:35.272 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-25T08:51:35.272 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-25T08:51:35.272 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-25T08:51:35.273 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-25T08:51:35.273 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-25T08:51:35.273 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-25T08:51:35.273 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-25T08:51:35.273 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-25T08:51:35.273 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-25T08:51:35.274 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-25T08:51:35.274 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.table_properties_collectors: 2023-08-25T08:51:35.274 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.inplace_update_support: 0 2023-08-25T08:51:35.274 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-25T08:51:35.274 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-25T08:51:35.274 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-25T08:51:35.274 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-25T08:51:35.274 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.bloom_locality: 0 2023-08-25T08:51:35.275 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.max_successive_merges: 0 2023-08-25T08:51:35.275 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-25T08:51:35.275 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.paranoid_file_checks: 0 2023-08-25T08:51:35.275 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.force_consistency_checks: 1 2023-08-25T08:51:35.275 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.report_bg_io_stats: 0 2023-08-25T08:51:35.275 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.ttl: 2592000 2023-08-25T08:51:35.275 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-25T08:51:35.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-25T08:51:35.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-25T08:51:35.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.enable_blob_files: false 2023-08-25T08:51:35.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.min_blob_size: 0 2023-08-25T08:51:35.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.blob_file_size: 268435456 2023-08-25T08:51:35.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-25T08:51:35.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-25T08:51:35.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-25T08:51:35.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-25T08:51:35.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-25T08:51:35.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.blob_file_starting_level: 0 2023-08-25T08:51:35.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-25T08:51:35.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-08-25T08:51:35.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-08-25T08:51:35.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bf6d91f4-1789-4cb5-88b1-25cde7173eaf 2023-08-25T08:51:35.278 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953494883996, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-08-25T08:51:35.278 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-08-25T08:51:35.278 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953494884416, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1692953494, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bf6d91f4-1789-4cb5-88b1-25cde7173eaf", "db_session_id": "JMMEP1NVMDHGFBVK2DJQ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-08-25T08:51:35.278 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953494884488, "job": 1, "event": "recovery_finished"} 2023-08-25T08:51:35.278 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-08-25T08:51:35.278 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-25T08:51:35.278 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ff93f3e000 2023-08-25T08:51:35.278 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: DB pointer 0x55ff93f2a000 2023-08-25T08:51:35.279 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-25T08:51:35.279 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-25T08:51:35.279 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** DB Stats ** 2023-08-25T08:51:35.279 INFO:journalctl@ceph.mon.c.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T08:51:35.279 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-25T08:51:35.279 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T08:51:35.279 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T08:51:35.280 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-25T08:51:35.280 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T08:51:35.280 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T08:51:35.280 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2023-08-25T08:51:35.280 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** Compaction Stats [default] ** 2023-08-25T08:51:35.280 INFO:journalctl@ceph.mon.c.smithi181.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T08:51:35.280 INFO:journalctl@ceph.mon.c.smithi181.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-25T08:51:35.281 INFO:journalctl@ceph.mon.c.smithi181.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T08:51:35.281 INFO:journalctl@ceph.mon.c.smithi181.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T08:51:35.281 INFO:journalctl@ceph.mon.c.smithi181.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T08:51:35.281 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2023-08-25T08:51:35.281 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** Compaction Stats [default] ** 2023-08-25T08:51:35.281 INFO:journalctl@ceph.mon.c.smithi181.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T08:51:35.281 INFO:journalctl@ceph.mon.c.smithi181.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-25T08:51:35.281 INFO:journalctl@ceph.mon.c.smithi181.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T08:51:35.282 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2023-08-25T08:51:35.282 INFO:journalctl@ceph.mon.c.smithi181.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-25T08:51:35.282 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2023-08-25T08:51:35.282 INFO:journalctl@ceph.mon.c.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T08:51:35.282 INFO:journalctl@ceph.mon.c.smithi181.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-25T08:51:35.282 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-25T08:51:35.282 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-25T08:51:35.282 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-25T08:51:35.283 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-25T08:51:35.283 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T08:51:35.283 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T08:51:35.283 INFO:journalctl@ceph.mon.c.smithi181.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-25T08:51:35.283 INFO:journalctl@ceph.mon.c.smithi181.stdout: Block cache BinnedLRUCache@0x55ff93ea11f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-08-25T08:51:35.283 INFO:journalctl@ceph.mon.c.smithi181.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-08-25T08:51:35.283 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2023-08-25T08:51:35.284 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-25T08:51:35.284 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-08-25T08:51:35.284 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: using public_addr v2:172.21.15.181:0/0 -> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2023-08-25T08:51:35.284 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: starting mon.c rank -1 at public addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] at bind addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:35.284 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(???) e0 preinit fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:35.284 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).mds e1 new map 2023-08-25T08:51:35.284 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).mds e1 print_map 2023-08-25T08:51:35.284 INFO:journalctl@ceph.mon.c.smithi181.stdout: e1 2023-08-25T08:51:35.285 INFO:journalctl@ceph.mon.c.smithi181.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-25T08:51:35.285 INFO:journalctl@ceph.mon.c.smithi181.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T08:51:35.285 INFO:journalctl@ceph.mon.c.smithi181.stdout: legacy client fscid: -1 2023-08-25T08:51:35.285 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2023-08-25T08:51:35.285 INFO:journalctl@ceph.mon.c.smithi181.stdout: No filesystems configured 2023-08-25T08:51:35.285 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-08-25T08:51:35.285 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-08-25T08:51:35.286 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-08-25T08:51:35.286 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-08-25T08:51:35.286 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-08-25T08:51:35.286 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-08-25T08:51:35.286 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-25T08:51:35.286 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:51:35.286 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:51:35.286 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:51:35.287 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:35.287 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.287 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.287 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.287 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.287 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:51:35.287 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:35.287 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:35.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Updating smithi157:/etc/ceph/ceph.conf 2023-08-25T08:51:35.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:35.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:35.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:51:35.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:35.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:35.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Deploying cephadm binary to smithi181 2023-08-25T08:51:35.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:35.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Added host smithi181 2023-08-25T08:51:35.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:35.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.290 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.290 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:35.290 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.290 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:51:35.290 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:35.290 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.290 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.290 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.291 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.291 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:51:35.291 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:35.291 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:35.291 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Updating smithi181:/etc/ceph/ceph.conf 2023-08-25T08:51:35.291 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:35.291 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2581435798' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-25T08:51:35.291 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.292 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.292 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.292 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:35.292 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:51:35.292 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:35.292 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2581435798' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-25T08:51:35.292 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:35.293 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:35.293 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.293 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:35.293 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:35.293 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:35.293 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:35.293 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:35.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:35.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:35.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c;count:3 2023-08-25T08:51:35.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: Deploying daemon mon.c on smithi181 2023-08-25T08:51:35.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:35.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 systemd[1]: Started Ceph mon.c for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:51:35.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-08-25T08:51:35.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-08-25T08:51:35.295 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta expand map: {default=false} 2023-08-25T08:51:35.295 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta from 'false' to 'false' 2023-08-25T08:51:35.295 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta expanded map: {default=false} 2023-08-25T08:51:35.295 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta expand map: {default=info} 2023-08-25T08:51:35.295 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta from 'info' to 'info' 2023-08-25T08:51:35.295 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta expanded map: {default=info} 2023-08-25T08:51:35.295 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta expand map: {default=daemon} 2023-08-25T08:51:35.295 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-25T08:51:35.296 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta expanded map: {default=daemon} 2023-08-25T08:51:35.296 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta expand map: {default=debug} 2023-08-25T08:51:35.296 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta from 'debug' to 'debug' 2023-08-25T08:51:35.296 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:34 smithi181 ceph-mon[131296]: expand_channel_meta expanded map: {default=debug} 2023-08-25T08:51:35.595 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-25T08:51:35.596 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph mon dump -f json 2023-08-25T08:51:36.154 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:51:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.258+0000 7ff30788e700 1 -- 172.21.15.181:0/2071013884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 msgr2=0x7ff2ec005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.258+0000 7ff30788e700 1 --2- 172.21.15.181:0/2071013884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 0x7ff2ec005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff300076310 tx=0x7ff2f000feb0 comp rx=0 tx=0).stop 2023-08-25T08:51:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.258+0000 7ff30788e700 1 -- 172.21.15.181:0/2071013884 shutdown_connections 2023-08-25T08:51:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.258+0000 7ff30788e700 1 --2- 172.21.15.181:0/2071013884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 0x7ff2ec005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.258+0000 7ff30788e700 1 -- 172.21.15.181:0/2071013884 >> 172.21.15.181:0/2071013884 conn(0x7ff300074ed0 msgr2=0x7ff3000752e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.258+0000 7ff30788e700 1 -- 172.21.15.181:0/2071013884 shutdown_connections 2023-08-25T08:51:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.258+0000 7ff30788e700 1 -- 172.21.15.181:0/2071013884 wait complete. 2023-08-25T08:51:40.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.259+0000 7ff30788e700 1 Processor -- start 2023-08-25T08:51:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.259+0000 7ff30788e700 1 -- start start 2023-08-25T08:51:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.259+0000 7ff30788e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 0x7ff30019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.259+0000 7ff30788e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 0x7ff300195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.259+0000 7ff30788e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30019c250 con 0x7ff2ec0035c0 2023-08-25T08:51:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.259+0000 7ff30788e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30019c3c0 con 0x7ff30019bd10 2023-08-25T08:51:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff304e29700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 0x7ff300195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff304e29700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 0x7ff300195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43042/0 (socket says 172.21.15.181:43042) 2023-08-25T08:51:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff304e29700 1 -- 172.21.15.181:0/1671920455 learned_addr learned my addr 172.21.15.181:0/1671920455 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:51:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff30562a700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 0x7ff30019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff304e29700 1 -- 172.21.15.181:0/1671920455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 msgr2=0x7ff300195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-08-25T08:51:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff304e29700 1 -- 172.21.15.181:0/1671920455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 msgr2=0x7ff300195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-25T08:51:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff304e29700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 0x7ff300195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T08:51:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff304e29700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 0x7ff300195850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:51:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff30562a700 1 -- 172.21.15.181:0/1671920455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 msgr2=0x7ff300195850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff30562a700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 0x7ff300195850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.260+0000 7ff30562a700 1 -- 172.21.15.181:0/1671920455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f000f900 con 0x7ff2ec0035c0 2023-08-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.261+0000 7ff30562a700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 0x7ff30019b7d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0000c00 tx=0x7ff2f00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.261+0000 7ff2f67fc700 1 -- 172.21.15.181:0/1671920455 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ff2f0013b00 con 0x7ff2ec0035c0 2023-08-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.261+0000 7ff2f67fc700 1 -- 172.21.15.181:0/1671920455 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2f00044f0 con 0x7ff2ec0035c0 2023-08-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.261+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff300195e50 con 0x7ff2ec0035c0 2023-08-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.261+0000 7ff2f67fc700 1 -- 172.21.15.181:0/1671920455 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ff2f0003d30 con 0x7ff2ec0035c0 2023-08-25T08:51:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.261+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3001963a0 con 0x7ff2ec0035c0 2023-08-25T08:51:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.262+0000 7ff2f67fc700 1 -- 172.21.15.181:0/1671920455 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7ff2f0011470 con 0x7ff2ec0035c0 2023-08-25T08:51:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.262+0000 7ff2f67fc700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff2e4037ff0 0x7ff2e403a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.262+0000 7ff2f67fc700 1 -- 172.21.15.181:0/1671920455 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff2f004d550 con 0x7ff2ec0035c0 2023-08-25T08:51:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.262+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff300061f50 con 0x7ff2ec0035c0 2023-08-25T08:51:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.263+0000 7ff304e29700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff2e4037ff0 0x7ff2e403a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.263+0000 7ff304e29700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff2e4037ff0 0x7ff2e403a4b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009180 tx=0x7ff2fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.268+0000 7ff2f67fc700 1 -- 172.21.15.181:0/1671920455 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff2f00109c0 con 0x7ff2ec0035c0 2023-08-25T08:51:40.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:51:40.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: mon.a calling monitor election 2023-08-25T08:51:40.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:40.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: mon.c calling monitor election 2023-08-25T08:51:40.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:40.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T08:51:40.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: monmap e2: 2 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:40.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: fsmap 2023-08-25T08:51:40.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:40.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: mgrmap e13: x(active, since 45s) 2023-08-25T08:51:40.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: overall HEALTH_OK 2023-08-25T08:51:40.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:40.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:40.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:40.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:40.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:40.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:51:40.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: mon.a calling monitor election 2023-08-25T08:51:40.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:40.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: mon.c calling monitor election 2023-08-25T08:51:40.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:40.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:40.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T08:51:40.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: monmap e2: 2 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:40.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: fsmap 2023-08-25T08:51:40.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:40.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: mgrmap e13: x(active, since 45s) 2023-08-25T08:51:40.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: overall HEALTH_OK 2023-08-25T08:51:40.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:40.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:40.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:40.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:40.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:40.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.534+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff300107120 con 0x7ff2ec0035c0 2023-08-25T08:51:40.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.534+0000 7ff2f67fc700 1 -- 172.21.15.181:0/1671920455 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7ff2f00103d0 con 0x7ff2ec0035c0 2023-08-25T08:51:40.535 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:51:40.535 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","modified":"2023-08-25T08:51:34.913245Z","created":"2023-08-25T08:49:48.573313Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-08-25T08:51:40.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff2e4037ff0 msgr2=0x7ff2e403a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:40.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff2e4037ff0 0x7ff2e403a4b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009180 tx=0x7ff2fc008040 comp rx=0 tx=0).stop 2023-08-25T08:51:40.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 msgr2=0x7ff30019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:40.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 0x7ff30019b7d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0000c00 tx=0x7ff2f00047a0 comp rx=0 tx=0).stop 2023-08-25T08:51:40.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 shutdown_connections 2023-08-25T08:51:40.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff2e4037ff0 0x7ff2e403a4b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:40.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec0035c0 0x7ff30019b7d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:40.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 --2- 172.21.15.181:0/1671920455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30019bd10 0x7ff300195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:40.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.536+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 >> 172.21.15.181:0/1671920455 conn(0x7ff300074ed0 msgr2=0x7ff30018d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:40.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.537+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 shutdown_connections 2023-08-25T08:51:40.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:40.537+0000 7ff30788e700 1 -- 172.21.15.181:0/1671920455 wait complete. 2023-08-25T08:51:40.541 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 2 2023-08-25T08:51:41.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:41 smithi181 ceph-mon[131296]: Deploying daemon mon.b on smithi157 2023-08-25T08:51:41.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:41 smithi181 ceph-mon[131296]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:41.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:41 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/1671920455' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T08:51:41.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:41 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:41.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:41 smithi099 ceph-mon[97641]: Deploying daemon mon.b on smithi157 2023-08-25T08:51:41.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:41 smithi099 ceph-mon[97641]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:41.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:41 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/1671920455' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T08:51:41.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:41 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:41.918 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-25T08:51:41.919 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph mon dump -f json 2023-08-25T08:51:42.482 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:51:42.814 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:42 smithi157 systemd[1]: Starting Ceph mon.b for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:51:43.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:42 smithi157 podman[130840]: 2023-08-25T08:51:43.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:42 smithi157 ceph-mon[130879]: set uid:gid to 167:167 (ceph:ceph) 2023-08-25T08:51:43.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:42 smithi157 ceph-mon[130879]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev), process ceph-mon, pid 7 2023-08-25T08:51:43.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:42 smithi157 ceph-mon[130879]: pidfile_write: ignore empty --pid-file 2023-08-25T08:51:43.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 bash[130840]: c9f6ce89376f952ae7f4db68c5abd402c965d94ef89df37c26711d6bbed45bae 2023-08-25T08:51:43.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: load: jerasure load: lrc 2023-08-25T08:51:43.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: RocksDB version: 7.9.2 2023-08-25T08:51:43.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Git sha 0 2023-08-25T08:51:43.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Compile date 2023-08-24 22:08:31 2023-08-25T08:51:43.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: DB SUMMARY 2023-08-25T08:51:43.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: DB Session ID: 1R5AY3DSOWRNXMKHXVZ6 2023-08-25T08:51:43.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: CURRENT file: CURRENT 2023-08-25T08:51:43.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: IDENTITY file: IDENTITY 2023-08-25T08:51:43.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-08-25T08:51:43.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-08-25T08:51:43.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-08-25T08:51:43.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.error_if_exists: 0 2023-08-25T08:51:43.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.create_if_missing: 0 2023-08-25T08:51:43.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.paranoid_checks: 1 2023-08-25T08:51:43.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-25T08:51:43.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-25T08:51:43.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-25T08:51:43.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.env: 0x557d3fc9dda0 2023-08-25T08:51:43.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.fs: PosixFileSystem 2023-08-25T08:51:43.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.info_log: 0x557d40761440 2023-08-25T08:51:43.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_file_opening_threads: 16 2023-08-25T08:51:43.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.statistics: (nil) 2023-08-25T08:51:43.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.use_fsync: 0 2023-08-25T08:51:43.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_log_file_size: 0 2023-08-25T08:51:43.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-25T08:51:43.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-25T08:51:43.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.keep_log_file_num: 1000 2023-08-25T08:51:43.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.recycle_log_file_num: 0 2023-08-25T08:51:43.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.allow_fallocate: 1 2023-08-25T08:51:43.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.allow_mmap_reads: 0 2023-08-25T08:51:43.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.allow_mmap_writes: 0 2023-08-25T08:51:43.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.use_direct_reads: 0 2023-08-25T08:51:43.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-25T08:51:43.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.create_missing_column_families: 0 2023-08-25T08:51:43.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.db_log_dir: 2023-08-25T08:51:43.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.wal_dir: 2023-08-25T08:51:43.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-25T08:51:43.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-25T08:51:43.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-25T08:51:43.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-25T08:51:43.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-25T08:51:43.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-25T08:51:43.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.advise_random_on_open: 1 2023-08-25T08:51:43.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.db_write_buffer_size: 0 2023-08-25T08:51:43.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.write_buffer_manager: 0x557d4071b4a0 2023-08-25T08:51:43.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-25T08:51:43.137 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-25T08:51:43.137 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-25T08:51:43.137 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.rate_limiter: (nil) 2023-08-25T08:51:43.137 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-25T08:51:43.137 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.wal_recovery_mode: 2 2023-08-25T08:51:43.137 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.enable_thread_tracking: 0 2023-08-25T08:51:43.138 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.enable_pipelined_write: 0 2023-08-25T08:51:43.138 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.unordered_write: 0 2023-08-25T08:51:43.138 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-25T08:51:43.138 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-25T08:51:43.138 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-25T08:51:43.139 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-25T08:51:43.139 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.row_cache: None 2023-08-25T08:51:43.139 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.wal_filter: None 2023-08-25T08:51:43.139 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-25T08:51:43.139 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.allow_ingest_behind: 0 2023-08-25T08:51:43.140 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.two_write_queues: 0 2023-08-25T08:51:43.140 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.manual_wal_flush: 0 2023-08-25T08:51:43.140 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.wal_compression: 0 2023-08-25T08:51:43.140 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.atomic_flush: 0 2023-08-25T08:51:43.140 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-25T08:51:43.140 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-25T08:51:43.140 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-25T08:51:43.140 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.log_readahead_size: 0 2023-08-25T08:51:43.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-25T08:51:43.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.best_efforts_recovery: 0 2023-08-25T08:51:43.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-25T08:51:43.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-25T08:51:43.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.allow_data_in_errors: 0 2023-08-25T08:51:43.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.db_host_id: __hostname__ 2023-08-25T08:51:43.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-25T08:51:43.142 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_background_jobs: 2 2023-08-25T08:51:43.142 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_background_compactions: -1 2023-08-25T08:51:43.142 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_subcompactions: 1 2023-08-25T08:51:43.142 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-25T08:51:43.142 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-25T08:51:43.142 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-25T08:51:43.142 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_total_wal_size: 0 2023-08-25T08:51:43.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-25T08:51:43.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-25T08:51:43.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-25T08:51:43.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-25T08:51:43.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_open_files: -1 2023-08-25T08:51:43.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bytes_per_sync: 0 2023-08-25T08:51:43.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-25T08:51:43.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-25T08:51:43.144 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_readahead_size: 0 2023-08-25T08:51:43.144 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_background_flushes: -1 2023-08-25T08:51:43.144 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Compression algorithms supported: 2023-08-25T08:51:43.144 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-25T08:51:43.144 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: kZSTD supported: 0 2023-08-25T08:51:43.144 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: kXpressCompression supported: 0 2023-08-25T08:51:43.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: kLZ4HCCompression supported: 1 2023-08-25T08:51:43.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: kZlibCompression supported: 1 2023-08-25T08:51:43.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: kSnappyCompression supported: 1 2023-08-25T08:51:43.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: kLZ4Compression supported: 1 2023-08-25T08:51:43.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: kBZip2Compression supported: 0 2023-08-25T08:51:43.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-25T08:51:43.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-25T08:51:43.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-08-25T08:51:43.146 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-25T08:51:43.146 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-25T08:51:43.146 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.merge_operator: 2023-08-25T08:51:43.146 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_filter: None 2023-08-25T08:51:43.146 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_filter_factory: None 2023-08-25T08:51:43.146 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.sst_partitioner_factory: None 2023-08-25T08:51:43.146 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-25T08:51:43.146 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-25T08:51:43.147 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557d40761540) 2023-08-25T08:51:43.147 INFO:journalctl@ceph.mon.b.smithi157.stdout: cache_index_and_filter_blocks: 1 2023-08-25T08:51:43.147 INFO:journalctl@ceph.mon.b.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-25T08:51:43.147 INFO:journalctl@ceph.mon.b.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-25T08:51:43.147 INFO:journalctl@ceph.mon.b.smithi157.stdout: pin_top_level_index_and_filter: 1 2023-08-25T08:51:43.147 INFO:journalctl@ceph.mon.b.smithi157.stdout: index_type: 0 2023-08-25T08:51:43.147 INFO:journalctl@ceph.mon.b.smithi157.stdout: data_block_index_type: 0 2023-08-25T08:51:43.148 INFO:journalctl@ceph.mon.b.smithi157.stdout: index_shortening: 1 2023-08-25T08:51:43.148 INFO:journalctl@ceph.mon.b.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-25T08:51:43.148 INFO:journalctl@ceph.mon.b.smithi157.stdout: checksum: 4 2023-08-25T08:51:43.148 INFO:journalctl@ceph.mon.b.smithi157.stdout: no_block_cache: 0 2023-08-25T08:51:43.148 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_cache: 0x557d407b91f0 2023-08-25T08:51:43.148 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_cache_name: BinnedLRUCache 2023-08-25T08:51:43.149 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_cache_options: 2023-08-25T08:51:43.149 INFO:journalctl@ceph.mon.b.smithi157.stdout: capacity : 536870912 2023-08-25T08:51:43.149 INFO:journalctl@ceph.mon.b.smithi157.stdout: num_shard_bits : 4 2023-08-25T08:51:43.149 INFO:journalctl@ceph.mon.b.smithi157.stdout: strict_capacity_limit : 0 2023-08-25T08:51:43.149 INFO:journalctl@ceph.mon.b.smithi157.stdout: high_pri_pool_ratio: 0.000 2023-08-25T08:51:43.149 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_cache_compressed: (nil) 2023-08-25T08:51:43.149 INFO:journalctl@ceph.mon.b.smithi157.stdout: persistent_cache: (nil) 2023-08-25T08:51:43.150 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_size: 4096 2023-08-25T08:51:43.150 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_size_deviation: 10 2023-08-25T08:51:43.150 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_restart_interval: 16 2023-08-25T08:51:43.150 INFO:journalctl@ceph.mon.b.smithi157.stdout: index_block_restart_interval: 1 2023-08-25T08:51:43.150 INFO:journalctl@ceph.mon.b.smithi157.stdout: metadata_block_size: 4096 2023-08-25T08:51:43.150 INFO:journalctl@ceph.mon.b.smithi157.stdout: partition_filters: 0 2023-08-25T08:51:43.150 INFO:journalctl@ceph.mon.b.smithi157.stdout: use_delta_encoding: 1 2023-08-25T08:51:43.150 INFO:journalctl@ceph.mon.b.smithi157.stdout: filter_policy: bloomfilter 2023-08-25T08:51:43.151 INFO:journalctl@ceph.mon.b.smithi157.stdout: whole_key_filtering: 1 2023-08-25T08:51:43.151 INFO:journalctl@ceph.mon.b.smithi157.stdout: verify_compression: 0 2023-08-25T08:51:43.151 INFO:journalctl@ceph.mon.b.smithi157.stdout: read_amp_bytes_per_bit: 0 2023-08-25T08:51:43.151 INFO:journalctl@ceph.mon.b.smithi157.stdout: format_version: 5 2023-08-25T08:51:43.151 INFO:journalctl@ceph.mon.b.smithi157.stdout: enable_index_compression: 1 2023-08-25T08:51:43.151 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_align: 0 2023-08-25T08:51:43.151 INFO:journalctl@ceph.mon.b.smithi157.stdout: max_auto_readahead_size: 262144 2023-08-25T08:51:43.151 INFO:journalctl@ceph.mon.b.smithi157.stdout: prepopulate_block_cache: 0 2023-08-25T08:51:43.152 INFO:journalctl@ceph.mon.b.smithi157.stdout: initial_auto_readahead_size: 8192 2023-08-25T08:51:43.152 INFO:journalctl@ceph.mon.b.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2023-08-25T08:51:43.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.write_buffer_size: 33554432 2023-08-25T08:51:43.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_write_buffer_number: 2 2023-08-25T08:51:43.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression: NoCompression 2023-08-25T08:51:43.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression: Disabled 2023-08-25T08:51:43.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.prefix_extractor: nullptr 2023-08-25T08:51:43.153 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-25T08:51:43.153 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.num_levels: 7 2023-08-25T08:51:43.153 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-25T08:51:43.153 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-25T08:51:43.153 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-25T08:51:43.153 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-25T08:51:43.153 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-25T08:51:43.154 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-25T08:51:43.155 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-25T08:51:43.155 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-25T08:51:43.155 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-25T08:51:43.155 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-25T08:51:43.155 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-25T08:51:43.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-25T08:51:43.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-25T08:51:43.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.level: 32767 2023-08-25T08:51:43.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.strategy: 0 2023-08-25T08:51:43.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-25T08:51:43.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-25T08:51:43.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-25T08:51:43.157 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-25T08:51:43.157 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.enabled: false 2023-08-25T08:51:43.157 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-25T08:51:43.157 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-25T08:51:43.157 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-25T08:51:43.157 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-25T08:51:43.157 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.target_file_size_base: 67108864 2023-08-25T08:51:43.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-25T08:51:43.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-25T08:51:43.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-25T08:51:43.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-25T08:51:43.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-25T08:51:43.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-25T08:51:43.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-25T08:51:43.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-25T08:51:43.159 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-25T08:51:43.159 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-25T08:51:43.159 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-25T08:51:43.159 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-25T08:51:43.159 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-25T08:51:43.159 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-25T08:51:43.159 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.arena_block_size: 1048576 2023-08-25T08:51:43.160 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-25T08:51:43.160 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-25T08:51:43.160 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.disable_auto_compactions: 0 2023-08-25T08:51:43.160 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-25T08:51:43.160 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-25T08:51:43.160 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-25T08:51:43.160 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-25T08:51:43.161 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-25T08:51:43.161 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-25T08:51:43.161 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-25T08:51:43.161 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-25T08:51:43.161 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-25T08:51:43.161 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-25T08:51:43.161 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.table_properties_collectors: 2023-08-25T08:51:43.161 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.inplace_update_support: 0 2023-08-25T08:51:43.162 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-25T08:51:43.162 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-25T08:51:43.162 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-25T08:51:43.162 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-25T08:51:43.162 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.bloom_locality: 0 2023-08-25T08:51:43.162 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.max_successive_merges: 0 2023-08-25T08:51:43.162 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-25T08:51:43.163 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.paranoid_file_checks: 0 2023-08-25T08:51:43.163 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.force_consistency_checks: 1 2023-08-25T08:51:43.163 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.report_bg_io_stats: 0 2023-08-25T08:51:43.163 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.ttl: 2592000 2023-08-25T08:51:43.163 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-25T08:51:43.163 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-25T08:51:43.163 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-25T08:51:43.163 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.enable_blob_files: false 2023-08-25T08:51:43.164 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.min_blob_size: 0 2023-08-25T08:51:43.164 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.blob_file_size: 268435456 2023-08-25T08:51:43.164 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-25T08:51:43.164 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-25T08:51:43.164 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-25T08:51:43.164 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-25T08:51:43.164 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-25T08:51:43.165 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.blob_file_starting_level: 0 2023-08-25T08:51:43.165 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-25T08:51:43.165 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-08-25T08:51:43.165 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-08-25T08:51:43.165 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9273c12b-d285-4d34-9236-46c90b112f29 2023-08-25T08:51:43.165 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953503004910, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-08-25T08:51:43.165 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-08-25T08:51:43.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953503005345, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1692953503, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9273c12b-d285-4d34-9236-46c90b112f29", "db_session_id": "1R5AY3DSOWRNXMKHXVZ6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-08-25T08:51:43.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692953503005419, "job": 1, "event": "recovery_finished"} 2023-08-25T08:51:43.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-08-25T08:51:43.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-25T08:51:43.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557d40856000 2023-08-25T08:51:43.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 systemd[1]: Started Ceph mon.b for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:51:43.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: DB pointer 0x557d40842000 2023-08-25T08:51:43.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-25T08:51:43.167 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-25T08:51:43.167 INFO:journalctl@ceph.mon.b.smithi157.stdout: ** DB Stats ** 2023-08-25T08:51:43.167 INFO:journalctl@ceph.mon.b.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T08:51:43.167 INFO:journalctl@ceph.mon.b.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-25T08:51:43.167 INFO:journalctl@ceph.mon.b.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T08:51:43.167 INFO:journalctl@ceph.mon.b.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T08:51:43.167 INFO:journalctl@ceph.mon.b.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-25T08:51:43.168 INFO:journalctl@ceph.mon.b.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T08:51:43.168 INFO:journalctl@ceph.mon.b.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T08:51:43.168 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2023-08-25T08:51:43.168 INFO:journalctl@ceph.mon.b.smithi157.stdout: ** Compaction Stats [default] ** 2023-08-25T08:51:43.168 INFO:journalctl@ceph.mon.b.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T08:51:43.168 INFO:journalctl@ceph.mon.b.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-25T08:51:43.168 INFO:journalctl@ceph.mon.b.smithi157.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T08:51:43.168 INFO:journalctl@ceph.mon.b.smithi157.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T08:51:43.169 INFO:journalctl@ceph.mon.b.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T08:51:43.169 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2023-08-25T08:51:43.169 INFO:journalctl@ceph.mon.b.smithi157.stdout: ** Compaction Stats [default] ** 2023-08-25T08:51:43.169 INFO:journalctl@ceph.mon.b.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T08:51:43.169 INFO:journalctl@ceph.mon.b.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-25T08:51:43.169 INFO:journalctl@ceph.mon.b.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T08:51:43.169 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2023-08-25T08:51:43.170 INFO:journalctl@ceph.mon.b.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-25T08:51:43.170 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2023-08-25T08:51:43.170 INFO:journalctl@ceph.mon.b.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T08:51:43.170 INFO:journalctl@ceph.mon.b.smithi157.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-25T08:51:43.170 INFO:journalctl@ceph.mon.b.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-25T08:51:43.170 INFO:journalctl@ceph.mon.b.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-25T08:51:43.170 INFO:journalctl@ceph.mon.b.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-25T08:51:43.170 INFO:journalctl@ceph.mon.b.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-25T08:51:43.171 INFO:journalctl@ceph.mon.b.smithi157.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T08:51:43.171 INFO:journalctl@ceph.mon.b.smithi157.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T08:51:43.171 INFO:journalctl@ceph.mon.b.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-25T08:51:43.171 INFO:journalctl@ceph.mon.b.smithi157.stdout: Block cache BinnedLRUCache@0x557d407b91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-08-25T08:51:43.171 INFO:journalctl@ceph.mon.b.smithi157.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-08-25T08:51:43.171 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2023-08-25T08:51:43.171 INFO:journalctl@ceph.mon.b.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-25T08:51:43.171 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-08-25T08:51:43.172 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: using public_addr v2:172.21.15.157:0/0 -> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2023-08-25T08:51:43.172 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: starting mon.b rank -1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:43.172 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(???) e0 preinit fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:43.172 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).mds e1 new map 2023-08-25T08:51:43.173 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).mds e1 print_map 2023-08-25T08:51:43.173 INFO:journalctl@ceph.mon.b.smithi157.stdout: e1 2023-08-25T08:51:43.173 INFO:journalctl@ceph.mon.b.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-25T08:51:43.173 INFO:journalctl@ceph.mon.b.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T08:51:43.173 INFO:journalctl@ceph.mon.b.smithi157.stdout: legacy client fscid: -1 2023-08-25T08:51:43.173 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2023-08-25T08:51:43.173 INFO:journalctl@ceph.mon.b.smithi157.stdout: No filesystems configured 2023-08-25T08:51:43.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-08-25T08:51:43.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-08-25T08:51:43.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-08-25T08:51:43.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-08-25T08:51:43.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-08-25T08:51:43.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-08-25T08:51:43.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-25T08:51:43.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:51:43.175 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:51:43.175 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T08:51:43.175 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.175 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.175 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.175 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.175 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.175 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:51:43.176 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:43.176 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:43.176 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Updating smithi157:/etc/ceph/ceph.conf 2023-08-25T08:51:43.176 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:43.176 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.176 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.176 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.177 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:43.177 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:51:43.177 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:43.177 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.177 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Deploying cephadm binary to smithi181 2023-08-25T08:51:43.177 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.177 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.177 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Added host smithi181 2023-08-25T08:51:43.178 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:43.178 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.178 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.178 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.178 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.178 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:51:43.178 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.178 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.179 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.179 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.179 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.179 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:51:43.179 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:43.179 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:43.179 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Updating smithi181:/etc/ceph/ceph.conf 2023-08-25T08:51:43.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:43.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2581435798' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-25T08:51:43.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:43.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:51:43.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.181 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2581435798' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-25T08:51:43.181 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:43.181 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.181 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.181 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:43.181 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:43.181 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:43.182 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.182 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:43.182 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:43.182 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:43.182 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c;count:3 2023-08-25T08:51:43.182 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Deploying daemon mon.c on smithi181 2023-08-25T08:51:43.182 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.182 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:51:43.183 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.a calling monitor election 2023-08-25T08:51:43.183 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:43.183 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:43.183 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.183 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:43.183 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.c calling monitor election 2023-08-25T08:51:43.183 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:43.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:43.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:43.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T08:51:43.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: monmap e2: 2 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:43.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: fsmap 2023-08-25T08:51:43.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:43.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mgrmap e13: x(active, since 45s) 2023-08-25T08:51:43.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: overall HEALTH_OK 2023-08-25T08:51:43.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:43.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:43.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:43.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: Deploying daemon mon.b on smithi157 2023-08-25T08:51:43.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:43.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/1671920455' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T08:51:43.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:43.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-08-25T08:51:43.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta expand map: {default=false} 2023-08-25T08:51:43.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta from 'false' to 'false' 2023-08-25T08:51:43.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta expanded map: {default=false} 2023-08-25T08:51:43.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta expand map: {default=info} 2023-08-25T08:51:43.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta from 'info' to 'info' 2023-08-25T08:51:43.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta expanded map: {default=info} 2023-08-25T08:51:43.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta expand map: {default=daemon} 2023-08-25T08:51:43.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-25T08:51:43.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta expanded map: {default=daemon} 2023-08-25T08:51:43.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta expand map: {default=debug} 2023-08-25T08:51:43.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta from 'debug' to 'debug' 2023-08-25T08:51:43.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: expand_channel_meta expanded map: {default=debug} 2023-08-25T08:51:43.188 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:43 smithi157 ceph-mon[130879]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-08-25T08:51:48.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.111+0000 7f907ce45700 1 -- 172.21.15.181:0/4172132840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078104be0 msgr2=0x7f9060007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.111+0000 7f907ce45700 1 --2- 172.21.15.181:0/4172132840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078104be0 0x7f9060007820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f906c005fa0 tx=0x7f906c010f80 comp rx=0 tx=0).stop 2023-08-25T08:51:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.111+0000 7f907ce45700 1 -- 172.21.15.181:0/4172132840 shutdown_connections 2023-08-25T08:51:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.111+0000 7f907ce45700 1 --2- 172.21.15.181:0/4172132840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078104be0 0x7f9060007820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:48.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.112+0000 7f907ce45700 1 -- 172.21.15.181:0/4172132840 >> 172.21.15.181:0/4172132840 conn(0x7f9078074670 msgr2=0x7f9078074a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:48.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.112+0000 7f907ce45700 1 -- 172.21.15.181:0/4172132840 shutdown_connections 2023-08-25T08:51:48.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.112+0000 7f907ce45700 1 -- 172.21.15.181:0/4172132840 wait complete. 2023-08-25T08:51:48.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907ce45700 1 Processor -- start 2023-08-25T08:51:48.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907ce45700 1 -- start start 2023-08-25T08:51:48.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907ce45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 0x7f9078199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:48.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907ce45700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078199b80 0x7f90781936c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:48.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907ce45700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 0x7f9078194080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:48.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907ce45700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907819a1f0 con 0x7f9078199b80 2023-08-25T08:51:48.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907ce45700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781a0460 con 0x7f9078193c00 2023-08-25T08:51:48.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907ce45700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781a05d0 con 0x7f9078104be0 2023-08-25T08:51:48.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 0x7f9078199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:48.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 0x7f9078199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43068/0 (socket says 172.21.15.181:43068) 2023-08-25T08:51:48.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.113+0000 7f907659c700 1 -- 172.21.15.181:0/2774289588 learned_addr learned my addr 172.21.15.181:0/2774289588 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:51:48.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9075d9b700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078199b80 0x7f90781936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:48.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f907659c700 1 -- 172.21.15.181:0/2774289588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 msgr2=0x7f9078199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-08-25T08:51:48.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f907659c700 1 -- 172.21.15.181:0/2774289588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 msgr2=0x7f9078199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-25T08:51:48.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f907659c700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 0x7f9078199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T08:51:48.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9076d9d700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 0x7f9078194080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:48.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f907659c700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 0x7f9078199640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:51:48.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9076d9d700 1 -- 172.21.15.181:0/2774289588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 msgr2=0x7f9078194080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-08-25T08:51:48.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9076d9d700 1 -- 172.21.15.181:0/2774289588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 msgr2=0x7f9078194080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-25T08:51:48.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9076d9d700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 0x7f9078194080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T08:51:48.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9076d9d700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 0x7f9078194080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:51:48.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9075d9b700 1 -- 172.21.15.181:0/2774289588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 msgr2=0x7f9078199640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:51:48.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9075d9b700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 0x7f9078199640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:48.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9075d9b700 1 -- 172.21.15.181:0/2774289588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 msgr2=0x7f9078194080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:51:48.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9075d9b700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 0x7f9078194080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:48.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9075d9b700 1 -- 172.21.15.181:0/2774289588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c010980 con 0x7f9078199b80 2023-08-25T08:51:48.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f9075d9b700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078199b80 0x7f90781936c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f906800b550 tx=0x7f906800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:48.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f90677fe700 1 -- 172.21.15.181:0/2774289588 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068004230 con 0x7f9078199b80 2023-08-25T08:51:48.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.114+0000 7f90677fe700 1 -- 172.21.15.181:0/2774289588 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9068004390 con 0x7f9078199b80 2023-08-25T08:51:48.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.115+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90781a08b0 con 0x7f9078199b80 2023-08-25T08:51:48.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.115+0000 7f90677fe700 1 -- 172.21.15.181:0/2774289588 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068011af0 con 0x7f9078199b80 2023-08-25T08:51:48.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.115+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90781a0e00 con 0x7f9078199b80 2023-08-25T08:51:48.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.116+0000 7f90677fe700 1 -- 172.21.15.181:0/2774289588 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f9068011c50 con 0x7f9078199b80 2023-08-25T08:51:48.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.116+0000 7f90677fe700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9058038680 0x7f905803ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:48.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.116+0000 7f90677fe700 1 -- 172.21.15.181:0/2774289588 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f906804b470 con 0x7f9078199b80 2023-08-25T08:51:48.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.116+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9078061f50 con 0x7f9078199b80 2023-08-25T08:51:48.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.116+0000 7f907659c700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9058038680 0x7f905803ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:48.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.117+0000 7f907659c700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9058038680 0x7f905803ab40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f906c005850 tx=0x7f906c0103d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:48.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.121+0000 7f90677fe700 1 -- 172.21.15.181:0/2774289588 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f906801a400 con 0x7f9078199b80 2023-08-25T08:51:48.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.397+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9078106010 con 0x7f9078199b80 2023-08-25T08:51:48.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.398+0000 7f90677fe700 1 -- 172.21.15.181:0/2774289588 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7f9068048030 con 0x7f9078199b80 2023-08-25T08:51:48.399 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-08-25T08:51:48.399 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","modified":"2023-08-25T08:51:43.028189Z","created":"2023-08-25T08:49:48.573313Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-08-25T08:51:48.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9058038680 msgr2=0x7f905803ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:48.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9058038680 0x7f905803ab40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f906c005850 tx=0x7f906c0103d0 comp rx=0 tx=0).stop 2023-08-25T08:51:48.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078199b80 msgr2=0x7f90781936c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:48.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078199b80 0x7f90781936c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f906800b550 tx=0x7f906800b910 comp rx=0 tx=0).stop 2023-08-25T08:51:48.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 shutdown_connections 2023-08-25T08:51:48.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9058038680 0x7f905803ab40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:48.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9078193c00 0x7f9078194080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:48.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9078104be0 0x7f9078199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:48.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 --2- 172.21.15.181:0/2774289588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9078199b80 0x7f90781936c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:48.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 >> 172.21.15.181:0/2774289588 conn(0x7f9078074670 msgr2=0x7f9078100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:48.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 shutdown_connections 2023-08-25T08:51:48.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:48.400+0000 7f907ce45700 1 -- 172.21.15.181:0/2774289588 wait complete. 2023-08-25T08:51:48.403 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 3 2023-08-25T08:51:48.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:51:48.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: mon.a calling monitor election 2023-08-25T08:51:48.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: mon.c calling monitor election 2023-08-25T08:51:48.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:48.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:48.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:48.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T08:51:48.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:48.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: fsmap 2023-08-25T08:51:48.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:48.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: mgrmap e13: x(active, since 53s) 2023-08-25T08:51:48.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-25T08:51:48.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-25T08:51:48.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-25T08:51:48.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: mon.b (rank 2) addr [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] is down (out of quorum) 2023-08-25T08:51:48.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:48.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:48.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:48.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:48.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:51:48.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: mon.a calling monitor election 2023-08-25T08:51:48.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: mon.c calling monitor election 2023-08-25T08:51:48.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:48.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:48.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:48.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:48.566 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T08:51:48.566 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:48.566 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: fsmap 2023-08-25T08:51:48.566 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:48.567 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: mgrmap e13: x(active, since 53s) 2023-08-25T08:51:48.567 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-25T08:51:48.567 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-25T08:51:48.567 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-25T08:51:48.568 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: mon.b (rank 2) addr [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] is down (out of quorum) 2023-08-25T08:51:48.568 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:48.568 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:48.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:48.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:48.733 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-08-25T08:51:48.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph config generate-minimal-conf 2023-08-25T08:51:49.459 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:49.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:49 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:49.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:49 smithi181 ceph-mon[131296]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:49.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:49 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/2774289588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T08:51:49.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:49 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:49.561 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:49 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:49 smithi099 ceph-mon[97641]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:49 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/2774289588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T08:51:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:49 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:50.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:51:50.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:50.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.a calling monitor election 2023-08-25T08:51:50.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.c calling monitor election 2023-08-25T08:51:50.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:51:50.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:50.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:50.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:50.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:50.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:50.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:50.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:50.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T08:51:50.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:50.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: fsmap 2023-08-25T08:51:50.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:50.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mgrmap e13: x(active, since 53s) 2023-08-25T08:51:50.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-25T08:51:50.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-25T08:51:50.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-25T08:51:50.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.b (rank 2) addr [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] is down (out of quorum) 2023-08-25T08:51:50.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:50.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:50.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:50.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:50.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:50.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:50.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/2774289588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T08:51:50.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.002+0000 7f35c8923700 1 -- 172.21.15.99:0/2634632374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 msgr2=0x7f35c40730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:51.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.002+0000 7f35c8923700 1 --2- 172.21.15.99:0/2634632374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 0x7f35c40730d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009910 tx=0x7f35b4009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:51.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.002+0000 7f35c8923700 1 -- 172.21.15.99:0/2634632374 shutdown_connections 2023-08-25T08:51:51.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.002+0000 7f35c8923700 1 --2- 172.21.15.99:0/2634632374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 0x7f35c40730d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:51.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.002+0000 7f35c8923700 1 -- 172.21.15.99:0/2634632374 >> 172.21.15.99:0/2634632374 conn(0x7f35c40fbee0 msgr2=0x7f35c40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:51.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.002+0000 7f35c8923700 1 -- 172.21.15.99:0/2634632374 shutdown_connections 2023-08-25T08:51:51.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.002+0000 7f35c8923700 1 -- 172.21.15.99:0/2634632374 wait complete. 2023-08-25T08:51:51.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.003+0000 7f35c8923700 1 Processor -- start 2023-08-25T08:51:51.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.003+0000 7f35c8923700 1 -- start start 2023-08-25T08:51:51.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c8923700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 0x7f35c419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:51.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c8923700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35c419dff0 0x7f35c4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c8923700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4198070 0x7f35c4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c8923700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c419e650 con 0x7f35c4072cf0 2023-08-25T08:51:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c8923700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c41a48d0 con 0x7f35c4198070 2023-08-25T08:51:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c8923700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c41a4a60 con 0x7f35c419dff0 2023-08-25T08:51:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 0x7f35c419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 0x7f35c419dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42564/0 (socket says 172.21.15.99:42564) 2023-08-25T08:51:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c259c700 1 -- 172.21.15.99:0/2718031533 learned_addr learned my addr 172.21.15.99:0/2718031533 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:51:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c1d9b700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35c419dff0 0x7f35c4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:51.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c2d9d700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4198070 0x7f35c4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:51.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c259c700 1 -- 172.21.15.99:0/2718031533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35c419dff0 msgr2=0x7f35c4197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:51.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c259c700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35c419dff0 0x7f35c4197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:51.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c259c700 1 -- 172.21.15.99:0/2718031533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4198070 msgr2=0x7f35c4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:51.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c259c700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4198070 0x7f35c4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:51.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c259c700 1 -- 172.21.15.99:0/2718031533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b40095f0 con 0x7f35c4072cf0 2023-08-25T08:51:51.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.004+0000 7f35c2d9d700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4198070 0x7f35c4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:51:51.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.005+0000 7f35c259c700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 0x7f35c419dab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f35b4004cd0 tx=0x7f35b4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:51.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.005+0000 7f35b37fe700 1 -- 172.21.15.99:0/2718031533 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b401d070 con 0x7f35c4072cf0 2023-08-25T08:51:51.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.005+0000 7f35b37fe700 1 -- 172.21.15.99:0/2718031533 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35b400be20 con 0x7f35c4072cf0 2023-08-25T08:51:51.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.005+0000 7f35b37fe700 1 -- 172.21.15.99:0/2718031533 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4022dc0 con 0x7f35c4072cf0 2023-08-25T08:51:51.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.005+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c41a4ce0 con 0x7f35c4072cf0 2023-08-25T08:51:51.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.005+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c41a5130 con 0x7f35c4072cf0 2023-08-25T08:51:51.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.006+0000 7f35b37fe700 1 -- 172.21.15.99:0/2718031533 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f35b4022470 con 0x7f35c4072cf0 2023-08-25T08:51:51.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.006+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c410a0b0 con 0x7f35c4072cf0 2023-08-25T08:51:51.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.006+0000 7f35b37fe700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f35ac038680 0x7f35ac03ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:51.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.006+0000 7f35b37fe700 1 -- 172.21.15.99:0/2718031533 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f35b404c1a0 con 0x7f35c4072cf0 2023-08-25T08:51:51.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.007+0000 7f35c1d9b700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f35ac038680 0x7f35ac03ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:51.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.007+0000 7f35c1d9b700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f35ac038680 0x7f35ac03ab40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009ad0 tx=0x7f35b8009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:51.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.012+0000 7f35b37fe700 1 -- 172.21.15.99:0/2718031533 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f35b400edb0 con 0x7f35c4072cf0 2023-08-25T08:51:51.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: mon.b calling monitor election 2023-08-25T08:51:51.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: mon.b calling monitor election 2023-08-25T08:51:51.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: mon.c calling monitor election 2023-08-25T08:51:51.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: mon.a calling monitor election 2023-08-25T08:51:51.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-25T08:51:51.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:51.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: fsmap 2023-08-25T08:51:51.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:51.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: mgrmap e13: x(active, since 55s) 2023-08-25T08:51:51.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-25T08:51:51.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: Cluster is now healthy 2023-08-25T08:51:51.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: overall HEALTH_OK 2023-08-25T08:51:51.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:51.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: mon.b calling monitor election 2023-08-25T08:51:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: mon.b calling monitor election 2023-08-25T08:51:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: mon.c calling monitor election 2023-08-25T08:51:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: mon.a calling monitor election 2023-08-25T08:51:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-25T08:51:51.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:51.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: fsmap 2023-08-25T08:51:51.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:51.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: mgrmap e13: x(active, since 55s) 2023-08-25T08:51:51.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-25T08:51:51.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: Cluster is now healthy 2023-08-25T08:51:51.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: overall HEALTH_OK 2023-08-25T08:51:51.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:51.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.243+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f35c410c910 con 0x7f35c4072cf0 2023-08-25T08:51:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.244+0000 7f35b37fe700 1 -- 172.21.15.99:0/2718031533 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f35b4027020 con 0x7f35c4072cf0 2023-08-25T08:51:51.247 INFO:teuthology.orchestra.run.smithi099.stdout:# minimal ceph.conf for 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:51.248 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2023-08-25T08:51:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:51:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: mon_host = [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2023-08-25T08:51:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.246+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f35ac038680 msgr2=0x7f35ac03ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.246+0000 7f35c8923700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f35ac038680 0x7f35ac03ab40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009ad0 tx=0x7f35b8009210 comp rx=0 tx=0).stop 2023-08-25T08:51:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.246+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 msgr2=0x7f35c419dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.246+0000 7f35c8923700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 0x7f35c419dab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f35b4004cd0 tx=0x7f35b4004d00 comp rx=0 tx=0).stop 2023-08-25T08:51:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.247+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 shutdown_connections 2023-08-25T08:51:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.247+0000 7f35c8923700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f35ac038680 0x7f35ac03ab40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.247+0000 7f35c8923700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c4198070 0x7f35c4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.247+0000 7f35c8923700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c4072cf0 0x7f35c419dab0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.247+0000 7f35c8923700 1 --2- 172.21.15.99:0/2718031533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35c419dff0 0x7f35c4197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.247+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 >> 172.21.15.99:0/2718031533 conn(0x7f35c40fbee0 msgr2=0x7f35c41071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.247+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 shutdown_connections 2023-08-25T08:51:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:51:51.247+0000 7f35c8923700 1 -- 172.21.15.99:0/2718031533 wait complete. 2023-08-25T08:51:51.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.b calling monitor election 2023-08-25T08:51:51.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.b calling monitor election 2023-08-25T08:51:51.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.c calling monitor election 2023-08-25T08:51:51.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.a calling monitor election 2023-08-25T08:51:51.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-25T08:51:51.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-08-25T08:51:51.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: fsmap 2023-08-25T08:51:51.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T08:51:51.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: mgrmap e13: x(active, since 55s) 2023-08-25T08:51:51.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-25T08:51:51.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: Cluster is now healthy 2023-08-25T08:51:51.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: overall HEALTH_OK 2023-08-25T08:51:51.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.384 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.384 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.384 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:51:51.384 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:51.385 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.385 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.385 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:51.747 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-08-25T08:51:51.748 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:51:51.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2023-08-25T08:51:51.823 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:51:51.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:51.934 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:51:51.934 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2023-08-25T08:51:51.971 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:51:51.972 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:52.041 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:51:52.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2023-08-25T08:51:52.078 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:51:52.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:51:52.149 INFO:tasks.cephadm:Adding mgr.x on smithi099 2023-08-25T08:51:52.150 INFO:tasks.cephadm:Adding mgr.y on smithi157 2023-08-25T08:51:52.150 INFO:tasks.cephadm:Adding mgr.z on smithi181 2023-08-25T08:51:52.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch apply mgr '3;smithi099=x;smithi157=y;smithi181=z' 2023-08-25T08:51:52.302 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.302 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:52.302 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.302 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.302 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.303 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: Reconfiguring mon.a (unknown last config time)... 2023-08-25T08:51:52.303 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:52.303 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:52.303 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:52.303 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: Reconfiguring daemon mon.a on smithi099 2023-08-25T08:51:52.303 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:52 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2718031533' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:52.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: Reconfiguring mon.a (unknown last config time)... 2023-08-25T08:51:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:52.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:52.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:52.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: Reconfiguring daemon mon.a on smithi099 2023-08-25T08:51:52.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:52 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2718031533' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:52.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:51:52.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:52.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: Reconfiguring mon.a (unknown last config time)... 2023-08-25T08:51:52.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:52.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:52.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:52.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: Reconfiguring daemon mon.a on smithi099 2023-08-25T08:51:52.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:52 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2718031533' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:52.745 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:51:53.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.941+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/4023936491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8100fc0 msgr2=0x7f7fc81033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:53.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.941+0000 7f7fcf4d9700 1 --2- 172.21.15.181:0/4023936491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8100fc0 0x7f7fc81033b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009910 tx=0x7f7fc4009c20 comp rx=0 tx=0).stop 2023-08-25T08:51:53.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.941+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/4023936491 shutdown_connections 2023-08-25T08:51:53.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.941+0000 7f7fcf4d9700 1 --2- 172.21.15.181:0/4023936491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8100fc0 0x7f7fc81033b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:53.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.941+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/4023936491 >> 172.21.15.181:0/4023936491 conn(0x7f7fc80fa9f0 msgr2=0x7f7fc80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:53.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.941+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/4023936491 shutdown_connections 2023-08-25T08:51:53.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.941+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/4023936491 wait complete. 2023-08-25T08:51:53.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcf4d9700 1 Processor -- start 2023-08-25T08:51:53.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcf4d9700 1 -- start start 2023-08-25T08:51:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcf4d9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8100fc0 0x7f7fc8195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcf4d9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8195c50 0x7f7fc819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcf4d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc81960d0 0x7f7fc819d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcf4d9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc819d9f0 con 0x7f7fc8195c50 2023-08-25T08:51:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcf4d9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc819db60 con 0x7f7fc8100fc0 2023-08-25T08:51:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcf4d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc819dd20 con 0x7f7fc81960d0 2023-08-25T08:51:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcda76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc81960d0 0x7f7fc819d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcda76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc81960d0 0x7f7fc819d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43082/0 (socket says 172.21.15.181:43082) 2023-08-25T08:51:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcda76700 1 -- 172.21.15.181:0/1090042216 learned_addr learned my addr 172.21.15.181:0/1090042216 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:51:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcd275700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8100fc0 0x7f7fc8195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcca74700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8195c50 0x7f7fc819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcda76700 1 -- 172.21.15.181:0/1090042216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8100fc0 msgr2=0x7f7fc8195710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcda76700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8100fc0 0x7f7fc8195710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcda76700 1 -- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8195c50 msgr2=0x7f7fc819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcda76700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8195c50 0x7f7fc819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcda76700 1 -- 172.21.15.181:0/1090042216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40095f0 con 0x7f7fc81960d0 2023-08-25T08:51:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.942+0000 7f7fcd275700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8100fc0 0x7f7fc8195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:51:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.943+0000 7f7fcca74700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8195c50 0x7f7fc819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:51:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.943+0000 7f7fcda76700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc81960d0 0x7f7fc819d250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000ba50 tx=0x7f7fc000be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.943+0000 7f7fbe7fc700 1 -- 172.21.15.181:0/1090042216 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000cab0 con 0x7f7fc81960d0 2023-08-25T08:51:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.943+0000 7f7fbe7fc700 1 -- 172.21.15.181:0/1090042216 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc000cc10 con 0x7f7fc81960d0 2023-08-25T08:51:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.943+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc819dfa0 con 0x7f7fc81960d0 2023-08-25T08:51:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.943+0000 7f7fbe7fc700 1 -- 172.21.15.181:0/1090042216 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00075b0 con 0x7f7fc81960d0 2023-08-25T08:51:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.943+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc819e4f0 con 0x7f7fc81960d0 2023-08-25T08:51:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.944+0000 7f7fbe7fc700 1 -- 172.21.15.181:0/1090042216 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f7fc0007710 con 0x7f7fc81960d0 2023-08-25T08:51:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.944+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc818f040 con 0x7f7fc81960d0 2023-08-25T08:51:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.944+0000 7f7fbe7fc700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7fb4038680 0x7f7fb403ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:51:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.944+0000 7f7fbe7fc700 1 -- 172.21.15.181:0/1090042216 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7fc004a3c0 con 0x7f7fc81960d0 2023-08-25T08:51:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.945+0000 7f7fcd275700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7fb4038680 0x7f7fb403ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:51:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.945+0000 7f7fcd275700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7fb4038680 0x7f7fb403ab40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005130 tx=0x7f7fc40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:51:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:53.949+0000 7f7fbe7fc700 1 -- 172.21.15.181:0/1090042216 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7fc001b8b0 con 0x7f7fc81960d0 2023-08-25T08:51:53.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:53 smithi181 ceph-mon[131296]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:53 smithi181 ceph-mon[131296]: Reconfiguring mon.b (monmap changed)... 2023-08-25T08:51:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:53 smithi181 ceph-mon[131296]: Reconfiguring daemon mon.b on smithi157 2023-08-25T08:51:54.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:53 smithi099 ceph-mon[97641]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:54.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:53 smithi099 ceph-mon[97641]: Reconfiguring mon.b (monmap changed)... 2023-08-25T08:51:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:53 smithi099 ceph-mon[97641]: Reconfiguring daemon mon.b on smithi157 2023-08-25T08:51:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:53 smithi157 ceph-mon[130879]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:53 smithi157 ceph-mon[130879]: Reconfiguring mon.b (monmap changed)... 2023-08-25T08:51:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:54.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:54.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:53 smithi157 ceph-mon[130879]: Reconfiguring daemon mon.b on smithi157 2023-08-25T08:51:54.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.160+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi157=y;smithi181=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fc802ceb0 con 0x7f7fb4038680 2023-08-25T08:51:54.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.167+0000 7f7fbe7fc700 1 -- 172.21.15.181:0/1090042216 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7fc802ceb0 con 0x7f7fb4038680 2023-08-25T08:51:54.167 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mgr update... 2023-08-25T08:51:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7fb4038680 msgr2=0x7f7fb403ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7fb4038680 0x7f7fb403ab40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005130 tx=0x7f7fc40058e0 comp rx=0 tx=0).stop 2023-08-25T08:51:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc81960d0 msgr2=0x7f7fc819d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:51:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc81960d0 0x7f7fc819d250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000ba50 tx=0x7f7fc000be10 comp rx=0 tx=0).stop 2023-08-25T08:51:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 shutdown_connections 2023-08-25T08:51:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc81960d0 0x7f7fc819d250 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7fb4038680 0x7f7fb403ab40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8100fc0 0x7f7fc8195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 --2- 172.21.15.181:0/1090042216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fc8195c50 0x7f7fc819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:51:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 >> 172.21.15.181:0/1090042216 conn(0x7f7fc80fa9f0 msgr2=0x7f7fc80fb660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:51:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 shutdown_connections 2023-08-25T08:51:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:51:54.169+0000 7f7fcf4d9700 1 -- 172.21.15.181:0/1090042216 wait complete. 2023-08-25T08:51:54.822 DEBUG:teuthology.orchestra.run.smithi157:mgr.y> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.y.service 2023-08-25T08:51:54.826 DEBUG:teuthology.orchestra.run.smithi181:mgr.z> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.z.service 2023-08-25T08:51:54.828 INFO:tasks.cephadm:Deploying OSDs... 2023-08-25T08:51:54.828 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:51:54.829 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T08:51:54.857 INFO:journalctl@ceph.mgr.y.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:51:54.858 INFO:journalctl@ceph.mgr.z.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:51:54.877 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T08:51:54.877 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2023-08-25T08:51:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T08:51:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:54.965 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-08-25T08:51:54.966 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:54.966 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:54.966 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-08-25 08:51:05.610933360 +0000 2023-08-25T08:51:54.966 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-08-25 08:45:20.491551761 +0000 2023-08-25T08:51:54.967 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-08-25 08:45:20.491551761 +0000 2023-08-25T08:51:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-08-25 08:45:20.460552480 +0000 2023-08-25T08:51:54.967 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T08:51:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-08-25T08:51:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-08-25T08:51:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000273954 s, 1.9 MB/s 2023-08-25T08:51:55.075 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T08:51:55.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: Reconfiguring mon.c (monmap changed)... 2023-08-25T08:51:55.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:55.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:55.077 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:55.077 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: Reconfiguring daemon mon.c on smithi181 2023-08-25T08:51:55.077 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi157=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:55.077 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: Saving service mgr spec with placement smithi099=x;smithi157=y;smithi181=z;count:3 2023-08-25T08:51:55.077 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.077 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:54 smithi181 ceph-mon[131296]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:55.123 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2023-08-25T08:51:55.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: Reconfiguring mon.c (monmap changed)... 2023-08-25T08:51:55.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:55.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: Reconfiguring daemon mon.c on smithi181 2023-08-25T08:51:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi157=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: Saving service mgr spec with placement smithi099=x;smithi157=y;smithi181=z;count:3 2023-08-25T08:51:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:54 smithi157 ceph-mon[130879]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:55.207 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T08:51:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:55.208 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-08-25T08:51:55.208 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:55.208 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:55.209 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-08-25 08:51:05.610933360 +0000 2023-08-25T08:51:55.209 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-08-25 08:45:21.508528186 +0000 2023-08-25T08:51:55.209 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-08-25 08:45:21.508528186 +0000 2023-08-25T08:51:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-08-25 08:45:21.479528858 +0000 2023-08-25T08:51:55.210 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T08:51:55.308 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.308 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.308 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: Reconfiguring mon.c (monmap changed)... 2023-08-25T08:51:55.308 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T08:51:55.308 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T08:51:55.309 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:55.309 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: Reconfiguring daemon mon.c on smithi181 2023-08-25T08:51:55.309 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi157=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:51:55.310 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: Saving service mgr spec with placement smithi099=x;smithi157=y;smithi181=z;count:3 2023-08-25T08:51:55.310 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:55.311 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:54 smithi099 ceph-mon[97641]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:55.312 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-08-25T08:51:55.312 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-08-25T08:51:55.313 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000316701 s, 1.6 MB/s 2023-08-25T08:51:55.315 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T08:51:55.360 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2023-08-25T08:51:55.445 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T08:51:55.445 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:55.446 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-08-25T08:51:55.446 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:55.446 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:55.446 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-08-25 08:51:05.610933360 +0000 2023-08-25T08:51:55.446 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-08-25 08:45:22.340508899 +0000 2023-08-25T08:51:55.446 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-08-25 08:45:22.340508899 +0000 2023-08-25T08:51:55.446 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-08-25 08:45:22.314509501 +0000 2023-08-25T08:51:55.447 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T08:51:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-08-25T08:51:55.553 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-08-25T08:51:55.553 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000303562 s, 1.7 MB/s 2023-08-25T08:51:55.555 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T08:51:55.600 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2023-08-25T08:51:55.684 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T08:51:55.684 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:55.684 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-08-25T08:51:55.684 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:55.685 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:55.685 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-08-25 08:51:05.610933360 +0000 2023-08-25T08:51:55.685 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-08-25 08:45:23.174489565 +0000 2023-08-25T08:51:55.685 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-08-25 08:45:23.174489565 +0000 2023-08-25T08:51:55.685 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-08-25 08:45:23.145490237 +0000 2023-08-25T08:51:55.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T08:51:55.789 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-08-25T08:51:55.789 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-08-25T08:51:55.789 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000316496 s, 1.6 MB/s 2023-08-25T08:51:55.791 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T08:51:55.836 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-08-25T08:51:55.836 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T08:51:55.858 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T08:51:55.859 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2023-08-25T08:51:55.919 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T08:51:55.919 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:55.920 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 390224 Links: 1 2023-08-25T08:51:55.920 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:55.920 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:55.920 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-08-25 08:51:42.469796856 +0000 2023-08-25T08:51:55.920 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-08-25 08:42:06.828788939 +0000 2023-08-25T08:51:55.921 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-08-25 08:42:06.828788939 +0000 2023-08-25T08:51:55.921 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-08-25T08:51:55.921 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T08:51:55.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-08-25T08:51:55.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-08-25T08:51:55.991 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000136807 s, 3.7 MB/s 2023-08-25T08:51:55.991 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T08:51:56.052 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2023-08-25T08:51:56.111 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T08:51:56.111 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:56.112 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 388701 Links: 1 2023-08-25T08:51:56.112 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:56.112 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:56.112 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-08-25 08:51:42.469796856 +0000 2023-08-25T08:51:56.112 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-08-25 08:42:07.359788843 +0000 2023-08-25T08:51:56.113 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-08-25 08:42:07.359788843 +0000 2023-08-25T08:51:56.113 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-08-25T08:51:56.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T08:51:56.179 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-08-25T08:51:56.180 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-08-25T08:51:56.180 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000112604 s, 4.5 MB/s 2023-08-25T08:51:56.181 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T08:51:56.241 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2023-08-25T08:51:56.299 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T08:51:56.300 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:56.300 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 388798 Links: 1 2023-08-25T08:51:56.300 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:56.300 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:56.301 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-08-25 08:51:42.470796856 +0000 2023-08-25T08:51:56.301 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-08-25 08:42:07.876788751 +0000 2023-08-25T08:51:56.301 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-08-25 08:42:07.876788751 +0000 2023-08-25T08:51:56.301 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-08-25T08:51:56.302 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T08:51:56.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-08-25T08:51:56.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-08-25T08:51:56.367 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000139164 s, 3.7 MB/s 2023-08-25T08:51:56.368 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T08:51:56.427 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2023-08-25T08:51:56.485 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T08:51:56.485 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:56.485 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 391224 Links: 1 2023-08-25T08:51:56.486 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:56.486 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:56.486 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-08-25 08:51:42.470796856 +0000 2023-08-25T08:51:56.486 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-08-25 08:42:08.420788653 +0000 2023-08-25T08:51:56.486 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-08-25 08:42:08.420788653 +0000 2023-08-25T08:51:56.486 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-08-25T08:51:56.487 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T08:51:56.551 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-08-25T08:51:56.552 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-08-25T08:51:56.552 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.6104e-05 s, 5.3 MB/s 2023-08-25T08:51:56.553 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T08:51:56.610 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-08-25T08:51:56.611 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T08:51:56.628 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T08:51:56.628 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2023-08-25T08:51:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:51:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T08:51:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:51:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:56 smithi157 ceph-mon[130879]: Deploying daemon mgr.z on smithi181 2023-08-25T08:51:56.685 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T08:51:56.686 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:56.686 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 386349 Links: 1 2023-08-25T08:51:56.686 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:56.686 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:56.686 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-08-25 08:51:34.081840331 +0000 2023-08-25T08:51:56.687 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-08-25 08:42:05.562261956 +0000 2023-08-25T08:51:56.687 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-08-25 08:42:05.562261956 +0000 2023-08-25T08:51:56.687 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-08-25T08:51:56.687 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T08:51:56.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:56.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:56.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:56.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:51:56.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T08:51:56.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:51:56.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:56.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:56 smithi181 ceph-mon[131296]: Deploying daemon mgr.z on smithi181 2023-08-25T08:51:56.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:51:56.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:56.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:51:56.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:56.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:51:56.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T08:51:56.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:51:56.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:56.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:56 smithi099 ceph-mon[97641]: Deploying daemon mgr.z on smithi181 2023-08-25T08:51:56.840 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-08-25T08:51:56.840 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-08-25T08:51:56.840 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000204228 s, 2.5 MB/s 2023-08-25T08:51:56.841 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T08:51:56.904 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2023-08-25T08:51:56.966 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T08:51:56.966 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:56.966 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 386484 Links: 1 2023-08-25T08:51:56.966 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:56.967 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:56.967 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-08-25 08:51:56.916400534 +0000 2023-08-25T08:51:56.967 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-08-25 08:42:06.098251743 +0000 2023-08-25T08:51:56.967 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-08-25 08:42:06.098251743 +0000 2023-08-25T08:51:56.967 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-08-25T08:51:56.967 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T08:51:57.033 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:56 smithi181 systemd[1]: Starting Ceph mgr.z for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:51:57.036 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-08-25T08:51:57.036 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-08-25T08:51:57.037 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000201863 s, 2.5 MB/s 2023-08-25T08:51:57.037 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T08:51:57.096 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2023-08-25T08:51:57.155 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T08:51:57.155 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:57.155 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 387444 Links: 1 2023-08-25T08:51:57.156 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:57.156 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:57.156 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-08-25 08:51:56.916400534 +0000 2023-08-25T08:51:57.156 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-08-25 08:42:06.618241835 +0000 2023-08-25T08:51:57.156 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-08-25 08:42:06.618241835 +0000 2023-08-25T08:51:57.156 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-08-25T08:51:57.157 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T08:51:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-08-25T08:51:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-08-25T08:51:57.222 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000274016 s, 1.9 MB/s 2023-08-25T08:51:57.223 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T08:51:57.281 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2023-08-25T08:51:57.338 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T08:51:57.339 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T08:51:57.339 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 388785 Links: 1 2023-08-25T08:51:57.339 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:51:57.339 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T08:51:57.339 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-08-25 08:51:56.916400534 +0000 2023-08-25T08:51:57.339 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-08-25 08:42:07.143231831 +0000 2023-08-25T08:51:57.339 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-08-25 08:42:07.143231831 +0000 2023-08-25T08:51:57.340 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-08-25T08:51:57.340 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T08:51:57.360 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:57 smithi181 podman[133272]: 2023-08-25T08:51:57.386 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-08-25T08:51:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-08-25T08:51:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000225464 s, 2.3 MB/s 2023-08-25T08:51:57.387 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T08:51:57.445 INFO:tasks.cephadm:Deploying osd.0 on smithi099 with /dev/vg_nvme/lv_4... 2023-08-25T08:51:57.445 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-25T08:51:57.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:57 smithi181 ceph-mon[131296]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:57.664 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:57 smithi181 bash[133272]: cff29435c3d9c276a994beb5b5ea4d2531b370a956da1a27d05abf2577d8c187 2023-08-25T08:51:57.664 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:57 smithi181 ceph-mgr[133436]: -- 172.21.15.181:0/1745607275 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5583786a31e0 con 0x558378678c00 2023-08-25T08:51:57.665 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:57 smithi181 systemd[1]: Started Ceph mgr.z for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:51:57.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:57 smithi099 ceph-mon[97641]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:57.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:57 smithi157 ceph-mon[130879]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:57.922 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:57.663+0000 7fafce531200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T08:51:57.923 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:57.742+0000 7fafce531200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T08:51:57.923 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:57.830+0000 7fafce531200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T08:51:57.923 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:57.922+0000 7fafce531200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T08:51:58.170 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:51:58.242 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:58.090+0000 7fafce531200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T08:51:58.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.741 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:51:58.741 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T08:51:58.741 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:51:58.741 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:58.741 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:58 smithi099 ceph-mon[97641]: Deploying daemon mgr.y on smithi157 2023-08-25T08:51:58.742 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:58.452+0000 7fafce531200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T08:51:58.742 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:58.584+0000 7fafce531200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T08:51:58.742 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:58.672+0000 7fafce531200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T08:51:58.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:51:58.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T08:51:58.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:51:58.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:58.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-mon[131296]: Deploying daemon mgr.y on smithi157 2023-08-25T08:51:58.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:51:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:51:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T08:51:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:51:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:51:58.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:58 smithi157 ceph-mon[130879]: Deploying daemon mgr.y on smithi157 2023-08-25T08:51:59.192 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:58.766+0000 7fafce531200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T08:51:59.485 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:51:59 smithi157 systemd[1]: Starting Ceph mgr.y for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:51:59.492 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:59.191+0000 7fafce531200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T08:51:59.749 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:51:59 smithi157 ceph-mon[130879]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:59.749 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:51:59 smithi157 podman[132318]: 2023-08-25T08:51:59.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:51:59 smithi099 ceph-mon[97641]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:59.939 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:51:59 smithi181 ceph-mon[131296]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:51:59.939 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:59.856+0000 7fafce531200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T08:52:00.045 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:51:59 smithi157 bash[132318]: c131e149a663c9365e3f710e3b417aae19dbea5f2ea77b9bcc514abdbc92c216 2023-08-25T08:52:00.045 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:51:59 smithi157 ceph-mgr[132358]: -- 172.21.15.157:0/2128935950 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ae4d0391e0 con 0x55ae4d00e800 2023-08-25T08:52:00.045 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:51:59 smithi157 systemd[1]: Started Ceph mgr.y for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:52:00.046 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:51:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:51:59.873+0000 7fc02a447200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T08:52:00.046 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:51:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:51:59.952+0000 7fc02a447200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T08:52:00.132 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-25T08:52:00.133 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-25T08:52:00.133 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-08-25T08:52:00.133 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-08-25T08:52:00.133 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0323642 s, 324 MB/s 2023-08-25T08:52:00.133 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-08-25T08:52:00.322 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:00.044+0000 7fc02a447200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T08:52:00.322 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:00.139+0000 7fc02a447200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T08:52:00.650 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2023-08-25T08:52:00.992 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:51:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:51:59.939+0000 7fafce531200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T08:52:00.993 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:00.740+0000 7fafce531200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:52:00.993 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:00.834+0000 7fafce531200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T08:52:00.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:00.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:00.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:00.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:00.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:01.040 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.041 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.041 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.041 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.041 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:01.042 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:00.321+0000 7fc02a447200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T08:52:01.042 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:00.700+0000 7fc02a447200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T08:52:01.042 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:00.838+0000 7fc02a447200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T08:52:01.042 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:00.942+0000 7fc02a447200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T08:52:01.042 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:01.039+0000 7fc02a447200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T08:52:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:01.317 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:01.014+0000 7fafce531200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:52:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:52:01.706 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:01.316+0000 7fafce531200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T08:52:01.706 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:01.555+0000 7fafce531200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T08:52:01.706 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:01.706+0000 7fafce531200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T08:52:01.874 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:01.509+0000 7fc02a447200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T08:52:01.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:01.963 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.963 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.963 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.963 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.963 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:01.963 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:01.964 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.964 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:52:01.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:52:01.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:01.971 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:01.971 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.971 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.971 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.972 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.972 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:01.972 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:01.972 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:01.972 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:52:01.972 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:52:01.972 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:02.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:02.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:02.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:02.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:02.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:02.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:02.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:02.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:02.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T08:52:02.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T08:52:02.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:02.242 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:01.970+0000 7fafce531200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T08:52:02.628 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:02.184+0000 7fc02a447200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T08:52:02.629 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:02.265+0000 7fc02a447200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T08:52:02.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:02 smithi181 ceph-mon[131296]: Reconfiguring mgr.x (unknown last config time)... 2023-08-25T08:52:02.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:02 smithi181 ceph-mon[131296]: Reconfiguring daemon mgr.x on smithi099 2023-08-25T08:52:02.988 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:02.665+0000 7fafce531200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T08:52:02.988 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:02.794+0000 7fafce531200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T08:52:03.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.012+0000 7fa8606f4700 1 -- 172.21.15.99:0/2182612104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8580748f0 msgr2=0x7fa858072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:03.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.012+0000 7fa8606f4700 1 --2- 172.21.15.99:0/2182612104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8580748f0 0x7fa858072d50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa854009990 tx=0x7fa854009ca0 comp rx=0 tx=0).stop 2023-08-25T08:52:03.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.013+0000 7fa8606f4700 1 -- 172.21.15.99:0/2182612104 shutdown_connections 2023-08-25T08:52:03.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.013+0000 7fa8606f4700 1 --2- 172.21.15.99:0/2182612104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8581088a0 0x7fa85810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:03.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.013+0000 7fa8606f4700 1 --2- 172.21.15.99:0/2182612104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8580748f0 0x7fa858072d50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:03.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.013+0000 7fa8606f4700 1 --2- 172.21.15.99:0/2182612104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa858073320 0x7fa8580737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:03.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.013+0000 7fa8606f4700 1 -- 172.21.15.99:0/2182612104 >> 172.21.15.99:0/2182612104 conn(0x7fa8580fbf40 msgr2=0x7fa8580fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:52:03.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.013+0000 7fa8606f4700 1 -- 172.21.15.99:0/2182612104 shutdown_connections 2023-08-25T08:52:03.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.013+0000 7fa8606f4700 1 -- 172.21.15.99:0/2182612104 wait complete. 2023-08-25T08:52:03.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa8606f4700 1 Processor -- start 2023-08-25T08:52:03.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa8606f4700 1 -- start start 2023-08-25T08:52:03.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa8606f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858073320 0x7fa8581988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:03.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa8606f4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8580748f0 0x7fa858198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:03.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa8606f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8581088a0 0x7fa85819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:03.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa8606f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581a0430 con 0x7fa858073320 2023-08-25T08:52:03.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa8606f4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581a0570 con 0x7fa8580748f0 2023-08-25T08:52:03.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa8606f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581a0700 con 0x7fa8581088a0 2023-08-25T08:52:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa85e490700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858073320 0x7fa8581988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa85ec91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8581088a0 0x7fa85819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa85dc8f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8580748f0 0x7fa858198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa85ec91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8581088a0 0x7fa85819fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:35290/0 (socket says 172.21.15.99:35290) 2023-08-25T08:52:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.014+0000 7fa85ec91700 1 -- 172.21.15.99:0/1151196061 learned_addr learned my addr 172.21.15.99:0/1151196061 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:52:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa85dc8f700 1 -- 172.21.15.99:0/1151196061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8581088a0 msgr2=0x7fa85819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa85dc8f700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8581088a0 0x7fa85819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:03.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa85dc8f700 1 -- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858073320 msgr2=0x7fa8581988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:03.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa85dc8f700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858073320 0x7fa8581988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:03.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa85dc8f700 1 -- 172.21.15.99:0/1151196061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540095f0 con 0x7fa8580748f0 2023-08-25T08:52:03.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa85e490700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858073320 0x7fa8581988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:52:03.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa85dc8f700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8580748f0 0x7fa858198de0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e9f0 tx=0x7fa84800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:52:03.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84800c9c0 con 0x7fa8580748f0 2023-08-25T08:52:03.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa848010430 con 0x7fa8580748f0 2023-08-25T08:52:03.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8581a09e0 con 0x7fa8580748f0 2023-08-25T08:52:03.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848018680 con 0x7fa8580748f0 2023-08-25T08:52:03.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.015+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8581a0f30 con 0x7fa8580748f0 2023-08-25T08:52:03.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.016+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fa84800cb20 con 0x7fa8580748f0 2023-08-25T08:52:03.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.016+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa858106fa0 con 0x7fa8580748f0 2023-08-25T08:52:03.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.017+0000 7fa84f7fe700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa8440383e0 0x7fa84403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:03.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.017+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa848014070 con 0x7fa8580748f0 2023-08-25T08:52:03.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.017+0000 7fa85e490700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa8440383e0 0x7fa84403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:03.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.018+0000 7fa85e490700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa8440383e0 0x7fa84403a8a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa85400b2e0 tx=0x7fa854005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:52:03.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.022+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa84801f780 con 0x7fa8580748f0 2023-08-25T08:52:03.144 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:02 smithi099 ceph-mon[97641]: Reconfiguring mgr.x (unknown last config time)... 2023-08-25T08:52:03.145 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:02 smithi099 ceph-mon[97641]: Reconfiguring daemon mgr.x on smithi099 2023-08-25T08:52:03.242 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:02.986+0000 7fafce531200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T08:52:03.242 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:03.099+0000 7fafce531200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T08:52:03.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:03.243+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa85802ce70 con 0x7fa8440383e0 2023-08-25T08:52:03.291 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:03.031+0000 7fc02a447200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:52:03.292 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:03.120+0000 7fc02a447200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T08:52:03.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:02 smithi157 ceph-mon[130879]: Reconfiguring mgr.x (unknown last config time)... 2023-08-25T08:52:03.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:02 smithi157 ceph-mon[130879]: Reconfiguring daemon mgr.x on smithi099 2023-08-25T08:52:03.567 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:03.290+0000 7fc02a447200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:52:03.742 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:03.313+0000 7fafce531200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T08:52:03.878 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:03.566+0000 7fc02a447200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T08:52:03.879 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:03.795+0000 7fc02a447200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T08:52:04.175 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:04.175 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:04.175 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:04.176 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:04.176 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:04.176 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.176 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.176 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:04.176 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:04.176 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:04.177 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.205 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:03.944+0000 7fc02a447200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T08:52:04.205 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:04.206 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:04.206 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:04.206 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:04.206 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:04.206 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.206 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.206 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:04.206 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:04.207 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:04.207 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:04.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:04.492 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:52:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:52:04.170+0000 7fafce531200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T08:52:04.628 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:04.203+0000 7fc02a447200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T08:52:04.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:04.941+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 14) v1 ==== 89903+0+0 (secure 0 0 0) 0x7fa8480187e0 con 0x7fa8580748f0 2023-08-25T08:52:05.221 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:04.899+0000 7fc02a447200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T08:52:05.222 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:05.030+0000 7fc02a447200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T08:52:05.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-mon[130879]: Standby manager daemon z started 2023-08-25T08:52:05.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T08:52:05.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:52:05.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T08:52:05.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:52:05.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-mon[130879]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:05.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:05.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:04 smithi181 ceph-mon[131296]: Standby manager daemon z started 2023-08-25T08:52:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:04 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T08:52:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:04 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:52:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:04 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T08:52:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:04 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:52:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:04 smithi181 ceph-mon[131296]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:05.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:04 smithi099 ceph-mon[97641]: Standby manager daemon z started 2023-08-25T08:52:05.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:04 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T08:52:05.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:04 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:52:05.286 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:04 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T08:52:05.286 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:04 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.181:0/4168605768' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:52:05.286 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:04 smithi099 ceph-mon[97641]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:05.286 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:05.547 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:05.220+0000 7fc02a447200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T08:52:05.548 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:05.332+0000 7fc02a447200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T08:52:05.878 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:05.546+0000 7fc02a447200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T08:52:06.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:05 smithi181 ceph-mon[131296]: mgrmap e14: x(active, since 70s), standbys: z 2023-08-25T08:52:06.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T08:52:06.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:05 smithi099 ceph-mon[97641]: mgrmap e14: x(active, since 70s), standbys: z 2023-08-25T08:52:06.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T08:52:06.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:05 smithi157 ceph-mon[130879]: mgrmap e14: x(active, since 70s), standbys: z 2023-08-25T08:52:06.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T08:52:06.879 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:52:06.401+0000 7fc02a447200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T08:52:07.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4105909089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]: dispatch 2023-08-25T08:52:07.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]: dispatch 2023-08-25T08:52:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]': finished 2023-08-25T08:52:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: osdmap e5: 1 total, 0 up, 1 in 2023-08-25T08:52:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: Standby manager daemon y started 2023-08-25T08:52:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T08:52:07.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:52:07.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T08:52:07.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:06 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:52:07.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:07.269+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mon.2 v2:172.21.15.157:3300/0 8 ==== mgrmap(e 15) v1 ==== 134615+0+0 (secure 0 0 0) 0x7fa848016330 con 0x7fa8580748f0 2023-08-25T08:52:07.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4105909089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]: dispatch 2023-08-25T08:52:07.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]: dispatch 2023-08-25T08:52:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]': finished 2023-08-25T08:52:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: osdmap e5: 1 total, 0 up, 1 in 2023-08-25T08:52:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:07.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: Standby manager daemon y started 2023-08-25T08:52:07.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T08:52:07.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:52:07.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T08:52:07.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:06 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:52:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4105909089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]: dispatch 2023-08-25T08:52:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]: dispatch 2023-08-25T08:52:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd38b95e-eec4-4af7-91aa-73e6e78584e3"}]': finished 2023-08-25T08:52:07.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: osdmap e5: 1 total, 0 up, 1 in 2023-08-25T08:52:07.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:07.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:07.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: Standby manager daemon y started 2023-08-25T08:52:07.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T08:52:07.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:52:07.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T08:52:07.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:06 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.157:0/2406643275' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:52:08.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:07 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4140694399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:08.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:07 smithi181 ceph-mon[131296]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-08-25T08:52:08.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T08:52:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:07 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4140694399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:07 smithi099 ceph-mon[97641]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-08-25T08:52:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T08:52:08.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:07 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4140694399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:07 smithi157 ceph-mon[130879]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-08-25T08:52:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T08:52:09.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:08 smithi181 ceph-mon[131296]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:09.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:08 smithi099 ceph-mon[97641]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:09.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:08 smithi157 ceph-mon[130879]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:11.709 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:11 smithi099 ceph-mon[97641]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:11.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:11 smithi181 ceph-mon[131296]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:11.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:11 smithi157 ceph-mon[130879]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:13.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:13 smithi181 ceph-mon[131296]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:13.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:13 smithi099 ceph-mon[97641]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:13.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:13 smithi157 ceph-mon[130879]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:14.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-25T08:52:14.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:14.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:14 smithi181 ceph-mon[131296]: Deploying daemon osd.0 on smithi099 2023-08-25T08:52:14.783 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-25T08:52:14.783 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:14.783 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:14 smithi099 ceph-mon[97641]: Deploying daemon osd.0 on smithi099 2023-08-25T08:52:14.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-25T08:52:14.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:14.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:14 smithi157 ceph-mon[130879]: Deploying daemon osd.0 on smithi099 2023-08-25T08:52:15.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:15 smithi181 ceph-mon[131296]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:15.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:15 smithi099 ceph-mon[97641]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:15.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:15 smithi157 ceph-mon[130879]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:17.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:17 smithi181 ceph-mon[131296]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:17.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:17 smithi099 ceph-mon[97641]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:17.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:17 smithi157 ceph-mon[130879]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:19.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:19 smithi181 ceph-mon[131296]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:19.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:19 smithi099 ceph-mon[97641]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:19.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:19 smithi157 ceph-mon[130879]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:20.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:20.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:20.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:20.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:20 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:20.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:20 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:20.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:20 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:20.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:20 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:20.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:20 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:20.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:20 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:21.643 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:21 smithi099 ceph-mon[97641]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:21.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:21 smithi157 ceph-mon[130879]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:21.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:21 smithi181 ceph-mon[131296]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:22.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.706+0000 7fa84f7fe700 1 -- 172.21.15.99:0/1151196061 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa85802ce70 con 0x7fa8440383e0 2023-08-25T08:52:22.707 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 0 on host 'smithi099' 2023-08-25T08:52:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa8440383e0 msgr2=0x7fa84403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa8440383e0 0x7fa84403a8a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa85400b2e0 tx=0x7fa854005fb0 comp rx=0 tx=0).stop 2023-08-25T08:52:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8580748f0 msgr2=0x7fa858198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8580748f0 0x7fa858198de0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e9f0 tx=0x7fa84800ed00 comp rx=0 tx=0).stop 2023-08-25T08:52:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 shutdown_connections 2023-08-25T08:52:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8581088a0 0x7fa85819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858073320 0x7fa8581988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa8440383e0 0x7fa84403a8a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 --2- 172.21.15.99:0/1151196061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8580748f0 0x7fa858198de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 >> 172.21.15.99:0/1151196061 conn(0x7fa8580fbf40 msgr2=0x7fa8580fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:52:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 shutdown_connections 2023-08-25T08:52:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:52:22.709+0000 7fa8606f4700 1 -- 172.21.15.99:0/1151196061 wait complete. 2023-08-25T08:52:23.202 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.203 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.203 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:23.203 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:23.203 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:23.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:23.205 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.206 DEBUG:teuthology.orchestra.run.smithi099:osd.0> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.0.service 2023-08-25T08:52:23.212 INFO:tasks.cephadm:Deploying osd.1 on smithi157 with /dev/vg_nvme/lv_4... 2023-08-25T08:52:23.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-25T08:52:23.285 INFO:journalctl@ceph.osd.0.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:52:23.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:23.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:23.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:23.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:23.771 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T08:52:23.820 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:52:23 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-0[110552]: 2023-08-25T08:52:23.480+0000 7f6ce48ed640 -1 osd.0 0 log_to_monitors true 2023-08-25T08:52:24.090 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:24 smithi099 ceph-mon[97641]: from='osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T08:52:24.090 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:24 smithi099 ceph-mon[97641]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T08:52:24.288 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:24 smithi157 ceph-mon[130879]: from='osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T08:52:24.288 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:24 smithi157 ceph-mon[130879]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T08:52:24.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:24 smithi181 ceph-mon[131296]: from='osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T08:52:24.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:24 smithi181 ceph-mon[131296]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T08:52:25.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:25 smithi157 ceph-mon[130879]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-25T08:52:25.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:25 smithi157 ceph-mon[130879]: osdmap e6: 1 total, 0 up, 1 in 2023-08-25T08:52:25.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:25 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:25.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:25 smithi157 ceph-mon[130879]: from='osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:52:25.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:25 smithi157 ceph-mon[130879]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:52:25.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:25 smithi157 ceph-mon[130879]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:25.312 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:52:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-0[110552]: 2023-08-25T08:52:25.050+0000 7f6cda765700 -1 osd.0 0 waiting for initial osdmap 2023-08-25T08:52:25.313 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:52:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-0[110552]: 2023-08-25T08:52:25.054+0000 7f6cd3d55700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:52:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:25 smithi099 ceph-mon[97641]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-25T08:52:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:25 smithi099 ceph-mon[97641]: osdmap e6: 1 total, 0 up, 1 in 2023-08-25T08:52:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:25 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:25 smithi099 ceph-mon[97641]: from='osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:52:25.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:25 smithi099 ceph-mon[97641]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:52:25.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:25 smithi099 ceph-mon[97641]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-25T08:52:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-25T08:52:25.340 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2023-08-25T08:52:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2023-08-25T08:52:25.340 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00810851 s, 1.3 GB/s 2023-08-25T08:52:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2023-08-25T08:52:25.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:25 smithi181 ceph-mon[131296]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-25T08:52:25.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:25 smithi181 ceph-mon[131296]: osdmap e6: 1 total, 0 up, 1 in 2023-08-25T08:52:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:25 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:25 smithi181 ceph-mon[131296]: from='osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:52:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:25 smithi181 ceph-mon[131296]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:52:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:25 smithi181 ceph-mon[131296]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:25.816 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2023-08-25T08:52:26.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:52:26.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: osdmap e7: 1 total, 0 up, 1 in 2023-08-25T08:52:26.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:26.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:52:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.368 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T08:52:26.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:52:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: osdmap e7: 1 total, 0 up, 1 in 2023-08-25T08:52:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:52:26.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:26.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:26.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:52:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: osdmap e7: 1 total, 0 up, 1 in 2023-08-25T08:52:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:52:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:26.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:26.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:26.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:27.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:52:27.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:52:27.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi099 2023-08-25T08:52:27.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi099 to 17267M 2023-08-25T08:52:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] boot 2023-08-25T08:52:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: osdmap e8: 1 total, 1 up, 1 in 2023-08-25T08:52:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:27 smithi099 ceph-mon[97641]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:27.335 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:52:27.335 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:52:27.336 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi099 2023-08-25T08:52:27.336 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi099 to 17267M 2023-08-25T08:52:27.336 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:27.336 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] boot 2023-08-25T08:52:27.336 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: osdmap e8: 1 total, 1 up, 1 in 2023-08-25T08:52:27.336 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:27.337 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:27 smithi157 ceph-mon[130879]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:27.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:52:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:52:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi099 2023-08-25T08:52:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi099 to 17267M 2023-08-25T08:52:27.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:27.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: osd.0 [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] boot 2023-08-25T08:52:27.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: osdmap e8: 1 total, 1 up, 1 in 2023-08-25T08:52:27.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:52:27.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:27 smithi181 ceph-mon[131296]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T08:52:27.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.523+0000 7fd549076700 1 -- 172.21.15.157:0/1918439928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5441014b0 msgr2=0x7fd544107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:27.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.523+0000 7fd549076700 1 --2- 172.21.15.157:0/1918439928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5441014b0 0x7fd544107d60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00b0d0 tx=0x7fd53c00b3e0 comp rx=0 tx=0).stop 2023-08-25T08:52:27.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.523+0000 7fd549076700 1 -- 172.21.15.157:0/1918439928 shutdown_connections 2023-08-25T08:52:27.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.523+0000 7fd549076700 1 --2- 172.21.15.157:0/1918439928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5441014b0 0x7fd544107d60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:27.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.523+0000 7fd549076700 1 --2- 172.21.15.157:0/1918439928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5440ff910 0x7fd5440ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:27.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.523+0000 7fd549076700 1 --2- 172.21.15.157:0/1918439928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544100b10 0x7fd544100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:27.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.523+0000 7fd549076700 1 -- 172.21.15.157:0/1918439928 >> 172.21.15.157:0/1918439928 conn(0x7fd5440faeb0 msgr2=0x7fd5440fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:52:27.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.523+0000 7fd549076700 1 -- 172.21.15.157:0/1918439928 shutdown_connections 2023-08-25T08:52:27.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.524+0000 7fd549076700 1 -- 172.21.15.157:0/1918439928 wait complete. 2023-08-25T08:52:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.524+0000 7fd549076700 1 Processor -- start 2023-08-25T08:52:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.524+0000 7fd549076700 1 -- start start 2023-08-25T08:52:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd549076700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5440ff910 0x7fd5441038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd549076700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544100b10 0x7fd544101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd549076700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5441014b0 0x7fd544102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd549076700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd544102bb0 con 0x7fd5441014b0 2023-08-25T08:52:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd549076700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd544102d20 con 0x7fd544100b10 2023-08-25T08:52:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd549076700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5441a0870 con 0x7fd5440ff910 2023-08-25T08:52:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544100b10 0x7fd544101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544100b10 0x7fd544101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57616/0 (socket says 172.21.15.157:57616) 2023-08-25T08:52:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 -- 172.21.15.157:0/1377482736 learned_addr learned my addr 172.21.15.157:0/1377482736 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-08-25T08:52:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd542d9d700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5440ff910 0x7fd5441038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54359e700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5441014b0 0x7fd544102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 -- 172.21.15.157:0/1377482736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5440ff910 msgr2=0x7fd5441038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5440ff910 0x7fd5441038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 -- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5441014b0 msgr2=0x7fd544102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5441014b0 0x7fd544102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 -- 172.21.15.157:0/1377482736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c009d00 con 0x7fd544100b10 2023-08-25T08:52:27.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd54259c700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544100b10 0x7fd544101ef0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd53800d980 tx=0x7fd53800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:52:27.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd52bfff700 1 -- 172.21.15.157:0/1377482736 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538009b40 con 0x7fd544100b10 2023-08-25T08:52:27.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5441a0b50 con 0x7fd544100b10 2023-08-25T08:52:27.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd52bfff700 1 -- 172.21.15.157:0/1377482736 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd53800de60 con 0x7fd544100b10 2023-08-25T08:52:27.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd52bfff700 1 -- 172.21.15.157:0/1377482736 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538005580 con 0x7fd544100b10 2023-08-25T08:52:27.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.525+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5441a1050 con 0x7fd544100b10 2023-08-25T08:52:27.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.526+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54410b1d0 con 0x7fd544100b10 2023-08-25T08:52:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.527+0000 7fd52bfff700 1 -- 172.21.15.157:0/1377482736 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 134615+0+0 (secure 0 0 0) 0x7fd538004750 con 0x7fd544100b10 2023-08-25T08:52:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.528+0000 7fd52bfff700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd52c09fec0 0x7fd52c0a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.528+0000 7fd52bfff700 1 -- 172.21.15.157:0/1377482736 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fd5380c9760 con 0x7fd544100b10 2023-08-25T08:52:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.528+0000 7fd542d9d700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd52c09fec0 0x7fd52c0a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:27.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.529+0000 7fd542d9d700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd52c09fec0 0x7fd52c0a2380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd534006fd0 tx=0x7fd534008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:52:27.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.532+0000 7fd52bfff700 1 -- 172.21.15.157:0/1377482736 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd538094f60 con 0x7fd544100b10 2023-08-25T08:52:27.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:27.728+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd544069450 con 0x7fd52c09fec0 2023-08-25T08:52:28.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:28 smithi157 ceph-mon[130879]: osdmap e9: 1 total, 1 up, 1 in 2023-08-25T08:52:28.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:28 smithi157 ceph-mon[130879]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:28.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:28 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:28.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:28 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:28.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:28 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:28.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:28 smithi181 ceph-mon[131296]: osdmap e9: 1 total, 1 up, 1 in 2023-08-25T08:52:28.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:28 smithi181 ceph-mon[131296]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:28.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:28 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:28.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:28 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:28.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:28 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:28 smithi099 ceph-mon[97641]: osdmap e9: 1 total, 1 up, 1 in 2023-08-25T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:28 smithi099 ceph-mon[97641]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:28 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:28 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:28.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:28 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:29.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:29 smithi157 ceph-mon[130879]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:29.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:29 smithi181 ceph-mon[131296]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:29.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:29 smithi099 ceph-mon[97641]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:30.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:30 smithi181 ceph-mon[131296]: from='client.? 172.21.15.157:0/1607951002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]: dispatch 2023-08-25T08:52:30.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:30 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]: dispatch 2023-08-25T08:52:30.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:30 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]': finished 2023-08-25T08:52:30.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:30 smithi181 ceph-mon[131296]: osdmap e10: 2 total, 1 up, 2 in 2023-08-25T08:52:30.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:30.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:30 smithi099 ceph-mon[97641]: from='client.? 172.21.15.157:0/1607951002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]: dispatch 2023-08-25T08:52:30.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:30 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]: dispatch 2023-08-25T08:52:30.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:30 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]': finished 2023-08-25T08:52:30.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:30 smithi099 ceph-mon[97641]: osdmap e10: 2 total, 1 up, 2 in 2023-08-25T08:52:30.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:30.628 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:30 smithi157 ceph-mon[130879]: from='client.? 172.21.15.157:0/1607951002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]: dispatch 2023-08-25T08:52:30.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:30 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]: dispatch 2023-08-25T08:52:30.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:30 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b212381-183b-4878-be8a-e088b3e62122"}]': finished 2023-08-25T08:52:30.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:30 smithi157 ceph-mon[130879]: osdmap e10: 2 total, 1 up, 2 in 2023-08-25T08:52:30.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:31.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:31 smithi181 ceph-mon[131296]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:31.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:31 smithi181 ceph-mon[131296]: from='client.? 172.21.15.157:0/3631106681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:31.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:31 smithi099 ceph-mon[97641]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:31.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:31 smithi099 ceph-mon[97641]: from='client.? 172.21.15.157:0/3631106681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:31.628 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:31 smithi157 ceph-mon[130879]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:31 smithi157 ceph-mon[130879]: from='client.? 172.21.15.157:0/3631106681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:33.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:33 smithi181 ceph-mon[131296]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:33.763 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:33 smithi157 ceph-mon[130879]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:33 smithi099 ceph-mon[97641]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:35.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:34 smithi181 ceph-mon[131296]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:35.269 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:34 smithi157 ceph-mon[130879]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:35.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:34 smithi099 ceph-mon[97641]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:37.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:37 smithi181 ceph-mon[131296]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:37 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-25T08:52:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:37 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:37 smithi181 ceph-mon[131296]: Deploying daemon osd.1 on smithi157 2023-08-25T08:52:37.756 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:37 smithi157 ceph-mon[130879]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:37.756 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:37 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-25T08:52:37.757 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:37 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:37.757 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:37 smithi157 ceph-mon[130879]: Deploying daemon osd.1 on smithi157 2023-08-25T08:52:37.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:37 smithi099 ceph-mon[97641]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:37 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-25T08:52:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:37 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:37 smithi099 ceph-mon[97641]: Deploying daemon osd.1 on smithi157 2023-08-25T08:52:39.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:39 smithi181 ceph-mon[131296]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:39.811 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:39 smithi099 ceph-mon[97641]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:39 smithi157 ceph-mon[130879]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:41.628 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:41 smithi157 ceph-mon[130879]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:41 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:41 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:41 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:41 smithi181 ceph-mon[131296]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:41 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:41 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:41.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:41 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:41.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:41 smithi099 ceph-mon[97641]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:41.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:41 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:41 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:41 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:43 smithi157 ceph-mon[130879]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:43.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:43 smithi181 ceph-mon[131296]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:43.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:43.797 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 1 on host 'smithi157' 2023-08-25T08:52:43.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.797+0000 7fd52bfff700 1 -- 172.21.15.157:0/1377482736 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd544069450 con 0x7fd52c09fec0 2023-08-25T08:52:43.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.798+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd52c09fec0 msgr2=0x7fd52c0a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:43.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.798+0000 7fd549076700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd52c09fec0 0x7fd52c0a2380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd534006fd0 tx=0x7fd534008040 comp rx=0 tx=0).stop 2023-08-25T08:52:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.798+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544100b10 msgr2=0x7fd544101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.798+0000 7fd549076700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544100b10 0x7fd544101ef0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd53800d980 tx=0x7fd53800dd40 comp rx=0 tx=0).stop 2023-08-25T08:52:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.799+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 shutdown_connections 2023-08-25T08:52:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.799+0000 7fd549076700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5441014b0 0x7fd544102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.799+0000 7fd549076700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5440ff910 0x7fd5441038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.799+0000 7fd549076700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd52c09fec0 0x7fd52c0a2380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.799+0000 7fd549076700 1 --2- 172.21.15.157:0/1377482736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd544100b10 0x7fd544101ef0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.799+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 >> 172.21.15.157:0/1377482736 conn(0x7fd5440faeb0 msgr2=0x7fd544068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:52:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.799+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 shutdown_connections 2023-08-25T08:52:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:52:43.799+0000 7fd549076700 1 -- 172.21.15.157:0/1377482736 wait complete. 2023-08-25T08:52:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:43 smithi099 ceph-mon[97641]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:44.258 DEBUG:teuthology.orchestra.run.smithi157:osd.1> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.1.service 2023-08-25T08:52:44.269 INFO:tasks.cephadm:Deploying osd.2 on smithi181 with /dev/vg_nvme/lv_4... 2023-08-25T08:52:44.270 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-25T08:52:44.296 INFO:journalctl@ceph.osd.1.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:52:44.831 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:52:45.006 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:44 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:45.006 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:44 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:45.006 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:44 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:45.007 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:52:44 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-1[135921]: 2023-08-25T08:52:44.922+0000 7f68c6815640 -1 osd.1 0 log_to_monitors true 2023-08-25T08:52:45.041 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:44 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:45.041 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:44 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:45.041 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:44 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:44 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:44 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:44 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:52:46.041 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:45 smithi157 ceph-mon[130879]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:46.041 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:45 smithi157 ceph-mon[130879]: from='osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T08:52:46.041 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:45 smithi157 ceph-mon[130879]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T08:52:46.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:45 smithi099 ceph-mon[97641]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:46.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:45 smithi099 ceph-mon[97641]: from='osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T08:52:46.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:45 smithi099 ceph-mon[97641]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T08:52:46.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:45 smithi181 ceph-mon[131296]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:45 smithi181 ceph-mon[131296]: from='osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T08:52:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:45 smithi181 ceph-mon[131296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T08:52:46.315 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-25T08:52:46.315 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-25T08:52:46.315 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-08-25T08:52:46.315 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-08-25T08:52:46.315 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114429 s, 916 MB/s 2023-08-25T08:52:46.315 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-08-25T08:52:46.673 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2023-08-25T08:52:46.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-25T08:52:46.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: osdmap e11: 2 total, 1 up, 2 in 2023-08-25T08:52:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:52:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:52:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:52:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:46.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:46.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:46.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-25T08:52:47.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: osdmap e11: 2 total, 1 up, 2 in 2023-08-25T08:52:47.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:47.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:52:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:52:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:52:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:47.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.128 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-1[135921]: 2023-08-25T08:52:46.772+0000 7f68bbe8c700 -1 osd.1 0 waiting for initial osdmap 2023-08-25T08:52:47.129 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-1[135921]: 2023-08-25T08:52:46.775+0000 7f68b647e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:52:47.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-25T08:52:47.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: osdmap e11: 2 total, 1 up, 2 in 2023-08-25T08:52:47.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:47.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:52:47.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:52:47.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:52:47.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:47.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:52:47.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:52:47.228 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:52:47.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:47 smithi181 ceph-mon[131296]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:47.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:47 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi157 2023-08-25T08:52:47.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:47 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi157 to 17168M 2023-08-25T08:52:47.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:47 smithi181 ceph-mon[131296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:52:47.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:47 smithi181 ceph-mon[131296]: osdmap e12: 2 total, 1 up, 2 in 2023-08-25T08:52:47.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:47.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:48.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:47 smithi099 ceph-mon[97641]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:48.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:47 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi157 2023-08-25T08:52:48.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:47 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi157 to 17168M 2023-08-25T08:52:48.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:47 smithi099 ceph-mon[97641]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:52:48.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:47 smithi099 ceph-mon[97641]: osdmap e12: 2 total, 1 up, 2 in 2023-08-25T08:52:48.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:48.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:48.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:47 smithi157 ceph-mon[130879]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:48.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:47 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi157 2023-08-25T08:52:48.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:47 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi157 to 17168M 2023-08-25T08:52:48.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:47 smithi157 ceph-mon[130879]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:52:48.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:47 smithi157 ceph-mon[130879]: osdmap e12: 2 total, 1 up, 2 in 2023-08-25T08:52:48.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:48.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:48.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 -- 172.21.15.181:0/2254872432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed01004c0 msgr2=0x7f3ed0107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:48.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/2254872432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed01004c0 0x7f3ed0107d60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800b0a0 tx=0x7f3ec800b3b0 comp rx=0 tx=0).stop 2023-08-25T08:52:48.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 -- 172.21.15.181:0/2254872432 shutdown_connections 2023-08-25T08:52:48.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/2254872432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed01004c0 0x7f3ed0107d60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:48.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/2254872432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed0100f30 0x7f3ed0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:48.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/2254872432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed00ffb00 0x7f3ed00fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 -- 172.21.15.181:0/2254872432 >> 172.21.15.181:0/2254872432 conn(0x7f3ed00fa5d0 msgr2=0x7f3ed00fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:52:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 -- 172.21.15.181:0/2254872432 shutdown_connections 2023-08-25T08:52:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.428+0000 7f3ed51f3700 1 -- 172.21.15.181:0/2254872432 wait complete. 2023-08-25T08:52:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ed51f3700 1 Processor -- start 2023-08-25T08:52:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ed51f3700 1 -- start start 2023-08-25T08:52:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ed51f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed00ffb00 0x7f3ed0196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ed51f3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed01004c0 0x7f3ed0196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ed51f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0100f30 0x7f3ed019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ed51f3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed019e2c0 con 0x7f3ed0100f30 2023-08-25T08:52:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ed51f3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed019e400 con 0x7f3ed01004c0 2023-08-25T08:52:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ed51f3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed019e590 con 0x7f3ed00ffb00 2023-08-25T08:52:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3eced9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed00ffb00 0x7f3ed0196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3eced9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed00ffb00 0x7f3ed0196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43110/0 (socket says 172.21.15.181:43110) 2023-08-25T08:52:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3eced9d700 1 -- 172.21.15.181:0/4105094612 learned_addr learned my addr 172.21.15.181:0/4105094612 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:52:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3ece59c700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed01004c0 0x7f3ed0196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:48.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3eced9d700 1 -- 172.21.15.181:0/4105094612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed01004c0 msgr2=0x7f3ed0196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:48.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3eced9d700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed01004c0 0x7f3ed0196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:48.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3eced9d700 1 -- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0100f30 msgr2=0x7f3ed019dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:52:48.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3eced9d700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0100f30 0x7f3ed019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:52:48.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.429+0000 7f3eced9d700 1 -- 172.21.15.181:0/4105094612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec8009d00 con 0x7f3ed00ffb00 2023-08-25T08:52:48.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.430+0000 7f3eced9d700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed00ffb00 0x7f3ed0196730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400d980 tx=0x7f3ec400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:52:48.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.430+0000 7f3eb7fff700 1 -- 172.21.15.181:0/4105094612 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4009b40 con 0x7f3ed00ffb00 2023-08-25T08:52:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.430+0000 7f3eb7fff700 1 -- 172.21.15.181:0/4105094612 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ec400b6e0 con 0x7f3ed00ffb00 2023-08-25T08:52:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.430+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed019e870 con 0x7f3ed00ffb00 2023-08-25T08:52:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.430+0000 7f3eb7fff700 1 -- 172.21.15.181:0/4105094612 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400f620 con 0x7f3ed00ffb00 2023-08-25T08:52:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.430+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed019ed70 con 0x7f3ed00ffb00 2023-08-25T08:52:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.431+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed0106e90 con 0x7f3ed00ffb00 2023-08-25T08:52:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.432+0000 7f3eb7fff700 1 -- 172.21.15.181:0/4105094612 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 134615+0+0 (secure 0 0 0) 0x7f3ec4010460 con 0x7f3ed00ffb00 2023-08-25T08:52:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.433+0000 7f3eb7fff700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3eb809ff40 0x7f3eb80a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:52:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.433+0000 7f3eb7fff700 1 -- 172.21.15.181:0/4105094612 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f3ec40c9cf0 con 0x7f3ed00ffb00 2023-08-25T08:52:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.433+0000 7f3ece59c700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3eb809ff40 0x7f3eb80a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:52:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.434+0000 7f3ece59c700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3eb809ff40 0x7f3eb80a2400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0005950 tx=0x7f3ec000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.437+0000 7f3eb7fff700 1 -- 172.21.15.181:0/4105094612 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3ec4095230 con 0x7f3ed00ffb00 2023-08-25T08:52:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:52:48.632+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ed002cf20 con 0x7f3eb809ff40 2023-08-25T08:52:49.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:48 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:52:49.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:48 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:52:49.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:48 smithi099 ceph-mon[97641]: osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] boot 2023-08-25T08:52:49.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:48 smithi099 ceph-mon[97641]: osdmap e13: 2 total, 2 up, 2 in 2023-08-25T08:52:49.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:49.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:49.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:49.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:49.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:48 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:52:49.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:48 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:52:49.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:48 smithi157 ceph-mon[130879]: osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] boot 2023-08-25T08:52:49.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:48 smithi157 ceph-mon[130879]: osdmap e13: 2 total, 2 up, 2 in 2023-08-25T08:52:49.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:48 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:49.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:48 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:49.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:48 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:49.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:48 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:49.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:48 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:52:49.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:48 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:52:49.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:48 smithi181 ceph-mon[131296]: osd.1 [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] boot 2023-08-25T08:52:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:48 smithi181 ceph-mon[131296]: osdmap e13: 2 total, 2 up, 2 in 2023-08-25T08:52:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:52:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:52:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:52:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:49.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:49 smithi181 ceph-mon[131296]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:49.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:49 smithi181 ceph-mon[131296]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:49.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:49 smithi181 ceph-mon[131296]: osdmap e14: 2 total, 2 up, 2 in 2023-08-25T08:52:50.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:49 smithi099 ceph-mon[97641]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:50.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:49 smithi099 ceph-mon[97641]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:50.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:49 smithi099 ceph-mon[97641]: osdmap e14: 2 total, 2 up, 2 in 2023-08-25T08:52:50.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:49 smithi157 ceph-mon[130879]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T08:52:50.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:49 smithi157 ceph-mon[130879]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:52:50.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:49 smithi157 ceph-mon[130879]: osdmap e14: 2 total, 2 up, 2 in 2023-08-25T08:52:52.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:51 smithi099 ceph-mon[97641]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:52.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:51 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/1561153062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]: dispatch 2023-08-25T08:52:52.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:51 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]: dispatch 2023-08-25T08:52:52.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:51 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]': finished 2023-08-25T08:52:52.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:51 smithi099 ceph-mon[97641]: osdmap e15: 3 total, 2 up, 3 in 2023-08-25T08:52:52.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:51 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:52:52.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:51 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/2404906020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:51 smithi157 ceph-mon[130879]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:51 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/1561153062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]: dispatch 2023-08-25T08:52:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:51 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]: dispatch 2023-08-25T08:52:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:51 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]': finished 2023-08-25T08:52:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:51 smithi157 ceph-mon[130879]: osdmap e15: 3 total, 2 up, 3 in 2023-08-25T08:52:52.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:51 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:52:52.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:51 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/2404906020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:52.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:51 smithi181 ceph-mon[131296]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:52.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:51 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/1561153062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]: dispatch 2023-08-25T08:52:52.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:51 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]: dispatch 2023-08-25T08:52:52.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:51 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2577a270-c667-48bf-9382-d1d5d2ec37df"}]': finished 2023-08-25T08:52:52.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:51 smithi181 ceph-mon[131296]: osdmap e15: 3 total, 2 up, 3 in 2023-08-25T08:52:52.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:51 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:52:52.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:51 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/2404906020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:52:54.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:53 smithi099 ceph-mon[97641]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:54.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:53 smithi157 ceph-mon[130879]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:54.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:53 smithi181 ceph-mon[131296]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:54.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:54 smithi181 ceph-mon[131296]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:54 smithi099 ceph-mon[97641]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:55.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:54 smithi157 ceph-mon[130879]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:57.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:57 smithi181 ceph-mon[131296]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:57.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:57 smithi099 ceph-mon[97641]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:57.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:57 smithi157 ceph-mon[130879]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:58.723 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-25T08:52:58.723 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:58.723 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:58 smithi181 ceph-mon[131296]: Deploying daemon osd.2 on smithi181 2023-08-25T08:52:58.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-25T08:52:58.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:58.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:58 smithi099 ceph-mon[97641]: Deploying daemon osd.2 on smithi181 2023-08-25T08:52:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-25T08:52:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:52:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:58 smithi157 ceph-mon[130879]: Deploying daemon osd.2 on smithi181 2023-08-25T08:52:59.710 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:52:59 smithi181 ceph-mon[131296]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:59.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:52:59 smithi099 ceph-mon[97641]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:52:59.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:52:59 smithi157 ceph-mon[130879]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:01.718 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:01 smithi181 ceph-mon[131296]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:01.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:01 smithi099 ceph-mon[97641]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:01.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:01 smithi157 ceph-mon[130879]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:02.948 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:02.948 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:02.949 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:03.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:03.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:03.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:03.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:03.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:03.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:03.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:03 smithi181 ceph-mon[131296]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:04.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:03 smithi157 ceph-mon[130879]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:03 smithi099 ceph-mon[97641]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:04.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.450+0000 7f3eb7fff700 1 -- 172.21.15.181:0/4105094612 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3ed002cf20 con 0x7f3eb809ff40 2023-08-25T08:53:04.451 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 2 on host 'smithi181' 2023-08-25T08:53:04.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3eb809ff40 msgr2=0x7f3eb80a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:04.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3eb809ff40 0x7f3eb80a2400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0005950 tx=0x7f3ec000a300 comp rx=0 tx=0).stop 2023-08-25T08:53:04.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed00ffb00 msgr2=0x7f3ed0196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:04.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed00ffb00 0x7f3ed0196730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400d980 tx=0x7f3ec400dc90 comp rx=0 tx=0).stop 2023-08-25T08:53:04.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 shutdown_connections 2023-08-25T08:53:04.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0100f30 0x7f3ed019dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:04.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed00ffb00 0x7f3ed0196730 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:04.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3eb809ff40 0x7f3eb80a2400 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:04.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 --2- 172.21.15.181:0/4105094612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed01004c0 0x7f3ed0196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:04.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 >> 172.21.15.181:0/4105094612 conn(0x7f3ed00fa5d0 msgr2=0x7f3ed0103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:53:04.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 shutdown_connections 2023-08-25T08:53:04.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:04.452+0000 7f3ed51f3700 1 -- 172.21.15.181:0/4105094612 wait complete. 2023-08-25T08:53:04.922 DEBUG:teuthology.orchestra.run.smithi181:osd.2> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.2.service 2023-08-25T08:53:04.925 INFO:tasks.cephadm:Deploying osd.3 on smithi099 with /dev/vg_nvme/lv_3... 2023-08-25T08:53:04.925 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-25T08:53:04.955 INFO:journalctl@ceph.osd.2.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:53:05.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:05.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:05.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:05.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:05.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:05.652 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:53:05.948 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-2[137088]: 2023-08-25T08:53:05.786+0000 7f8a310bc640 -1 osd.2 0 log_to_monitors true 2023-08-25T08:53:06.139 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:06 smithi099 ceph-mon[97641]: from='osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T08:53:06.139 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:06 smithi099 ceph-mon[97641]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T08:53:06.212 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:06 smithi181 ceph-mon[131296]: from='osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T08:53:06.212 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:06 smithi181 ceph-mon[131296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T08:53:06.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:06 smithi157 ceph-mon[130879]: from='osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T08:53:06.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:06 smithi157 ceph-mon[130879]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T08:53:07.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:07 smithi099 ceph-mon[97641]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-25T08:53:07.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:07 smithi099 ceph-mon[97641]: osdmap e16: 3 total, 2 up, 3 in 2023-08-25T08:53:07.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:07.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:07 smithi099 ceph-mon[97641]: from='osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:53:07.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:07 smithi099 ceph-mon[97641]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:53:07.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:07 smithi099 ceph-mon[97641]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:07 smithi157 ceph-mon[130879]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-25T08:53:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:07 smithi157 ceph-mon[130879]: osdmap e16: 3 total, 2 up, 3 in 2023-08-25T08:53:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:07 smithi157 ceph-mon[130879]: from='osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:53:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:07 smithi157 ceph-mon[130879]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:53:07.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:07 smithi157 ceph-mon[130879]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:07.492 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-2[137088]: 2023-08-25T08:53:07.029+0000 7f8a26733700 -1 osd.2 0 waiting for initial osdmap 2023-08-25T08:53:07.493 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-2[137088]: 2023-08-25T08:53:07.033+0000 7f8a1e520700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:53:07.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:07 smithi181 ceph-mon[131296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-25T08:53:07.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:07 smithi181 ceph-mon[131296]: osdmap e16: 3 total, 2 up, 3 in 2023-08-25T08:53:07.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:07.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:07 smithi181 ceph-mon[131296]: from='osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:53:07.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:07 smithi181 ceph-mon[131296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:53:07.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:07 smithi181 ceph-mon[131296]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T08:53:07.515 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-25T08:53:07.516 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-25T08:53:07.516 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-08-25T08:53:07.516 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-08-25T08:53:07.516 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0310264 s, 338 MB/s 2023-08-25T08:53:07.517 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-08-25T08:53:08.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2023-08-25T08:53:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:53:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: osdmap e17: 3 total, 2 up, 3 in 2023-08-25T08:53:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi181 2023-08-25T08:53:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi181 to 17168M 2023-08-25T08:53:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:08.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:08.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:53:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: osdmap e17: 3 total, 2 up, 3 in 2023-08-25T08:53:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:08.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi181 2023-08-25T08:53:08.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:08.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi181 to 17168M 2023-08-25T08:53:08.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:08.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:08.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:53:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: osdmap e17: 3 total, 2 up, 3 in 2023-08-25T08:53:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi181 2023-08-25T08:53:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi181 to 17168M 2023-08-25T08:53:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:53:09.288 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:09 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:53:09.288 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:09 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:53:09.288 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:09 smithi099 ceph-mon[97641]: osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] boot 2023-08-25T08:53:09.288 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:09 smithi099 ceph-mon[97641]: osdmap e18: 3 total, 3 up, 3 in 2023-08-25T08:53:09.288 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:09.289 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:09 smithi099 ceph-mon[97641]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:09.289 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-25T08:53:09.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:09 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:53:09.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:09 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:53:09.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:09 smithi157 ceph-mon[130879]: osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] boot 2023-08-25T08:53:09.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:09 smithi157 ceph-mon[130879]: osdmap e18: 3 total, 3 up, 3 in 2023-08-25T08:53:09.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:09.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:09 smithi157 ceph-mon[130879]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:09.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-25T08:53:09.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:09 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:53:09.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:09 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:53:09.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:09 smithi181 ceph-mon[131296]: osd.2 [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] boot 2023-08-25T08:53:09.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:09 smithi181 ceph-mon[131296]: osdmap e18: 3 total, 3 up, 3 in 2023-08-25T08:53:09.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:53:09.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:09 smithi181 ceph-mon[131296]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:09.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-25T08:53:10.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.216+0000 7fd9ad280700 1 -- 172.21.15.99:0/1936630433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 msgr2=0x7fd9a8100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:10.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.216+0000 7fd9ad280700 1 --2- 172.21.15.99:0/1936630433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 0x7fd9a8100dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9980099c0 tx=0x7fd998009cd0 comp rx=0 tx=0).stop 2023-08-25T08:53:10.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.216+0000 7fd9ad280700 1 -- 172.21.15.99:0/1936630433 shutdown_connections 2023-08-25T08:53:10.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.216+0000 7fd9ad280700 1 --2- 172.21.15.99:0/1936630433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8102550 0x7fd9a8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:10.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.216+0000 7fd9ad280700 1 --2- 172.21.15.99:0/1936630433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 0x7fd9a8100dd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:10.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.216+0000 7fd9ad280700 1 --2- 172.21.15.99:0/1936630433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9a8101bb0 0x7fd9a8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:10.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.216+0000 7fd9ad280700 1 -- 172.21.15.99:0/1936630433 >> 172.21.15.99:0/1936630433 conn(0x7fd9a80fbf10 msgr2=0x7fd9a80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:53:10.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.216+0000 7fd9ad280700 1 -- 172.21.15.99:0/1936630433 shutdown_connections 2023-08-25T08:53:10.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.217+0000 7fd9ad280700 1 -- 172.21.15.99:0/1936630433 wait complete. 2023-08-25T08:53:10.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9ad280700 1 Processor -- start 2023-08-25T08:53:10.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9ad280700 1 -- start start 2023-08-25T08:53:10.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9ad280700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 0x7fd9a8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:10.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9ad280700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9a8101bb0 0x7fd9a8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:10.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9ad280700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8102550 0x7fd9a819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:10.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9ad280700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81a0420 con 0x7fd9a81009b0 2023-08-25T08:53:10.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9ad280700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81a0560 con 0x7fd9a8102550 2023-08-25T08:53:10.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9ad280700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81a06f0 con 0x7fd9a8101bb0 2023-08-25T08:53:10.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 0x7fd9a8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:10.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 0x7fd9a8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60756/0 (socket says 172.21.15.99:60756) 2023-08-25T08:53:10.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9a6ffd700 1 -- 172.21.15.99:0/3792541554 learned_addr learned my addr 172.21.15.99:0/3792541554 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:53:10.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9a77fe700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8102550 0x7fd9a819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:10.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.218+0000 7fd9a67fc700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9a8101bb0 0x7fd9a8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:10.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd9a6ffd700 1 -- 172.21.15.99:0/3792541554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9a8101bb0 msgr2=0x7fd9a8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:10.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd9a6ffd700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9a8101bb0 0x7fd9a8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:10.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd9a6ffd700 1 -- 172.21.15.99:0/3792541554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8102550 msgr2=0x7fd9a819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:10.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd9a6ffd700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8102550 0x7fd9a819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:10.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd9a6ffd700 1 -- 172.21.15.99:0/3792541554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980095f0 con 0x7fd9a81009b0 2023-08-25T08:53:10.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd9a77fe700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8102550 0x7fd9a819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:53:10.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd9a6ffd700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 0x7fd9a8198890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd9980098e0 tx=0x7fd99800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:53:10.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd98ffff700 1 -- 172.21.15.99:0/3792541554 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99801d070 con 0x7fd9a81009b0 2023-08-25T08:53:10.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd98ffff700 1 -- 172.21.15.99:0/3792541554 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd998005030 con 0x7fd9a81009b0 2023-08-25T08:53:10.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a81a0970 con 0x7fd9a81009b0 2023-08-25T08:53:10.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.219+0000 7fd98ffff700 1 -- 172.21.15.99:0/3792541554 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998004a90 con 0x7fd9a81009b0 2023-08-25T08:53:10.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.220+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a81a0de0 con 0x7fd9a81009b0 2023-08-25T08:53:10.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.220+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8192ff0 con 0x7fd9a81009b0 2023-08-25T08:53:10.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.222+0000 7fd98ffff700 1 -- 172.21.15.99:0/3792541554 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) v1 ==== 134615+0+0 (secure 0 0 0) 0x7fd9980051a0 con 0x7fd9a81009b0 2023-08-25T08:53:10.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.224+0000 7fd98ffff700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd99009ff40 0x7fd9900a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:10.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.224+0000 7fd98ffff700 1 -- 172.21.15.99:0/3792541554 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fd9980caf30 con 0x7fd9a81009b0 2023-08-25T08:53:10.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.224+0000 7fd9a67fc700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd99009ff40 0x7fd9900a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:10.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.225+0000 7fd9a67fc700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd99009ff40 0x7fd9900a2400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009c60 tx=0x7fd99c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:53:10.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.227+0000 7fd98ffff700 1 -- 172.21.15.99:0/3792541554 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd998096280 con 0x7fd9a81009b0 2023-08-25T08:53:10.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-25T08:53:10.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:10 smithi099 ceph-mon[97641]: osdmap e19: 3 total, 3 up, 3 in 2023-08-25T08:53:10.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T08:53:10.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:10 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-25T08:53:10.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:10 smithi157 ceph-mon[130879]: osdmap e19: 3 total, 3 up, 3 in 2023-08-25T08:53:10.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:10 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T08:53:10.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:10.458+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9a802ce30 con 0x7fd99009ff40 2023-08-25T08:53:10.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:10 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-25T08:53:10.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:10 smithi181 ceph-mon[131296]: osdmap e19: 3 total, 3 up, 3 in 2023-08-25T08:53:10.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:10 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T08:53:11.312 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:53:10 smithi099 sudo[113553]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T08:53:11.312 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:53:10 smithi099 sudo[113553]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:11.313 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:53:10 smithi099 sudo[113553]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:11.313 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:53:10 smithi099 sudo[113553]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:11.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-25T08:53:11.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:11 smithi099 ceph-mon[97641]: osdmap e20: 3 total, 3 up, 3 in 2023-08-25T08:53:11.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:11 smithi099 ceph-mon[97641]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:11.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:11 smithi099 ceph-mon[97641]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:11.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:11.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:11.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-25T08:53:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:11 smithi157 ceph-mon[130879]: osdmap e20: 3 total, 3 up, 3 in 2023-08-25T08:53:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:11 smithi157 ceph-mon[130879]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:11 smithi157 ceph-mon[130879]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:11.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:11.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:11.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-25T08:53:11.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:11 smithi181 ceph-mon[131296]: osdmap e20: 3 total, 3 up, 3 in 2023-08-25T08:53:11.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:11 smithi181 ceph-mon[131296]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:11.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:11 smithi181 ceph-mon[131296]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:11.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:11.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:11.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:11.771 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:53:11 smithi099 sudo[113591]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T08:53:11.772 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:53:11 smithi099 sudo[113591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:11.772 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:53:11 smithi099 sudo[113591]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:11.772 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 08:53:11 smithi099 sudo[113591]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:12.047 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:12 smithi157 ceph-mon[130879]: osdmap e21: 3 total, 3 up, 3 in 2023-08-25T08:53:12.047 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:12 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:12.048 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:53:11 smithi157 sudo[138107]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T08:53:12.048 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:53:11 smithi157 sudo[138107]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:12.048 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:53:11 smithi157 sudo[138107]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:12.049 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:53:11 smithi157 sudo[138107]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:12.312 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:12 smithi181 ceph-mon[131296]: osdmap e21: 3 total, 3 up, 3 in 2023-08-25T08:53:12.312 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:12 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:12.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:12 smithi099 ceph-mon[97641]: osdmap e21: 3 total, 3 up, 3 in 2023-08-25T08:53:12.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:12 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:12.378 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:53:12 smithi157 sudo[138110]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T08:53:12.379 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:53:12 smithi157 sudo[138110]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:12.379 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:53:12 smithi157 sudo[138110]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:12.379 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 08:53:12 smithi157 sudo[138110]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:12.564 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:12 smithi181 sudo[139241]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T08:53:12.565 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:12 smithi181 sudo[139241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:12.565 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:12 smithi181 sudo[139241]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:12.565 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:12 smithi181 sudo[139241]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:12.992 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:12 smithi181 sudo[139244]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T08:53:12.992 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:12 smithi181 sudo[139244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:12.992 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:12 smithi181 sudo[139244]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:12.993 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 08:53:12 smithi181 sudo[139244]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 sudo[113710]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T08:53:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 sudo[113710]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 sudo[113710]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 sudo[113710]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:13.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:13.658+0000 7fd98ffff700 1 -- 172.21.15.99:0/3792541554 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fd998004bf0 con 0x7fd9a81009b0 2023-08-25T08:53:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 sudo[113734]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T08:53:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 sudo[113734]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 sudo[113734]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 sudo[113734]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:14.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:14.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:13 smithi099 ceph-mon[97641]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:14.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:14.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:14.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:13 smithi157 ceph-mon[130879]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:14.139 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:13 smithi181 sudo[139247]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T08:53:14.139 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:13 smithi181 sudo[139247]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:14.139 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:13 smithi181 sudo[139247]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:14.140 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:13 smithi181 sudo[139247]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:14.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 sudo[139250]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T08:53:14.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 sudo[139250]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:14.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 sudo[139250]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:14.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 sudo[139250]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:14.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:14.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:14.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:14.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 sudo[138113]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T08:53:14.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 sudo[138113]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:14.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 sudo[138113]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:14.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 sudo[138113]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:14.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:14.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4082832586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]: dispatch 2023-08-25T08:53:14.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:14.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:14.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:14.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]: dispatch 2023-08-25T08:53:14.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:14.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:14.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-25T08:53:14.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:14.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:14.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:14.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:14.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:14.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:14.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:14.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:14.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]': finished 2023-08-25T08:53:14.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: osdmap e22: 4 total, 3 up, 4 in 2023-08-25T08:53:14.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:14.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:14 smithi181 ceph-mon[131296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:15.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:15.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4082832586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]: dispatch 2023-08-25T08:53:15.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:15.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:15.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:15.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]: dispatch 2023-08-25T08:53:15.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:15.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:15.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-25T08:53:15.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:15.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:15.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:15.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:15.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:15.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:15.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:15.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:15.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]': finished 2023-08-25T08:53:15.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: osdmap e22: 4 total, 3 up, 4 in 2023-08-25T08:53:15.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:15.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:14 smithi099 ceph-mon[97641]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:15.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 sudo[138116]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T08:53:15.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 sudo[138116]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T08:53:15.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 sudo[138116]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T08:53:15.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 sudo[138116]: pam_unix(sudo:session): session closed for user root 2023-08-25T08:53:15.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:15.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4082832586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]: dispatch 2023-08-25T08:53:15.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:15.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:15.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:15.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]: dispatch 2023-08-25T08:53:15.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:15.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:15.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-25T08:53:15.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:15.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:15.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:15.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:15.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T08:53:15.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:53:15.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:53:15.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:53:15.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5e672d-2f3e-4afd-8207-bcc4f2aa6530"}]': finished 2023-08-25T08:53:15.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: osdmap e22: 4 total, 3 up, 4 in 2023-08-25T08:53:15.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:15.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:14 smithi157 ceph-mon[130879]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T08:53:15.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:15 smithi181 ceph-mon[131296]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:15.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:15 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3640625947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:53:16.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:15 smithi099 ceph-mon[97641]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:16.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:15 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3640625947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:53:16.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:15 smithi157 ceph-mon[130879]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:16.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:15 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3640625947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:53:17.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:17 smithi181 ceph-mon[131296]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:18.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:17 smithi099 ceph-mon[97641]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:18.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:17 smithi157 ceph-mon[130879]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:19.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:19 smithi181 ceph-mon[131296]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:20.061 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:19 smithi099 ceph-mon[97641]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:20.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:19 smithi157 ceph-mon[130879]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:21.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:21 smithi099 ceph-mon[97641]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2023-08-25T08:53:21.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:21 smithi181 ceph-mon[131296]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2023-08-25T08:53:22.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:21 smithi157 ceph-mon[130879]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2023-08-25T08:53:22.928 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:22 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-25T08:53:22.928 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:22 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:22.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:22 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-25T08:53:22.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:22 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:23.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:22 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-25T08:53:23.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:22 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:23.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:23 smithi099 ceph-mon[97641]: Deploying daemon osd.3 on smithi099 2023-08-25T08:53:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:23 smithi099 ceph-mon[97641]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-08-25T08:53:23.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:23 smithi181 ceph-mon[131296]: Deploying daemon osd.3 on smithi099 2023-08-25T08:53:23.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:23 smithi181 ceph-mon[131296]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-08-25T08:53:24.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:23 smithi157 ceph-mon[130879]: Deploying daemon osd.3 on smithi099 2023-08-25T08:53:24.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:23 smithi157 ceph-mon[130879]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-08-25T08:53:25.107 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:24 smithi099 ceph-mon[97641]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1000 B/s rd, 26 KiB/s wr, 2 op/s 2023-08-25T08:53:25.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:24 smithi181 ceph-mon[131296]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1000 B/s rd, 26 KiB/s wr, 2 op/s 2023-08-25T08:53:25.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:24 smithi157 ceph-mon[130879]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1000 B/s rd, 26 KiB/s wr, 2 op/s 2023-08-25T08:53:27.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:27 smithi099 ceph-mon[97641]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:27.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:27 smithi181 ceph-mon[131296]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:27 smithi157 ceph-mon[130879]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:29.342 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:29 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:29.342 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:29 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:29.342 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:29 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:29.342 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:29 smithi099 ceph-mon[97641]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:29.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:29 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:29.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:29 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:29.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:29 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:29.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:29 smithi157 ceph-mon[130879]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:29.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:29 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:29.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:29 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:29.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:29 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:29.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:29 smithi181 ceph-mon[131296]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:31.425 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 3 on host 'smithi099' 2023-08-25T08:53:31.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.423+0000 7fd98ffff700 1 -- 172.21.15.99:0/3792541554 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd9a802ce30 con 0x7fd99009ff40 2023-08-25T08:53:31.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.426+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd99009ff40 msgr2=0x7fd9900a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:31.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.426+0000 7fd9ad280700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd99009ff40 0x7fd9900a2400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009c60 tx=0x7fd99c009260 comp rx=0 tx=0).stop 2023-08-25T08:53:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 msgr2=0x7fd9a8198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 0x7fd9a8198890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd9980098e0 tx=0x7fd99800f690 comp rx=0 tx=0).stop 2023-08-25T08:53:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 shutdown_connections 2023-08-25T08:53:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd99009ff40 0x7fd9900a2400 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8102550 0x7fd9a819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a81009b0 0x7fd9a8198890 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 --2- 172.21.15.99:0/3792541554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9a8101bb0 0x7fd9a8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 >> 172.21.15.99:0/3792541554 conn(0x7fd9a80fbf10 msgr2=0x7fd9a8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:53:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 shutdown_connections 2023-08-25T08:53:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:53:31.427+0000 7fd9ad280700 1 -- 172.21.15.99:0/3792541554 wait complete. 2023-08-25T08:53:31.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:31.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:31.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:31.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:31.976 DEBUG:teuthology.orchestra.run.smithi099:osd.3> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.3.service 2023-08-25T08:53:31.980 INFO:tasks.cephadm:Deploying osd.4 on smithi157 with /dev/vg_nvme/lv_3... 2023-08-25T08:53:31.981 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-25T08:53:31.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:31.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:31.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:31.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:31.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:32.014 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T08:53:32.014 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:32.014 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:32.014 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:32.014 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:32.014 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:32.015 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:32.015 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:32.015 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:32.062 INFO:journalctl@ceph.osd.3.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:53:32.312 INFO:journalctl@ceph.osd.3.smithi099.stdout:Aug 25 08:53:32 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-3[116007]: 2023-08-25T08:53:32.076+0000 7fde120df640 -1 osd.3 0 log_to_monitors true 2023-08-25T08:53:32.547 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T08:53:32.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:32 smithi099 ceph-mon[97641]: from='osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T08:53:32.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:32 smithi099 ceph-mon[97641]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T08:53:32.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:32 smithi181 ceph-mon[131296]: from='osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T08:53:32.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:32 smithi181 ceph-mon[131296]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T08:53:33.068 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:32 smithi157 ceph-mon[130879]: from='osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T08:53:33.068 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:32 smithi157 ceph-mon[130879]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T08:53:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:33 smithi099 ceph-mon[97641]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:33 smithi099 ceph-mon[97641]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-25T08:53:33.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:33 smithi099 ceph-mon[97641]: osdmap e23: 4 total, 3 up, 4 in 2023-08-25T08:53:33.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:33 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:33.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:33 smithi099 ceph-mon[97641]: from='osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:53:33.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:33 smithi099 ceph-mon[97641]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:53:33.815 INFO:journalctl@ceph.osd.3.smithi099.stdout:Aug 25 08:53:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-3[116007]: 2023-08-25T08:53:33.750+0000 7fde07756700 -1 osd.3 0 waiting for initial osdmap 2023-08-25T08:53:33.815 INFO:journalctl@ceph.osd.3.smithi099.stdout:Aug 25 08:53:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-3[116007]: 2023-08-25T08:53:33.759+0000 7fddff543700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:53:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:33 smithi157 ceph-mon[130879]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:33 smithi157 ceph-mon[130879]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-25T08:53:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:33 smithi157 ceph-mon[130879]: osdmap e23: 4 total, 3 up, 4 in 2023-08-25T08:53:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:33 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:33 smithi157 ceph-mon[130879]: from='osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:53:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:33 smithi157 ceph-mon[130879]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:53:34.087 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-25T08:53:34.088 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-25T08:53:34.088 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2023-08-25T08:53:34.088 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2023-08-25T08:53:34.088 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0083064 s, 1.3 GB/s 2023-08-25T08:53:34.088 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2023-08-25T08:53:34.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:33 smithi181 ceph-mon[131296]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:34.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:33 smithi181 ceph-mon[131296]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-25T08:53:34.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:33 smithi181 ceph-mon[131296]: osdmap e23: 4 total, 3 up, 4 in 2023-08-25T08:53:34.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:33 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:34.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:33 smithi181 ceph-mon[131296]: from='osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:53:34.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:33 smithi181 ceph-mon[131296]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:53:34.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2023-08-25T08:53:35.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:34 smithi099 ceph-mon[97641]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:53:35.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:34 smithi099 ceph-mon[97641]: osdmap e24: 4 total, 3 up, 4 in 2023-08-25T08:53:35.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:34 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:34 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:34 smithi099 ceph-mon[97641]: osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] boot 2023-08-25T08:53:35.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:34 smithi099 ceph-mon[97641]: osdmap e25: 4 total, 4 up, 4 in 2023-08-25T08:53:35.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:34 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.092 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T08:53:35.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:34 smithi157 ceph-mon[130879]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:53:35.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:34 smithi157 ceph-mon[130879]: osdmap e24: 4 total, 3 up, 4 in 2023-08-25T08:53:35.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:34 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:34 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:34 smithi157 ceph-mon[130879]: osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] boot 2023-08-25T08:53:35.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:34 smithi157 ceph-mon[130879]: osdmap e25: 4 total, 4 up, 4 in 2023-08-25T08:53:35.132 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:34 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:34 smithi181 ceph-mon[131296]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:53:35.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:34 smithi181 ceph-mon[131296]: osdmap e24: 4 total, 3 up, 4 in 2023-08-25T08:53:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:34 smithi181 ceph-mon[131296]: osd.3 [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] boot 2023-08-25T08:53:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:34 smithi181 ceph-mon[131296]: osdmap e25: 4 total, 4 up, 4 in 2023-08-25T08:53:35.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:34 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:53:35.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:53:35.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:53:35.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:35.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi099 2023-08-25T08:53:35.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi099 to 8633M 2023-08-25T08:53:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:35.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:35.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:35 smithi157 ceph-mon[130879]: osdmap e26: 4 total, 4 up, 4 in 2023-08-25T08:53:36.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:53:36.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:53:36.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:36.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi099 2023-08-25T08:53:36.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi099 to 8633M 2023-08-25T08:53:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:36.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:36.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:35 smithi099 ceph-mon[97641]: osdmap e26: 4 total, 4 up, 4 in 2023-08-25T08:53:36.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:53:36.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:53:36.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T08:53:36.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi099 2023-08-25T08:53:36.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:36.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:36.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:36.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:36.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi099 to 8633M 2023-08-25T08:53:36.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:36.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:36.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:36.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:36.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:35 smithi181 ceph-mon[131296]: osdmap e26: 4 total, 4 up, 4 in 2023-08-25T08:53:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/4033354907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 msgr2=0x7f8cf4107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/4033354907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 0x7f8cf4107d00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00b0d0 tx=0x7f8cec00b3e0 comp rx=0 tx=0).stop 2023-08-25T08:53:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/4033354907 shutdown_connections 2023-08-25T08:53:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/4033354907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 0x7f8cf4107d00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/4033354907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cf40ff8b0 0x7f8cf40ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/4033354907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cf4100ab0 0x7f8cf4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/4033354907 >> 172.21.15.157:0/4033354907 conn(0x7f8cf40fae30 msgr2=0x7f8cf40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:53:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/4033354907 shutdown_connections 2023-08-25T08:53:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.304+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/4033354907 wait complete. 2023-08-25T08:53:36.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.305+0000 7f8cf8dbd700 1 Processor -- start 2023-08-25T08:53:36.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.305+0000 7f8cf8dbd700 1 -- start start 2023-08-25T08:53:36.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.305+0000 7f8cf8dbd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cf40ff8b0 0x7f8cf4103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:36.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.305+0000 7f8cf8dbd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cf4100ab0 0x7f8cf41021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:36.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.305+0000 7f8cf8dbd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 0x7f8cf41026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:36.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.305+0000 7f8cf8dbd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4102e60 con 0x7f8cf40ff8b0 2023-08-25T08:53:36.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.305+0000 7f8cf8dbd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4102fd0 con 0x7f8cf4101450 2023-08-25T08:53:36.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.305+0000 7f8cf8dbd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41a0930 con 0x7f8cf4100ab0 2023-08-25T08:53:36.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 0x7f8cf41026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:36.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 0x7f8cf41026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57692/0 (socket says 172.21.15.157:57692) 2023-08-25T08:53:36.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 -- 172.21.15.157:0/1544029442 learned_addr learned my addr 172.21.15.157:0/1544029442 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-08-25T08:53:36.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf1d9b700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cf4100ab0 0x7f8cf41021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:36.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf259c700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cf40ff8b0 0x7f8cf4103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:36.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 -- 172.21.15.157:0/1544029442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cf4100ab0 msgr2=0x7f8cf41021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:36.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cf4100ab0 0x7f8cf41021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:36.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 -- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cf40ff8b0 msgr2=0x7f8cf4103b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:36.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cf40ff8b0 0x7f8cf4103b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:36.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 -- 172.21.15.157:0/1544029442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec009d00 con 0x7f8cf4101450 2023-08-25T08:53:36.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf259c700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cf40ff8b0 0x7f8cf4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:53:36.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf2d9d700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 0x7f8cf41026e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cec007a00 tx=0x7f8cec009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:53:36.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8ce37fe700 1 -- 172.21.15.157:0/1544029442 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00e040 con 0x7f8cf4101450 2023-08-25T08:53:36.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8ce37fe700 1 -- 172.21.15.157:0/1544029442 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cec003e10 con 0x7f8cf4101450 2023-08-25T08:53:36.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf41a0bb0 con 0x7f8cf4101450 2023-08-25T08:53:36.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8ce37fe700 1 -- 172.21.15.157:0/1544029442 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec005750 con 0x7f8cf4101450 2023-08-25T08:53:36.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.306+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf41a0fc0 con 0x7f8cf4101450 2023-08-25T08:53:36.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.307+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf41079a0 con 0x7f8cf4101450 2023-08-25T08:53:36.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.308+0000 7f8ce37fe700 1 -- 172.21.15.157:0/1544029442 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f8cec012cb0 con 0x7f8cf4101450 2023-08-25T08:53:36.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.309+0000 7f8ce37fe700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8cdc09ff50 0x7f8cdc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:36.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.309+0000 7f8ce37fe700 1 -- 172.21.15.157:0/1544029442 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f8cec0cc0d0 con 0x7f8cf4101450 2023-08-25T08:53:36.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.309+0000 7f8cf259c700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8cdc09ff50 0x7f8cdc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:36.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.310+0000 7f8cf259c700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8cdc09ff50 0x7f8cdc0a2410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4100910 tx=0x7f8ce8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:53:36.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.313+0000 7f8ce37fe700 1 -- 172.21.15.157:0/1544029442 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8cec0970f0 con 0x7f8cf4101450 2023-08-25T08:53:36.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:36.510+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cf402cdc0 con 0x7f8cdc09ff50 2023-08-25T08:53:37.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:36 smithi181 ceph-mon[131296]: osdmap e27: 4 total, 4 up, 4 in 2023-08-25T08:53:37.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:36 smithi181 ceph-mon[131296]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:37.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:36 smithi181 ceph-mon[131296]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:37.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:36 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:37.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:36 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:37.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:36 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:37.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:36 smithi157 ceph-mon[130879]: osdmap e27: 4 total, 4 up, 4 in 2023-08-25T08:53:37.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:36 smithi157 ceph-mon[130879]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:37.282 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:36 smithi157 ceph-mon[130879]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:37.282 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:36 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:37.282 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:36 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:37.282 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:36 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:37.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:36 smithi099 ceph-mon[97641]: osdmap e27: 4 total, 4 up, 4 in 2023-08-25T08:53:37.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:36 smithi099 ceph-mon[97641]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:37.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:36 smithi099 ceph-mon[97641]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:37.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:36 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:37.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:36 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:37.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:36 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:38.145 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:37 smithi157 ceph-mon[130879]: osdmap e28: 4 total, 4 up, 4 in 2023-08-25T08:53:38.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:37 smithi181 ceph-mon[131296]: osdmap e28: 4 total, 4 up, 4 in 2023-08-25T08:53:38.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:37 smithi099 ceph-mon[97641]: osdmap e28: 4 total, 4 up, 4 in 2023-08-25T08:53:39.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:38 smithi181 ceph-mon[131296]: osdmap e29: 4 total, 4 up, 4 in 2023-08-25T08:53:39.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:38 smithi181 ceph-mon[131296]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:39.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:38 smithi181 ceph-mon[131296]: from='client.? 172.21.15.157:0/2453897332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]: dispatch 2023-08-25T08:53:39.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:38 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]: dispatch 2023-08-25T08:53:39.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:38 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]': finished 2023-08-25T08:53:39.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:38 smithi181 ceph-mon[131296]: osdmap e30: 5 total, 4 up, 5 in 2023-08-25T08:53:39.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:38 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:39.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:38 smithi099 ceph-mon[97641]: osdmap e29: 4 total, 4 up, 4 in 2023-08-25T08:53:39.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:38 smithi099 ceph-mon[97641]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:39.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:38 smithi099 ceph-mon[97641]: from='client.? 172.21.15.157:0/2453897332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]: dispatch 2023-08-25T08:53:39.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:38 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]: dispatch 2023-08-25T08:53:39.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:38 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]': finished 2023-08-25T08:53:39.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:38 smithi099 ceph-mon[97641]: osdmap e30: 5 total, 4 up, 5 in 2023-08-25T08:53:39.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:38 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:39.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:38 smithi157 ceph-mon[130879]: osdmap e29: 4 total, 4 up, 4 in 2023-08-25T08:53:39.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:38 smithi157 ceph-mon[130879]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:39.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:38 smithi157 ceph-mon[130879]: from='client.? 172.21.15.157:0/2453897332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]: dispatch 2023-08-25T08:53:39.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:38 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]: dispatch 2023-08-25T08:53:39.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:38 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91518e8f-f259-4741-9f5b-17b1e1d1b662"}]': finished 2023-08-25T08:53:39.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:38 smithi157 ceph-mon[130879]: osdmap e30: 5 total, 4 up, 5 in 2023-08-25T08:53:39.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:38 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:40.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:39 smithi181 ceph-mon[131296]: from='client.? 172.21.15.157:0/3978472088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:53:40.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:39 smithi099 ceph-mon[97641]: from='client.? 172.21.15.157:0/3978472088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:53:40.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:39 smithi157 ceph-mon[130879]: from='client.? 172.21.15.157:0/3978472088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:53:41.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:40 smithi099 ceph-mon[97641]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-08-25T08:53:41.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:40 smithi157 ceph-mon[130879]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-08-25T08:53:41.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:40 smithi181 ceph-mon[131296]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-08-25T08:53:43.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:43 smithi181 ceph-mon[131296]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:53:43.764 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:43 smithi157 ceph-mon[130879]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:53:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:43 smithi099 ceph-mon[97641]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:53:45.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:44 smithi181 ceph-mon[131296]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-08-25T08:53:45.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:44 smithi099 ceph-mon[97641]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-08-25T08:53:45.363 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:44 smithi157 ceph-mon[130879]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-08-25T08:53:46.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-25T08:53:46.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:45 smithi181 ceph-mon[131296]: Deploying daemon osd.4 on smithi157 2023-08-25T08:53:46.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-25T08:53:46.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:46.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:45 smithi099 ceph-mon[97641]: Deploying daemon osd.4 on smithi157 2023-08-25T08:53:46.323 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-25T08:53:46.324 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:46.324 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:45 smithi157 ceph-mon[130879]: Deploying daemon osd.4 on smithi157 2023-08-25T08:53:47.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:46 smithi157 ceph-mon[130879]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-08-25T08:53:47.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:46 smithi181 ceph-mon[131296]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-08-25T08:53:47.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:46 smithi099 ceph-mon[97641]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-08-25T08:53:49.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:49 smithi181 ceph-mon[131296]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-25T08:53:49.772 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:49 smithi157 ceph-mon[130879]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-25T08:53:49.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:49 smithi099 ceph-mon[97641]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-25T08:53:51.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:51.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:51.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:51.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:51.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:51.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:51.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:51.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:52.113 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:51 smithi157 ceph-mon[130879]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:52.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:51 smithi181 ceph-mon[131296]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:52.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:51 smithi099 ceph-mon[97641]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.529+0000 7f8ce37fe700 1 -- 172.21.15.157:0/1544029442 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8cf402cdc0 con 0x7f8cdc09ff50 2023-08-25T08:53:52.530 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 4 on host 'smithi157' 2023-08-25T08:53:52.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.531+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8cdc09ff50 msgr2=0x7f8cdc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8cdc09ff50 0x7f8cdc0a2410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4100910 tx=0x7f8ce8008040 comp rx=0 tx=0).stop 2023-08-25T08:53:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 msgr2=0x7f8cf41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 0x7f8cf41026e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cec007a00 tx=0x7f8cec009570 comp rx=0 tx=0).stop 2023-08-25T08:53:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 shutdown_connections 2023-08-25T08:53:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8cdc09ff50 0x7f8cdc0a2410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4101450 0x7f8cf41026e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cf40ff8b0 0x7f8cf4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 --2- 172.21.15.157:0/1544029442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cf4100ab0 0x7f8cf41021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 >> 172.21.15.157:0/1544029442 conn(0x7f8cf40fae30 msgr2=0x7f8cf40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:53:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 shutdown_connections 2023-08-25T08:53:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:53:52.532+0000 7f8cf8dbd700 1 -- 172.21.15.157:0/1544029442 wait complete. 2023-08-25T08:53:52.982 DEBUG:teuthology.orchestra.run.smithi157:osd.4> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.4.service 2023-08-25T08:53:52.984 INFO:tasks.cephadm:Deploying osd.5 on smithi181 with /dev/vg_nvme/lv_3... 2023-08-25T08:53:52.985 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-25T08:53:53.016 INFO:journalctl@ceph.osd.4.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:53:53.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:53.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:53.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:53.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:53.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:53.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:53.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:53.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:53.546 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:53:53.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:53.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:53:54.112 INFO:journalctl@ceph.osd.4.smithi157.stdout:Aug 25 08:53:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-4[141296]: 2023-08-25T08:53:53.860+0000 7f47eaa73640 -1 osd.4 0 log_to_monitors true 2023-08-25T08:53:54.378 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:54 smithi181 ceph-mon[131296]: from='osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T08:53:54.378 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:54 smithi181 ceph-mon[131296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T08:53:54.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:54 smithi157 ceph-mon[130879]: from='osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T08:53:54.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:54 smithi157 ceph-mon[130879]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T08:53:54.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:54 smithi099 ceph-mon[97641]: from='osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T08:53:54.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:54 smithi099 ceph-mon[97641]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T08:53:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-25T08:53:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-25T08:53:55.020 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-08-25T08:53:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-08-25T08:53:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110622 s, 948 MB/s 2023-08-25T08:53:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-08-25T08:53:55.379 INFO:journalctl@ceph.osd.4.smithi157.stdout:Aug 25 08:53:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-4[141296]: 2023-08-25T08:53:55.125+0000 7f47e00ea700 -1 osd.4 0 waiting for initial osdmap 2023-08-25T08:53:55.379 INFO:journalctl@ceph.osd.4.smithi157.stdout:Aug 25 08:53:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-4[141296]: 2023-08-25T08:53:55.131+0000 7f47d86d8700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:53:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:55 smithi157 ceph-mon[130879]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-25T08:53:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:55 smithi157 ceph-mon[130879]: osdmap e31: 5 total, 4 up, 5 in 2023-08-25T08:53:55.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:55.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:55 smithi157 ceph-mon[130879]: from='osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:53:55.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:55 smithi157 ceph-mon[130879]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:53:55.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:55 smithi157 ceph-mon[130879]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:55.383 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:55 smithi181 ceph-mon[131296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-25T08:53:55.384 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:55 smithi181 ceph-mon[131296]: osdmap e31: 5 total, 4 up, 5 in 2023-08-25T08:53:55.384 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:55.384 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:55 smithi181 ceph-mon[131296]: from='osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:53:55.384 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:55 smithi181 ceph-mon[131296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:53:55.384 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:55 smithi181 ceph-mon[131296]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:55.401 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2023-08-25T08:53:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:55 smithi099 ceph-mon[97641]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-25T08:53:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:55 smithi099 ceph-mon[97641]: osdmap e31: 5 total, 4 up, 5 in 2023-08-25T08:53:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:55.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:55 smithi099 ceph-mon[97641]: from='osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:53:55.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:55 smithi099 ceph-mon[97641]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:53:55.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:55 smithi099 ceph-mon[97641]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:55.959 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:53:56.473 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:53:56.473 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: osdmap e32: 5 total, 4 up, 5 in 2023-08-25T08:53:56.474 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:56.474 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:56.474 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi157 2023-08-25T08:53:56.474 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.474 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.475 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:56.475 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:56.475 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi157 to 8584M 2023-08-25T08:53:56.475 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.476 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:56.476 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:56.476 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:53:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: osdmap e32: 5 total, 4 up, 5 in 2023-08-25T08:53:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi157 2023-08-25T08:53:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi157 to 8584M 2023-08-25T08:53:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:56.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:56.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:53:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: osdmap e32: 5 total, 4 up, 5 in 2023-08-25T08:53:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi157 2023-08-25T08:53:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:53:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi157 to 8584M 2023-08-25T08:53:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:56.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:56.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:53:56.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:53:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.162+0000 7f18b76ff700 1 -- 172.21.15.181:0/2800988215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 msgr2=0x7f18b0107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.162+0000 7f18b76ff700 1 --2- 172.21.15.181:0/2800988215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 0x7f18b0107d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f18a800b0a0 tx=0x7f18a800b3b0 comp rx=0 tx=0).stop 2023-08-25T08:53:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.162+0000 7f18b76ff700 1 -- 172.21.15.181:0/2800988215 shutdown_connections 2023-08-25T08:53:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.162+0000 7f18b76ff700 1 --2- 172.21.15.181:0/2800988215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 0x7f18b0107d00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.162+0000 7f18b76ff700 1 --2- 172.21.15.181:0/2800988215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b00ff8b0 0x7f18b00ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.162+0000 7f18b76ff700 1 --2- 172.21.15.181:0/2800988215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b0100ab0 0x7f18b0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.162+0000 7f18b76ff700 1 -- 172.21.15.181:0/2800988215 >> 172.21.15.181:0/2800988215 conn(0x7f18b00fae30 msgr2=0x7f18b00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:53:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.163+0000 7f18b76ff700 1 -- 172.21.15.181:0/2800988215 shutdown_connections 2023-08-25T08:53:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.163+0000 7f18b76ff700 1 -- 172.21.15.181:0/2800988215 wait complete. 2023-08-25T08:53:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.163+0000 7f18b76ff700 1 Processor -- start 2023-08-25T08:53:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.163+0000 7f18b76ff700 1 -- start start 2023-08-25T08:53:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.163+0000 7f18b76ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00ff8b0 0x7f18b01955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b76ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0100ab0 0x7f18b0195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b76ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 0x7f18b019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:57.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b76ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019d130 con 0x7f18b0100ab0 2023-08-25T08:53:57.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b76ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019d270 con 0x7f18b00ff8b0 2023-08-25T08:53:57.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b76ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019d400 con 0x7f18b0101450 2023-08-25T08:53:57.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 0x7f18b019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:57.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 0x7f18b019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43196/0 (socket says 172.21.15.181:43196) 2023-08-25T08:53:57.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 -- 172.21.15.181:0/222292009 learned_addr learned my addr 172.21.15.181:0/222292009 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:53:57.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b549b700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00ff8b0 0x7f18b01955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:57.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 -- 172.21.15.181:0/222292009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00ff8b0 msgr2=0x7f18b01955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:57.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00ff8b0 0x7f18b01955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:57.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 -- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0100ab0 msgr2=0x7f18b0195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:53:57.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0100ab0 0x7f18b0195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:53:57.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 -- 172.21.15.181:0/222292009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8009d00 con 0x7f18b0101450 2023-08-25T08:53:57.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b5c9c700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 0x7f18b019cb60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f18a8007ae0 tx=0x7f18a8004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:53:57.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18a67fc700 1 -- 172.21.15.181:0/222292009 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800e040 con 0x7f18b0101450 2023-08-25T08:53:57.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18a67fc700 1 -- 172.21.15.181:0/222292009 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18a801db50 con 0x7f18b0101450 2023-08-25T08:53:57.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b019d680 con 0x7f18b0101450 2023-08-25T08:53:57.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18a67fc700 1 -- 172.21.15.181:0/222292009 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a801bc00 con 0x7f18b0101450 2023-08-25T08:53:57.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.164+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b019db40 con 0x7f18b0101450 2023-08-25T08:53:57.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.165+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b010b0c0 con 0x7f18b0101450 2023-08-25T08:53:57.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.166+0000 7f18a67fc700 1 -- 172.21.15.181:0/222292009 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f18a801dcc0 con 0x7f18b0101450 2023-08-25T08:53:57.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.167+0000 7f18a67fc700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f189c0a0050 0x7f189c0a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:53:57.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.167+0000 7f18a67fc700 1 -- 172.21.15.181:0/222292009 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f18a80cbd70 con 0x7f18b0101450 2023-08-25T08:53:57.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.167+0000 7f18b549b700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f189c0a0050 0x7f189c0a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:53:57.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.167+0000 7f18b549b700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f189c0a0050 0x7f189c0a2510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18b0100910 tx=0x7f18ac00a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:53:57.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.171+0000 7f18a67fc700 1 -- 172.21.15.181:0/222292009 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f18a8096b60 con 0x7f18b0101450 2023-08-25T08:53:57.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:57 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:53:57.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:57 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:53:57.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:57 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:57.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:57 smithi181 ceph-mon[131296]: osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] boot 2023-08-25T08:53:57.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:57 smithi181 ceph-mon[131296]: osdmap e33: 5 total, 5 up, 5 in 2023-08-25T08:53:57.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:57 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:57.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:57 smithi181 ceph-mon[131296]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:57.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:53:57.367+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f18b0060d60 con 0x7f189c0a0050 2023-08-25T08:53:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:57 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:53:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:57 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:53:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:57 smithi099 ceph-mon[97641]: osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] boot 2023-08-25T08:53:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:57 smithi099 ceph-mon[97641]: osdmap e33: 5 total, 5 up, 5 in 2023-08-25T08:53:57.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:57.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:57 smithi099 ceph-mon[97641]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:57.628 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:57 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:57 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:57 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:57 smithi157 ceph-mon[130879]: osd.4 [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] boot 2023-08-25T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:57 smithi157 ceph-mon[130879]: osdmap e33: 5 total, 5 up, 5 in 2023-08-25T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:57 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:57 smithi157 ceph-mon[130879]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T08:53:58.456 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:58 smithi181 ceph-mon[131296]: osdmap e34: 5 total, 5 up, 5 in 2023-08-25T08:53:58.456 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:58 smithi181 ceph-mon[131296]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:58.456 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:58.457 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:58.457 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:58.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:58 smithi099 ceph-mon[97641]: osdmap e34: 5 total, 5 up, 5 in 2023-08-25T08:53:58.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:58 smithi099 ceph-mon[97641]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:58.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:58.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:58.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:58.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:58 smithi157 ceph-mon[130879]: osdmap e34: 5 total, 5 up, 5 in 2023-08-25T08:53:58.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:58 smithi157 ceph-mon[130879]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:53:58.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:53:58.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:53:58.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:53:59.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:53:59 smithi181 ceph-mon[131296]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:53:59.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:53:59 smithi099 ceph-mon[97641]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:53:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:53:59 smithi157 ceph-mon[130879]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:00.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:00 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/2986977623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]: dispatch 2023-08-25T08:54:00.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:00 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]: dispatch 2023-08-25T08:54:00.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:00 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]': finished 2023-08-25T08:54:00.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:00 smithi181 ceph-mon[131296]: osdmap e35: 6 total, 5 up, 6 in 2023-08-25T08:54:00.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:00.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:00 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/2986977623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]: dispatch 2023-08-25T08:54:00.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:00 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]: dispatch 2023-08-25T08:54:00.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:00 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]': finished 2023-08-25T08:54:00.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:00 smithi099 ceph-mon[97641]: osdmap e35: 6 total, 5 up, 6 in 2023-08-25T08:54:00.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:00.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:00 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/2986977623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]: dispatch 2023-08-25T08:54:00.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:00 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]: dispatch 2023-08-25T08:54:00.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:00 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c17bcccb-4491-4d2c-8864-f2e744438d5e"}]': finished 2023-08-25T08:54:00.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:00 smithi157 ceph-mon[130879]: osdmap e35: 6 total, 5 up, 6 in 2023-08-25T08:54:00.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:01.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:01 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/1302631180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:01.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:01 smithi181 ceph-mon[131296]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:01.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:01 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/1302631180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:01.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:01 smithi099 ceph-mon[97641]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:01 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/1302631180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:01 smithi157 ceph-mon[130879]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:03 smithi181 ceph-mon[131296]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:03.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:03 smithi099 ceph-mon[97641]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:03 smithi157 ceph-mon[130879]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:05.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:04 smithi181 ceph-mon[131296]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:04 smithi099 ceph-mon[97641]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:04 smithi157 ceph-mon[130879]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:06.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:06 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-25T08:54:06.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:06 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:06.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:06 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-25T08:54:06.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:06 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:06.628 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:06 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-25T08:54:06.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:06 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:07.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:07 smithi181 ceph-mon[131296]: Deploying daemon osd.5 on smithi181 2023-08-25T08:54:07.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:07 smithi181 ceph-mon[131296]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:07.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:07 smithi099 ceph-mon[97641]: Deploying daemon osd.5 on smithi181 2023-08-25T08:54:07.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:07 smithi099 ceph-mon[97641]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:07 smithi157 ceph-mon[130879]: Deploying daemon osd.5 on smithi181 2023-08-25T08:54:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:07 smithi157 ceph-mon[130879]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:09 smithi181 ceph-mon[131296]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:09.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:09 smithi099 ceph-mon[97641]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:09 smithi157 ceph-mon[130879]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:11.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:11 smithi181 ceph-mon[131296]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:11.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:11.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:11 smithi099 ceph-mon[97641]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:11.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:11 smithi157 ceph-mon[130879]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:11.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:11.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:11.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:13.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.179+0000 7f18a67fc700 1 -- 172.21.15.181:0/222292009 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f18b0060d60 con 0x7f189c0a0050 2023-08-25T08:54:13.180 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 5 on host 'smithi181' 2023-08-25T08:54:13.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.181+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f189c0a0050 msgr2=0x7f189c0a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:13.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.181+0000 7f18b76ff700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f189c0a0050 0x7f189c0a2510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18b0100910 tx=0x7f18ac00a3b0 comp rx=0 tx=0).stop 2023-08-25T08:54:13.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.181+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 msgr2=0x7f18b019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:13.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.181+0000 7f18b76ff700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 0x7f18b019cb60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f18a8007ae0 tx=0x7f18a8004d00 comp rx=0 tx=0).stop 2023-08-25T08:54:13.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.181+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 shutdown_connections 2023-08-25T08:54:13.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.181+0000 7f18b76ff700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b0101450 0x7f18b019cb60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:13.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.182+0000 7f18b76ff700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f189c0a0050 0x7f189c0a2510 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:13.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.182+0000 7f18b76ff700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00ff8b0 0x7f18b01955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:13.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.182+0000 7f18b76ff700 1 --2- 172.21.15.181:0/222292009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0100ab0 0x7f18b0195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:13.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.182+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 >> 172.21.15.181:0/222292009 conn(0x7f18b00fae30 msgr2=0x7f18b00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:54:13.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.182+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 shutdown_connections 2023-08-25T08:54:13.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:54:13.182+0000 7f18b76ff700 1 -- 172.21.15.181:0/222292009 wait complete. 2023-08-25T08:54:13.544 DEBUG:teuthology.orchestra.run.smithi181:osd.5> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.5.service 2023-08-25T08:54:13.546 INFO:tasks.cephadm:Deploying osd.6 on smithi099 with /dev/vg_nvme/lv_2... 2023-08-25T08:54:13.546 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-25T08:54:13.597 INFO:journalctl@ceph.osd.5.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:54:14.059 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:14.060 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.060 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.060 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:14.060 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:14.060 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.060 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:14.061 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.061 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:14.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:14.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:14.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:14.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:14.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:14.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:14.285 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:54:14.742 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 08:54:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T08:54:14.366+0000 7fb058833640 -1 osd.5 0 log_to_monitors true 2023-08-25T08:54:15.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:14 smithi181 ceph-mon[131296]: from='osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T08:54:15.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:14 smithi181 ceph-mon[131296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T08:54:15.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:14 smithi181 ceph-mon[131296]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:15.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:14 smithi099 ceph-mon[97641]: from='osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T08:54:15.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:14 smithi099 ceph-mon[97641]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T08:54:15.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:14 smithi099 ceph-mon[97641]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:14 smithi157 ceph-mon[130879]: from='osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T08:54:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:14 smithi157 ceph-mon[130879]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T08:54:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:14 smithi157 ceph-mon[130879]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:16.242 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T08:54:15.982+0000 7fb04deaa700 -1 osd.5 0 waiting for initial osdmap 2023-08-25T08:54:16.242 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T08:54:15.989+0000 7fb048c9d700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:54:16.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-25T08:54:16.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: osdmap e36: 6 total, 5 up, 6 in 2023-08-25T08:54:16.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:16.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:54:16.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:54:16.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi181 2023-08-25T08:54:16.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:16.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:16.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi181 to 8584M 2023-08-25T08:54:16.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:16.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:16.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.297 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-25T08:54:16.298 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-25T08:54:16.298 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-08-25T08:54:16.298 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-08-25T08:54:16.298 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0349517 s, 300 MB/s 2023-08-25T08:54:16.299 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-08-25T08:54:16.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-25T08:54:16.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: osdmap e36: 6 total, 5 up, 6 in 2023-08-25T08:54:16.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:16.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:54:16.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:54:16.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi181 2023-08-25T08:54:16.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:16.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:16.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi181 to 8584M 2023-08-25T08:54:16.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:16.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:16.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-25T08:54:16.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: osdmap e36: 6 total, 5 up, 6 in 2023-08-25T08:54:16.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:16.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:54:16.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:54:16.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi181 2023-08-25T08:54:16.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:16.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:16.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi181 to 8584M 2023-08-25T08:54:16.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:16.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:16.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:16.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2023-08-25T08:54:17.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:16 smithi099 ceph-mon[97641]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:54:17.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:16 smithi099 ceph-mon[97641]: osdmap e37: 6 total, 5 up, 6 in 2023-08-25T08:54:17.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:16 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:17.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:16 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:17.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:16 smithi099 ceph-mon[97641]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:17.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:16 smithi181 ceph-mon[131296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:54:17.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:16 smithi181 ceph-mon[131296]: osdmap e37: 6 total, 5 up, 6 in 2023-08-25T08:54:17.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:16 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:17.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:16 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:17.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:16 smithi181 ceph-mon[131296]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:17.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:16 smithi157 ceph-mon[130879]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:54:17.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:16 smithi157 ceph-mon[130879]: osdmap e37: 6 total, 5 up, 6 in 2023-08-25T08:54:17.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:16 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:17.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:16 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:17.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:16 smithi157 ceph-mon[130879]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T08:54:17.484 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:54:18.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:17 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:54:18.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:17 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:54:18.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:18.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:17 smithi099 ceph-mon[97641]: osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] boot 2023-08-25T08:54:18.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:17 smithi099 ceph-mon[97641]: osdmap e38: 6 total, 6 up, 6 in 2023-08-25T08:54:18.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:18.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:17 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:54:18.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:17 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:54:18.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:17 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:18.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:17 smithi157 ceph-mon[130879]: osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] boot 2023-08-25T08:54:18.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:17 smithi157 ceph-mon[130879]: osdmap e38: 6 total, 6 up, 6 in 2023-08-25T08:54:18.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:17 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:18.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:17 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:54:18.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:17 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:54:18.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:17 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:18.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:17 smithi181 ceph-mon[131296]: osd.5 [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] boot 2023-08-25T08:54:18.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:17 smithi181 ceph-mon[131296]: osdmap e38: 6 total, 6 up, 6 in 2023-08-25T08:54:18.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:17 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:54:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 -- 172.21.15.99:0/2243976687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2020102550 msgr2=0x7f2020108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 --2- 172.21.15.99:0/2243976687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2020102550 0x7f2020108e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2018009fc0 tx=0x7f201800b330 comp rx=0 tx=0).stop 2023-08-25T08:54:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 -- 172.21.15.99:0/2243976687 shutdown_connections 2023-08-25T08:54:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 --2- 172.21.15.99:0/2243976687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2020102550 0x7f2020108e00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 --2- 172.21.15.99:0/2243976687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20201009b0 0x7f2020100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 --2- 172.21.15.99:0/2243976687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020101bb0 0x7f2020102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 -- 172.21.15.99:0/2243976687 >> 172.21.15.99:0/2243976687 conn(0x7f20200fbf10 msgr2=0x7f20200fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:54:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 -- 172.21.15.99:0/2243976687 shutdown_connections 2023-08-25T08:54:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.975+0000 7f2027d68700 1 -- 172.21.15.99:0/2243976687 wait complete. 2023-08-25T08:54:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.976+0000 7f2027d68700 1 Processor -- start 2023-08-25T08:54:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2027d68700 1 -- start start 2023-08-25T08:54:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2027d68700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20201009b0 0x7f20201989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:54:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2027d68700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020101bb0 0x7f2020198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:54:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2027d68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020102550 0x7f202019ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:54:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2027d68700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20201a04f0 con 0x7f20201009b0 2023-08-25T08:54:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2027d68700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20201a0630 con 0x7f2020101bb0 2023-08-25T08:54:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2027d68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20201a07c0 con 0x7f2020102550 2023-08-25T08:54:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2025b04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20201009b0 0x7f20201989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:54:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2025b04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20201009b0 0x7f20201989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35762/0 (socket says 172.21.15.99:35762) 2023-08-25T08:54:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2025b04700 1 -- 172.21.15.99:0/1333439770 learned_addr learned my addr 172.21.15.99:0/1333439770 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:54:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.977+0000 7f2025303700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020101bb0 0x7f2020198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:54:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2026305700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020102550 0x7f202019ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:54:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2025b04700 1 -- 172.21.15.99:0/1333439770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020102550 msgr2=0x7f202019ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2025b04700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020102550 0x7f202019ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2025b04700 1 -- 172.21.15.99:0/1333439770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020101bb0 msgr2=0x7f2020198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2025b04700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020101bb0 0x7f2020198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2025b04700 1 -- 172.21.15.99:0/1333439770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018009c40 con 0x7f20201009b0 2023-08-25T08:54:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2026305700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020102550 0x7f202019ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:54:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2025b04700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20201009b0 0x7f20201989b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f201c00ea20 tx=0x7f201c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:54:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2016ffd700 1 -- 172.21.15.99:0/1333439770 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00cbd0 con 0x7f20201009b0 2023-08-25T08:54:18.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2016ffd700 1 -- 172.21.15.99:0/1333439770 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f201c00ee50 con 0x7f20201009b0 2023-08-25T08:54:18.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20201a0aa0 con 0x7f20201009b0 2023-08-25T08:54:18.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2016ffd700 1 -- 172.21.15.99:0/1333439770 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c003ce0 con 0x7f20201009b0 2023-08-25T08:54:18.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.978+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20201a0fa0 con 0x7f20201009b0 2023-08-25T08:54:18.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.980+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020193000 con 0x7f20201009b0 2023-08-25T08:54:18.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.981+0000 7f2016ffd700 1 -- 172.21.15.99:0/1333439770 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f201c003e40 con 0x7f20201009b0 2023-08-25T08:54:18.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.983+0000 7f2016ffd700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f200c09ff50 0x7f200c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:54:18.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.983+0000 7f2016ffd700 1 -- 172.21.15.99:0/1333439770 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f201c014070 con 0x7f20201009b0 2023-08-25T08:54:18.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.983+0000 7f2025303700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f200c09ff50 0x7f200c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:54:18.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.983+0000 7f2025303700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f200c09ff50 0x7f200c0a2410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2010009c60 tx=0x7f2010009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:54:18.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:18.987+0000 7f2016ffd700 1 -- 172.21.15.99:0/1333439770 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f201c0951b0 con 0x7f20201009b0 2023-08-25T08:54:19.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:19.197+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f202002cde0 con 0x7f200c09ff50 2023-08-25T08:54:19.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:19 smithi099 ceph-mon[97641]: osdmap e39: 6 total, 6 up, 6 in 2023-08-25T08:54:19.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:19 smithi099 ceph-mon[97641]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:19.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:18 smithi157 ceph-mon[130879]: osdmap e39: 6 total, 6 up, 6 in 2023-08-25T08:54:19.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:18 smithi157 ceph-mon[130879]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:19.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:18 smithi181 ceph-mon[131296]: osdmap e39: 6 total, 6 up, 6 in 2023-08-25T08:54:19.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:18 smithi181 ceph-mon[131296]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:20.275 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:20 smithi099 ceph-mon[97641]: osdmap e40: 6 total, 6 up, 6 in 2023-08-25T08:54:20.275 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:20 smithi099 ceph-mon[97641]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:54:20.275 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:54:20.276 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:54:20.276 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:20.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:20 smithi157 ceph-mon[130879]: osdmap e40: 6 total, 6 up, 6 in 2023-08-25T08:54:20.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:20 smithi157 ceph-mon[130879]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:54:20.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:20 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:54:20.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:20 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:54:20.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:20 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:20.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:20 smithi181 ceph-mon[131296]: osdmap e40: 6 total, 6 up, 6 in 2023-08-25T08:54:20.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:20 smithi181 ceph-mon[131296]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:54:20.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:20 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:54:20.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:20 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:54:20.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:20 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:21.263 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:21 smithi099 ceph-mon[97641]: osdmap e41: 6 total, 6 up, 6 in 2023-08-25T08:54:21.263 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:21 smithi099 ceph-mon[97641]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:21.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:21 smithi157 ceph-mon[130879]: osdmap e41: 6 total, 6 up, 6 in 2023-08-25T08:54:21.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:21 smithi157 ceph-mon[130879]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:21.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:21 smithi181 ceph-mon[131296]: osdmap e41: 6 total, 6 up, 6 in 2023-08-25T08:54:21.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:21 smithi181 ceph-mon[131296]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:22.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:22 smithi099 ceph-mon[97641]: osdmap e42: 6 total, 6 up, 6 in 2023-08-25T08:54:22.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:22 smithi157 ceph-mon[130879]: osdmap e42: 6 total, 6 up, 6 in 2023-08-25T08:54:22.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:22 smithi181 ceph-mon[131296]: osdmap e42: 6 total, 6 up, 6 in 2023-08-25T08:54:23.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:23 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1888071463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8629c9d9-a959-4532-94c1-9b923a0760a2"}]: dispatch 2023-08-25T08:54:23.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:23 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1888071463' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8629c9d9-a959-4532-94c1-9b923a0760a2"}]': finished 2023-08-25T08:54:23.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:23 smithi099 ceph-mon[97641]: osdmap e43: 7 total, 6 up, 7 in 2023-08-25T08:54:23.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:23.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:23 smithi099 ceph-mon[97641]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:23 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1888071463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8629c9d9-a959-4532-94c1-9b923a0760a2"}]: dispatch 2023-08-25T08:54:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:23 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1888071463' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8629c9d9-a959-4532-94c1-9b923a0760a2"}]': finished 2023-08-25T08:54:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:23 smithi157 ceph-mon[130879]: osdmap e43: 7 total, 6 up, 7 in 2023-08-25T08:54:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:23 smithi157 ceph-mon[130879]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:23.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:23 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1888071463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8629c9d9-a959-4532-94c1-9b923a0760a2"}]: dispatch 2023-08-25T08:54:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:23 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1888071463' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8629c9d9-a959-4532-94c1-9b923a0760a2"}]': finished 2023-08-25T08:54:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:23 smithi181 ceph-mon[131296]: osdmap e43: 7 total, 6 up, 7 in 2023-08-25T08:54:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:23 smithi181 ceph-mon[131296]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:24.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:24 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3290847109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:24.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:24 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3290847109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:24.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:24 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3290847109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:25.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:25 smithi099 ceph-mon[97641]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-08-25T08:54:25.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:25 smithi157 ceph-mon[130879]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-08-25T08:54:25.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:25 smithi181 ceph-mon[131296]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-08-25T08:54:27.714 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:27 smithi099 ceph-mon[97641]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-08-25T08:54:27.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:27 smithi181 ceph-mon[131296]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-08-25T08:54:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:27 smithi157 ceph-mon[130879]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-08-25T08:54:29.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:29 smithi181 ceph-mon[131296]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T08:54:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:29 smithi099 ceph-mon[97641]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T08:54:29.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:29 smithi157 ceph-mon[130879]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T08:54:30.721 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-25T08:54:30.722 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:30.722 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:30 smithi099 ceph-mon[97641]: Deploying daemon osd.6 on smithi099 2023-08-25T08:54:30.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-25T08:54:30.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:30.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:30 smithi181 ceph-mon[131296]: Deploying daemon osd.6 on smithi099 2023-08-25T08:54:30.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-25T08:54:30.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:30.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:30 smithi157 ceph-mon[130879]: Deploying daemon osd.6 on smithi099 2023-08-25T08:54:31.734 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:31 smithi099 ceph-mon[97641]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-08-25T08:54:31.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:31 smithi181 ceph-mon[131296]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-08-25T08:54:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:31 smithi157 ceph-mon[130879]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-08-25T08:54:33.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:33 smithi181 ceph-mon[131296]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-08-25T08:54:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:33 smithi099 ceph-mon[97641]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-08-25T08:54:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:33 smithi157 ceph-mon[130879]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-08-25T08:54:35.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:34 smithi181 ceph-mon[131296]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T08:54:35.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:34 smithi099 ceph-mon[97641]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T08:54:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:34 smithi157 ceph-mon[130879]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T08:54:37.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:36 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:37.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:36 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:37.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:36 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:37.128 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:36 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:37.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:36 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:37.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:36 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:37.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:36 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:37.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:36 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:37.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:36 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:38.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:37 smithi099 ceph-mon[97641]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:38.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:37 smithi157 ceph-mon[130879]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:38.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:37 smithi181 ceph-mon[131296]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.960+0000 7f2016ffd700 1 -- 172.21.15.99:0/1333439770 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f202002cde0 con 0x7f200c09ff50 2023-08-25T08:54:38.961 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 6 on host 'smithi099' 2023-08-25T08:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f200c09ff50 msgr2=0x7f200c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f200c09ff50 0x7f200c0a2410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2010009c60 tx=0x7f2010009260 comp rx=0 tx=0).stop 2023-08-25T08:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20201009b0 msgr2=0x7f20201989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20201009b0 0x7f20201989b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f201c00ea20 tx=0x7f201c00ed30 comp rx=0 tx=0).stop 2023-08-25T08:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 shutdown_connections 2023-08-25T08:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020102550 0x7f202019ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20201009b0 0x7f20201989b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f200c09ff50 0x7f200c0a2410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 --2- 172.21.15.99:0/1333439770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020101bb0 0x7f2020198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 >> 172.21.15.99:0/1333439770 conn(0x7f20200fbf10 msgr2=0x7f2020107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 shutdown_connections 2023-08-25T08:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:54:38.963+0000 7f2027d68700 1 -- 172.21.15.99:0/1333439770 wait complete. 2023-08-25T08:54:39.403 DEBUG:teuthology.orchestra.run.smithi099:osd.6> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.6.service 2023-08-25T08:54:39.408 INFO:tasks.cephadm:Deploying osd.7 on smithi157 with /dev/vg_nvme/lv_2... 2023-08-25T08:54:39.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-25T08:54:39.479 INFO:journalctl@ceph.osd.6.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:54:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:39.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:39.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:39 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T08:54:39.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:39.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:39.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:39.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:39.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:39 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:40.127 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 08:54:39 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T08:54:39.841+0000 7fcccc7e1640 -1 osd.6 0 log_to_monitors true 2023-08-25T08:54:40.788 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:40 smithi099 ceph-mon[97641]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-25T08:54:40.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:40 smithi157 ceph-mon[130879]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-25T08:54:40.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:40 smithi181 ceph-mon[131296]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-25T08:54:41.505 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-25T08:54:41.506 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-25T08:54:41.506 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2023-08-25T08:54:41.506 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2023-08-25T08:54:41.506 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834769 s, 1.3 GB/s 2023-08-25T08:54:41.506 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2023-08-25T08:54:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:41 smithi157 ceph-mon[130879]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:41 smithi157 ceph-mon[130879]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-25T08:54:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:41 smithi157 ceph-mon[130879]: osdmap e44: 7 total, 6 up, 7 in 2023-08-25T08:54:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:41 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:41 smithi157 ceph-mon[130879]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:54:41.940 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2023-08-25T08:54:41.960 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:41 smithi099 ceph-mon[97641]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:41.961 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:41 smithi099 ceph-mon[97641]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-25T08:54:41.961 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:41 smithi099 ceph-mon[97641]: osdmap e44: 7 total, 6 up, 7 in 2023-08-25T08:54:41.961 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:41 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:41.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:41 smithi099 ceph-mon[97641]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:54:41.962 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 08:54:41 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T08:54:41.587+0000 7fccc1e58700 -1 osd.6 0 waiting for initial osdmap 2023-08-25T08:54:41.962 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 08:54:41 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T08:54:41.598+0000 7fccba446700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:54:41.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:41 smithi181 ceph-mon[131296]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:41.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:41 smithi181 ceph-mon[131296]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-25T08:54:41.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:41 smithi181 ceph-mon[131296]: osdmap e44: 7 total, 6 up, 7 in 2023-08-25T08:54:41.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:41 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:41.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:41 smithi181 ceph-mon[131296]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:54:42.497 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T08:54:42.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:54:42.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: osdmap e45: 7 total, 6 up, 7 in 2023-08-25T08:54:42.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:42.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:42.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:42.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:42.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:42 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:54:42.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: osdmap e45: 7 total, 6 up, 7 in 2023-08-25T08:54:42.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:42.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:42.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:42.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:42.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:42 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:54:42.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: osdmap e45: 7 total, 6 up, 7 in 2023-08-25T08:54:42.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:42.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:42.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:54:42.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:42.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:42.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:54:42.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:42 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:43.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 -- 172.21.15.157:0/1882685513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c1009d0 msgr2=0x7f5e2c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:43.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 --2- 172.21.15.157:0/1882685513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c1009d0 0x7f5e2c100df0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0099c0 tx=0x7f5e1c009cd0 comp rx=0 tx=0).stop 2023-08-25T08:54:43.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 -- 172.21.15.157:0/1882685513 shutdown_connections 2023-08-25T08:54:43.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 --2- 172.21.15.157:0/1882685513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e2c102570 0x7f5e2c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:43.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 --2- 172.21.15.157:0/1882685513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c1009d0 0x7f5e2c100df0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:43.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 --2- 172.21.15.157:0/1882685513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e2c101bd0 0x7f5e2c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:43.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 -- 172.21.15.157:0/1882685513 >> 172.21.15.157:0/1882685513 conn(0x7f5e2c0fbf30 msgr2=0x7f5e2c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:54:43.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 -- 172.21.15.157:0/1882685513 shutdown_connections 2023-08-25T08:54:43.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 -- 172.21.15.157:0/1882685513 wait complete. 2023-08-25T08:54:43.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.689+0000 7f5e32652700 1 Processor -- start 2023-08-25T08:54:43.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e32652700 1 -- start start 2023-08-25T08:54:43.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e32652700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e2c1009d0 0x7f5e2c0746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:54:43.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e32652700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e2c101bd0 0x7f5e2c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:54:43.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e32652700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c102570 0x7f5e2c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:54:43.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e32652700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0739d0 con 0x7f5e2c101bd0 2023-08-25T08:54:43.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e32652700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c073b40 con 0x7f5e2c102570 2023-08-25T08:54:43.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e32652700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c1a07a0 con 0x7f5e2c1009d0 2023-08-25T08:54:43.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c102570 0x7f5e2c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:54:43.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c102570 0x7f5e2c073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57804/0 (socket says 172.21.15.157:57804) 2023-08-25T08:54:43.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 -- 172.21.15.157:0/2905650952 learned_addr learned my addr 172.21.15.157:0/2905650952 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-08-25T08:54:43.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e2bfff700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e2c1009d0 0x7f5e2c0746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:54:43.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 -- 172.21.15.157:0/2905650952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e2c1009d0 msgr2=0x7f5e2c0746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:43.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e2c1009d0 0x7f5e2c0746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:43.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 -- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e2c101bd0 msgr2=0x7f5e2c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:43.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e2c101bd0 0x7f5e2c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:43.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 -- 172.21.15.157:0/2905650952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c0095f0 con 0x7f5e2c102570 2023-08-25T08:54:43.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e2bfff700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e2c1009d0 0x7f5e2c0746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:54:43.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.690+0000 7f5e30bef700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c102570 0x7f5e2c073250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007ee0 tx=0x7f5e2400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:54:43.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.691+0000 7f5e297fa700 1 -- 172.21.15.157:0/2905650952 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400ddd0 con 0x7f5e2c102570 2023-08-25T08:54:43.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.691+0000 7f5e297fa700 1 -- 172.21.15.157:0/2905650952 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2400f040 con 0x7f5e2c102570 2023-08-25T08:54:43.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.691+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c1a0a80 con 0x7f5e2c102570 2023-08-25T08:54:43.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.691+0000 7f5e297fa700 1 -- 172.21.15.157:0/2905650952 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24015400 con 0x7f5e2c102570 2023-08-25T08:54:43.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.691+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c1a0fd0 con 0x7f5e2c102570 2023-08-25T08:54:43.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.692+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c10c250 con 0x7f5e2c102570 2023-08-25T08:54:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.693+0000 7f5e297fa700 1 -- 172.21.15.157:0/2905650952 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f5e24004020 con 0x7f5e2c102570 2023-08-25T08:54:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.693+0000 7f5e297fa700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5e1409ffd0 0x7f5e140a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:54:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.693+0000 7f5e297fa700 1 -- 172.21.15.157:0/2905650952 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f5e240ca670 con 0x7f5e2c102570 2023-08-25T08:54:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.694+0000 7f5e2bfff700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5e1409ffd0 0x7f5e140a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:54:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.694+0000 7f5e2bfff700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5e1409ffd0 0x7f5e140a2490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00b580 tx=0x7f5e1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:54:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.697+0000 7f5e297fa700 1 -- 172.21.15.157:0/2905650952 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5e24095110 con 0x7f5e2c102570 2023-08-25T08:54:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:43 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:54:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:43 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:54:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:43 smithi157 ceph-mon[130879]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:43 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi099 2023-08-25T08:54:43.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:43 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi099 to 5755M 2023-08-25T08:54:43.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:43 smithi157 ceph-mon[130879]: osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] boot 2023-08-25T08:54:43.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:43 smithi157 ceph-mon[130879]: osdmap e46: 7 total, 7 up, 7 in 2023-08-25T08:54:43.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:43.893+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e2c060d60 con 0x7f5e1409ffd0 2023-08-25T08:54:43.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:43 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:54:43.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:43 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:54:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:43 smithi181 ceph-mon[131296]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:43 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi099 2023-08-25T08:54:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:43 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi099 to 5755M 2023-08-25T08:54:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:43 smithi181 ceph-mon[131296]: osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] boot 2023-08-25T08:54:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:43 smithi181 ceph-mon[131296]: osdmap e46: 7 total, 7 up, 7 in 2023-08-25T08:54:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:44.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:43 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:54:44.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:43 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:54:44.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:43 smithi099 ceph-mon[97641]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T08:54:44.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:43 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi099 2023-08-25T08:54:44.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:43 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi099 to 5755M 2023-08-25T08:54:44.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:43 smithi099 ceph-mon[97641]: osd.6 [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] boot 2023-08-25T08:54:44.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:43 smithi099 ceph-mon[97641]: osdmap e46: 7 total, 7 up, 7 in 2023-08-25T08:54:44.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:54:44.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:44 smithi157 ceph-mon[130879]: osdmap e47: 7 total, 7 up, 7 in 2023-08-25T08:54:44.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:44 smithi157 ceph-mon[130879]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:54:44.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:44 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:54:44.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:44 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:54:44.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:44 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:44.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:44 smithi181 ceph-mon[131296]: osdmap e47: 7 total, 7 up, 7 in 2023-08-25T08:54:44.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:44 smithi181 ceph-mon[131296]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:54:44.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:44 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:54:44.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:44 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:54:44.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:44 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:44 smithi099 ceph-mon[97641]: osdmap e47: 7 total, 7 up, 7 in 2023-08-25T08:54:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:44 smithi099 ceph-mon[97641]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:54:45.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:44 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:54:45.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:44 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:54:45.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:44 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:45.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:45 smithi157 ceph-mon[130879]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:45.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:45 smithi181 ceph-mon[131296]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:46.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:45 smithi099 ceph-mon[97641]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:46.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:46 smithi157 ceph-mon[130879]: from='client.? 172.21.15.157:0/488297654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]: dispatch 2023-08-25T08:54:46.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:46 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]: dispatch 2023-08-25T08:54:46.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:46 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]': finished 2023-08-25T08:54:46.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:46 smithi157 ceph-mon[130879]: osdmap e48: 8 total, 7 up, 8 in 2023-08-25T08:54:46.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:46 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:54:46.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:46 smithi181 ceph-mon[131296]: from='client.? 172.21.15.157:0/488297654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]: dispatch 2023-08-25T08:54:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:46 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]: dispatch 2023-08-25T08:54:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:46 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]': finished 2023-08-25T08:54:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:46 smithi181 ceph-mon[131296]: osdmap e48: 8 total, 7 up, 8 in 2023-08-25T08:54:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:46 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:54:47.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:46 smithi099 ceph-mon[97641]: from='client.? 172.21.15.157:0/488297654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]: dispatch 2023-08-25T08:54:47.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:46 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]: dispatch 2023-08-25T08:54:47.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:46 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5"}]': finished 2023-08-25T08:54:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:46 smithi099 ceph-mon[97641]: osdmap e48: 8 total, 7 up, 8 in 2023-08-25T08:54:47.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:46 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:54:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:47 smithi157 ceph-mon[130879]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:47 smithi157 ceph-mon[130879]: from='client.? 172.21.15.157:0/3312353323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:47.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:47 smithi181 ceph-mon[131296]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:47.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:47 smithi181 ceph-mon[131296]: from='client.? 172.21.15.157:0/3312353323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:48.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:47 smithi099 ceph-mon[97641]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:48.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:47 smithi099 ceph-mon[97641]: from='client.? 172.21.15.157:0/3312353323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:54:49.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:49 smithi157 ceph-mon[130879]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:49.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:49 smithi181 ceph-mon[131296]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:50.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:49 smithi099 ceph-mon[97641]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:51 smithi157 ceph-mon[130879]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:51.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:51 smithi181 ceph-mon[131296]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:52.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:51 smithi099 ceph-mon[97641]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:53 smithi157 ceph-mon[130879]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-25T08:54:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:53 smithi157 ceph-mon[130879]: Deploying daemon osd.7 on smithi157 2023-08-25T08:54:53.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:53 smithi181 ceph-mon[131296]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-25T08:54:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:53 smithi181 ceph-mon[131296]: Deploying daemon osd.7 on smithi157 2023-08-25T08:54:54.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:53 smithi099 ceph-mon[97641]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-25T08:54:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:54:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:53 smithi099 ceph-mon[97641]: Deploying daemon osd.7 on smithi157 2023-08-25T08:54:55.070 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:54 smithi157 ceph-mon[130879]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:55.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:54 smithi181 ceph-mon[131296]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:54 smithi099 ceph-mon[97641]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:58.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:57 smithi157 ceph-mon[130879]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:58.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:57 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:58.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:57 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:58.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:57 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:58.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:57 smithi181 ceph-mon[131296]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:58.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:57 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:58.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:57 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:58.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:57 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:58.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:57 smithi099 ceph-mon[97641]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:54:58.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:58.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:54:58.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:54:59.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.605+0000 7f5e297fa700 1 -- 172.21.15.157:0/2905650952 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5e2c060d60 con 0x7f5e1409ffd0 2023-08-25T08:54:59.606 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 7 on host 'smithi157' 2023-08-25T08:54:59.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5e1409ffd0 msgr2=0x7f5e140a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:59.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5e1409ffd0 0x7f5e140a2490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00b580 tx=0x7f5e1c005fb0 comp rx=0 tx=0).stop 2023-08-25T08:54:59.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c102570 msgr2=0x7f5e2c073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:54:59.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c102570 0x7f5e2c073250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007ee0 tx=0x7f5e2400d3b0 comp rx=0 tx=0).stop 2023-08-25T08:54:59.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 shutdown_connections 2023-08-25T08:54:59.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f5e1409ffd0 0x7f5e140a2490 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:59.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2c102570 0x7f5e2c073250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:59.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e2c1009d0 0x7f5e2c0746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:59.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 --2- 172.21.15.157:0/2905650952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e2c101bd0 0x7f5e2c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:54:59.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 >> 172.21.15.157:0/2905650952 conn(0x7f5e2c0fbf30 msgr2=0x7f5e2c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:54:59.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 shutdown_connections 2023-08-25T08:54:59.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:54:59.607+0000 7f5e32652700 1 -- 172.21.15.157:0/2905650952 wait complete. 2023-08-25T08:54:59.986 DEBUG:teuthology.orchestra.run.smithi157:osd.7> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.7.service 2023-08-25T08:54:59.989 INFO:tasks.cephadm:Deploying osd.8 on smithi181 with /dev/vg_nvme/lv_2... 2023-08-25T08:54:59.990 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-25T08:55:00.018 INFO:journalctl@ceph.osd.7.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:55:00.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:00.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:00.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:00.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:54:59 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:00.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:00.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:00.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:00.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:54:59 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:00.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:00.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:00.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:00.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:00.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:54:59 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:00.552 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:55:01.057 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 08:55:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T08:55:00.866+0000 7f774b0f3640 -1 osd.7 0 log_to_monitors true 2023-08-25T08:55:01.058 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:00 smithi157 ceph-mon[130879]: from='osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T08:55:01.058 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:00 smithi157 ceph-mon[130879]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T08:55:01.071 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:00 smithi181 ceph-mon[131296]: from='osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T08:55:01.071 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:00 smithi181 ceph-mon[131296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T08:55:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:00 smithi099 ceph-mon[97641]: from='osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T08:55:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:00 smithi099 ceph-mon[97641]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T08:55:01.903 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:01 smithi181 ceph-mon[131296]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:01.903 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:01 smithi181 ceph-mon[131296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-25T08:55:01.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:01 smithi181 ceph-mon[131296]: osdmap e49: 8 total, 7 up, 8 in 2023-08-25T08:55:01.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:01.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:01 smithi181 ceph-mon[131296]: from='osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:55:01.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:01 smithi181 ceph-mon[131296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-08-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-08-25T08:55:02.078 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115432 s, 908 MB/s 2023-08-25T08:55:02.078 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-08-25T08:55:02.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:01 smithi157 ceph-mon[130879]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:02.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:01 smithi157 ceph-mon[130879]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-25T08:55:02.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:01 smithi157 ceph-mon[130879]: osdmap e49: 8 total, 7 up, 8 in 2023-08-25T08:55:02.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:02.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:01 smithi157 ceph-mon[130879]: from='osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:55:02.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:01 smithi157 ceph-mon[130879]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:55:02.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 08:55:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T08:55:01.902+0000 7f774076a700 -1 osd.7 0 waiting for initial osdmap 2023-08-25T08:55:02.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 08:55:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T08:55:01.912+0000 7f773ad5c700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:55:02.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:01 smithi099 ceph-mon[97641]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:02.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:01 smithi099 ceph-mon[97641]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-25T08:55:02.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:01 smithi099 ceph-mon[97641]: osdmap e49: 8 total, 7 up, 8 in 2023-08-25T08:55:02.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:02.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:01 smithi099 ceph-mon[97641]: from='osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:55:02.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:01 smithi099 ceph-mon[97641]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:55:02.504 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2023-08-25T08:55:03.055 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:55:03.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:55:03.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: osdmap e50: 8 total, 7 up, 8 in 2023-08-25T08:55:03.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:03.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:03.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:03.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:03.246 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:02 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:55:03.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: osdmap e50: 8 total, 7 up, 8 in 2023-08-25T08:55:03.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:03.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:03.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:03.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:03.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:02 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:55:03.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: osdmap e50: 8 total, 7 up, 8 in 2023-08-25T08:55:03.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:03.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:03.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:03.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:03.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:03.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:03.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:02 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:04.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.232+0000 7fde64b39700 1 -- 172.21.15.181:0/1650223690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 msgr2=0x7fde600fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.232+0000 7fde64b39700 1 --2- 172.21.15.181:0/1650223690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 0x7fde600fffe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fde540099c0 tx=0x7fde54009cd0 comp rx=0 tx=0).stop 2023-08-25T08:55:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 -- 172.21.15.181:0/1650223690 shutdown_connections 2023-08-25T08:55:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 --2- 172.21.15.181:0/1650223690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde60100520 0x7fde60107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 --2- 172.21.15.181:0/1650223690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde60100eb0 0x7fde601012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 --2- 172.21.15.181:0/1650223690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 0x7fde600fffe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 -- 172.21.15.181:0/1650223690 >> 172.21.15.181:0/1650223690 conn(0x7fde600fa5d0 msgr2=0x7fde600fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:55:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 -- 172.21.15.181:0/1650223690 shutdown_connections 2023-08-25T08:55:04.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 -- 172.21.15.181:0/1650223690 wait complete. 2023-08-25T08:55:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 Processor -- start 2023-08-25T08:55:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.233+0000 7fde64b39700 1 -- start start 2023-08-25T08:55:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde64b39700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 0x7fde600ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde64b39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde60100520 0x7fde6019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde64b39700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde60100eb0 0x7fde6019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde64b39700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6019e2e0 con 0x7fde60100520 2023-08-25T08:55:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde64b39700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6019e450 con 0x7fde60100eb0 2023-08-25T08:55:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde64b39700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6019e610 con 0x7fde600ffb60 2023-08-25T08:55:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 0x7fde600ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 0x7fde600ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43308/0 (socket says 172.21.15.181:43308) 2023-08-25T08:55:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 -- 172.21.15.181:0/3001864586 learned_addr learned my addr 172.21.15.181:0/3001864586 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:55:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 -- 172.21.15.181:0/3001864586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde60100eb0 msgr2=0x7fde6019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde60100eb0 0x7fde6019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 -- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde60100520 msgr2=0x7fde6019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde60100520 0x7fde6019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 -- 172.21.15.181:0/3001864586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde540095f0 con 0x7fde600ffb60 2023-08-25T08:55:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde5e59c700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 0x7fde600ff5b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde5000c970 tx=0x7fde5000cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:55:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde4f7fe700 1 -- 172.21.15.181:0/3001864586 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50004220 con 0x7fde600ffb60 2023-08-25T08:55:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde4f7fe700 1 -- 172.21.15.181:0/3001864586 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde50007d60 con 0x7fde600ffb60 2023-08-25T08:55:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde4f7fe700 1 -- 172.21.15.181:0/3001864586 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50003d00 con 0x7fde600ffb60 2023-08-25T08:55:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6019e8f0 con 0x7fde600ffb60 2023-08-25T08:55:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.234+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6019eda0 con 0x7fde600ffb60 2023-08-25T08:55:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.235+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde60107010 con 0x7fde600ffb60 2023-08-25T08:55:04.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.237+0000 7fde4f7fe700 1 -- 172.21.15.181:0/3001864586 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fde50004380 con 0x7fde600ffb60 2023-08-25T08:55:04.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.238+0000 7fde4f7fe700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fde4809ff50 0x7fde480a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:04.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.238+0000 7fde4f7fe700 1 -- 172.21.15.181:0/3001864586 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fde500ca710 con 0x7fde600ffb60 2023-08-25T08:55:04.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.238+0000 7fde5dd9b700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fde4809ff50 0x7fde480a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:04.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.238+0000 7fde5dd9b700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fde4809ff50 0x7fde480a2410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde540036f0 tx=0x7fde54000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:55:04.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.241+0000 7fde4f7fe700 1 -- 172.21.15.181:0/3001864586 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fde50094f40 con 0x7fde600ffb60 2023-08-25T08:55:04.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:55:04.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:55:04.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi157 2023-08-25T08:55:04.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi157 to 5722M 2023-08-25T08:55:04.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:04.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:04.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] boot 2023-08-25T08:55:04.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: osdmap e51: 8 total, 8 up, 8 in 2023-08-25T08:55:04.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:55:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:55:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi157 2023-08-25T08:55:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi157 to 5722M 2023-08-25T08:55:04.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] boot 2023-08-25T08:55:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: osdmap e51: 8 total, 8 up, 8 in 2023-08-25T08:55:04.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:04.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:55:04.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:55:04.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi157 2023-08-25T08:55:04.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi157 to 5722M 2023-08-25T08:55:04.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T08:55:04.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:04.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: osd.7 [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] boot 2023-08-25T08:55:04.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: osdmap e51: 8 total, 8 up, 8 in 2023-08-25T08:55:04.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:55:04.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:04.438+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fde6002cf50 con 0x7fde4809ff50 2023-08-25T08:55:05.217 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:04 smithi181 ceph-mon[131296]: osdmap e52: 8 total, 8 up, 8 in 2023-08-25T08:55:05.217 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:05.218 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:05.218 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:04 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:04 smithi099 ceph-mon[97641]: osdmap e52: 8 total, 8 up, 8 in 2023-08-25T08:55:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:05.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:05.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:04 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:04 smithi157 ceph-mon[130879]: osdmap e52: 8 total, 8 up, 8 in 2023-08-25T08:55:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:04 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:06.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:05 smithi181 ceph-mon[131296]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:06.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:05 smithi181 ceph-mon[131296]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:06.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:05 smithi181 ceph-mon[131296]: osdmap e53: 8 total, 8 up, 8 in 2023-08-25T08:55:06.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:05 smithi099 ceph-mon[97641]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:06.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:05 smithi099 ceph-mon[97641]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:06.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:05 smithi099 ceph-mon[97641]: osdmap e53: 8 total, 8 up, 8 in 2023-08-25T08:55:06.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:05 smithi157 ceph-mon[130879]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:06.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:05 smithi157 ceph-mon[130879]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:06.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:05 smithi157 ceph-mon[130879]: osdmap e53: 8 total, 8 up, 8 in 2023-08-25T08:55:07.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:06 smithi181 ceph-mon[131296]: osdmap e54: 8 total, 8 up, 8 in 2023-08-25T08:55:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:06 smithi181 ceph-mon[131296]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:06 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/3331812625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]: dispatch 2023-08-25T08:55:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:06 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]: dispatch 2023-08-25T08:55:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:06 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]': finished 2023-08-25T08:55:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:06 smithi181 ceph-mon[131296]: osdmap e55: 9 total, 8 up, 9 in 2023-08-25T08:55:07.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:06 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:07.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:06 smithi099 ceph-mon[97641]: osdmap e54: 8 total, 8 up, 8 in 2023-08-25T08:55:07.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:06 smithi099 ceph-mon[97641]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:06 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/3331812625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]: dispatch 2023-08-25T08:55:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:06 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]: dispatch 2023-08-25T08:55:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:06 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]': finished 2023-08-25T08:55:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:06 smithi099 ceph-mon[97641]: osdmap e55: 9 total, 8 up, 9 in 2023-08-25T08:55:07.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:06 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:07.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:06 smithi157 ceph-mon[130879]: osdmap e54: 8 total, 8 up, 8 in 2023-08-25T08:55:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:06 smithi157 ceph-mon[130879]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:06 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/3331812625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]: dispatch 2023-08-25T08:55:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:06 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]: dispatch 2023-08-25T08:55:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:06 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91174f3e-a7c9-496c-a2c7-bdf90a0d59f6"}]': finished 2023-08-25T08:55:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:06 smithi157 ceph-mon[130879]: osdmap e55: 9 total, 8 up, 9 in 2023-08-25T08:55:07.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:06 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:08.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:07 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/3384999408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:07 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/3384999408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:07 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/3384999408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:09.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:08 smithi181 ceph-mon[131296]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:09.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:08 smithi099 ceph-mon[97641]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:09.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:08 smithi157 ceph-mon[130879]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:11.735 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:11 smithi181 ceph-mon[131296]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:11 smithi099 ceph-mon[97641]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:11.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:11 smithi157 ceph-mon[130879]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:13.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:13 smithi181 ceph-mon[131296]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-25T08:55:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:13.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:13 smithi099 ceph-mon[97641]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:13.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-25T08:55:13.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:13.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:13 smithi157 ceph-mon[130879]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:13.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-25T08:55:13.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:14.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:14 smithi181 ceph-mon[131296]: Deploying daemon osd.8 on smithi181 2023-08-25T08:55:14.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:14 smithi099 ceph-mon[97641]: Deploying daemon osd.8 on smithi181 2023-08-25T08:55:14.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:14 smithi157 ceph-mon[130879]: Deploying daemon osd.8 on smithi181 2023-08-25T08:55:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:15 smithi181 ceph-mon[131296]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:15.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:15 smithi099 ceph-mon[97641]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:15.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:15 smithi157 ceph-mon[130879]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:17 smithi181 ceph-mon[131296]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:17.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:17 smithi099 ceph-mon[97641]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:17.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:17 smithi157 ceph-mon[130879]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:18.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:18 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:18.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:18 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:18.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:18 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:19.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:18 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:19.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:18 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:19.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:18 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:19.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:18 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:19.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:18 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:19.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:18 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:20.176 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:19 smithi181 ceph-mon[131296]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:20.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:19 smithi099 ceph-mon[97641]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:20.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:19 smithi157 ceph-mon[130879]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:20.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.455+0000 7fde4f7fe700 1 -- 172.21.15.181:0/3001864586 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fde6002cf50 con 0x7fde4809ff50 2023-08-25T08:55:20.456 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 8 on host 'smithi181' 2023-08-25T08:55:20.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fde4809ff50 msgr2=0x7fde480a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:20.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fde4809ff50 0x7fde480a2410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde540036f0 tx=0x7fde54000bc0 comp rx=0 tx=0).stop 2023-08-25T08:55:20.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 msgr2=0x7fde600ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:20.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 0x7fde600ff5b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde5000c970 tx=0x7fde5000cd30 comp rx=0 tx=0).stop 2023-08-25T08:55:20.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 shutdown_connections 2023-08-25T08:55:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fde4809ff50 0x7fde480a2410 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde60100eb0 0x7fde6019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600ffb60 0x7fde600ff5b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 --2- 172.21.15.181:0/3001864586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde60100520 0x7fde6019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 >> 172.21.15.181:0/3001864586 conn(0x7fde600fa5d0 msgr2=0x7fde60104150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:55:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 shutdown_connections 2023-08-25T08:55:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:55:20.458+0000 7fde64b39700 1 -- 172.21.15.181:0/3001864586 wait complete. 2023-08-25T08:55:21.044 DEBUG:teuthology.orchestra.run.smithi181:osd.8> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.8.service 2023-08-25T08:55:21.046 INFO:tasks.cephadm:Deploying osd.9 on smithi099 with /dev/vg_nvme/lv_1... 2023-08-25T08:55:21.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-25T08:55:21.079 INFO:journalctl@ceph.osd.8.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:55:21.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:21.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:21.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:21.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:21.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:21.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:21.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:21.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:21.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:21.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:21.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:21.776 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:55:21.993 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 08:55:21 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T08:55:21.938+0000 7f031bbdc640 -1 osd.8 0 log_to_monitors true 2023-08-25T08:55:22.308 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:22 smithi099 ceph-mon[97641]: from='osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T08:55:22.309 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:22 smithi099 ceph-mon[97641]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T08:55:22.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:22 smithi181 ceph-mon[131296]: from='osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T08:55:22.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:22 smithi181 ceph-mon[131296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T08:55:22.628 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:22 smithi157 ceph-mon[130879]: from='osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T08:55:22.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:22 smithi157 ceph-mon[130879]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T08:55:23.492 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T08:55:23.264+0000 7f0311253700 -1 osd.8 0 waiting for initial osdmap 2023-08-25T08:55:23.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T08:55:23.275+0000 7f030b845700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:55:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-25T08:55:23.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: osdmap e56: 9 total, 8 up, 9 in 2023-08-25T08:55:23.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:23.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:55:23.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:55:23.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:23.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.496 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:23.496 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:23.496 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-25T08:55:23.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: osdmap e56: 9 total, 8 up, 9 in 2023-08-25T08:55:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:55:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:55:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:23.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:23.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-25T08:55:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-25T08:55:23.620 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-08-25T08:55:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-08-25T08:55:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0309237 s, 339 MB/s 2023-08-25T08:55:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-08-25T08:55:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-25T08:55:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: osdmap e56: 9 total, 8 up, 9 in 2023-08-25T08:55:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:23.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:55:23.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:55:23.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T08:55:23.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:23.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:23.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:23.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:23.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:24.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2023-08-25T08:55:24.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi181 2023-08-25T08:55:24.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi181 to 5722M 2023-08-25T08:55:24.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:55:24.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: osdmap e57: 9 total, 8 up, 9 in 2023-08-25T08:55:24.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] boot 2023-08-25T08:55:24.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: osdmap e58: 9 total, 9 up, 9 in 2023-08-25T08:55:24.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:24 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi181 2023-08-25T08:55:24.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi181 to 5722M 2023-08-25T08:55:24.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:55:24.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: osdmap e57: 9 total, 8 up, 9 in 2023-08-25T08:55:24.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] boot 2023-08-25T08:55:24.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: osdmap e58: 9 total, 9 up, 9 in 2023-08-25T08:55:24.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:24 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi181 2023-08-25T08:55:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi181 to 5722M 2023-08-25T08:55:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:55:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: osdmap e57: 9 total, 8 up, 9 in 2023-08-25T08:55:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: osd.8 [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] boot 2023-08-25T08:55:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: osdmap e58: 9 total, 9 up, 9 in 2023-08-25T08:55:24.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:24 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:55:24.868 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:55:25.466 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:25 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:55:25.467 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:25 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:55:25.467 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:25 smithi099 ceph-mon[97641]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:25.467 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:25 smithi099 ceph-mon[97641]: osdmap e59: 9 total, 9 up, 9 in 2023-08-25T08:55:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:25 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:55:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:25 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:55:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:25 smithi157 ceph-mon[130879]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:25 smithi157 ceph-mon[130879]: osdmap e59: 9 total, 9 up, 9 in 2023-08-25T08:55:25.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:25 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:55:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:25 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:55:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:25 smithi181 ceph-mon[131296]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:25 smithi181 ceph-mon[131296]: osdmap e59: 9 total, 9 up, 9 in 2023-08-25T08:55:26.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 -- 172.21.15.99:0/4158257361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2440101bc0 msgr2=0x7f2440102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:26.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 --2- 172.21.15.99:0/4158257361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2440101bc0 0x7f2440102020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f24340099c0 tx=0x7f2434009cd0 comp rx=0 tx=0).stop 2023-08-25T08:55:26.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 -- 172.21.15.99:0/4158257361 shutdown_connections 2023-08-25T08:55:26.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 --2- 172.21.15.99:0/4158257361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2440102560 0x7f2440108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:26.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 --2- 172.21.15.99:0/4158257361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24401009c0 0x7f2440100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:26.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 --2- 172.21.15.99:0/4158257361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2440101bc0 0x7f2440102020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:26.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 -- 172.21.15.99:0/4158257361 >> 172.21.15.99:0/4158257361 conn(0x7f24400fbf40 msgr2=0x7f24400fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:55:26.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 -- 172.21.15.99:0/4158257361 shutdown_connections 2023-08-25T08:55:26.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.385+0000 7f2444844700 1 -- 172.21.15.99:0/4158257361 wait complete. 2023-08-25T08:55:26.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.386+0000 7f2444844700 1 Processor -- start 2023-08-25T08:55:26.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.386+0000 7f2444844700 1 -- start start 2023-08-25T08:55:26.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f2444844700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24401009c0 0x7f24401988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:26.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f2444844700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440101bc0 0x7f2440198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:26.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f2444844700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2440102560 0x7f244019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:26.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f2444844700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401a0440 con 0x7f2440102560 2023-08-25T08:55:26.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f2444844700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401a0580 con 0x7f24401009c0 2023-08-25T08:55:26.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f2444844700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401a0710 con 0x7f2440101bc0 2023-08-25T08:55:26.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2440102560 0x7f244019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:26.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24401009c0 0x7f24401988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:26.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2440102560 0x7f244019fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39792/0 (socket says 172.21.15.99:39792) 2023-08-25T08:55:26.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243ed9d700 1 -- 172.21.15.99:0/1587068464 learned_addr learned my addr 172.21.15.99:0/1587068464 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:55:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243dd9b700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440101bc0 0x7f2440198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243ed9d700 1 -- 172.21.15.99:0/1587068464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440101bc0 msgr2=0x7f2440198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243ed9d700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440101bc0 0x7f2440198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243ed9d700 1 -- 172.21.15.99:0/1587068464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24401009c0 msgr2=0x7f24401988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243ed9d700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24401009c0 0x7f24401988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243ed9d700 1 -- 172.21.15.99:0/1587068464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2438009c40 con 0x7f2440102560 2023-08-25T08:55:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.387+0000 7f243dd9b700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440101bc0 0x7f2440198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:55:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.388+0000 7f243ed9d700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2440102560 0x7f244019fe70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f243800c3a0 tx=0x7f243800c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:55:26.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.388+0000 7f242f7fe700 1 -- 172.21.15.99:0/1587068464 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438017070 con 0x7f2440102560 2023-08-25T08:55:26.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.388+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24340095f0 con 0x7f2440102560 2023-08-25T08:55:26.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.388+0000 7f242f7fe700 1 -- 172.21.15.99:0/1587068464 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f243800f040 con 0x7f2440102560 2023-08-25T08:55:26.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.388+0000 7f242f7fe700 1 -- 172.21.15.99:0/1587068464 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800e050 con 0x7f2440102560 2023-08-25T08:55:26.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.388+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401a0d50 con 0x7f2440102560 2023-08-25T08:55:26.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.389+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2440193000 con 0x7f2440102560 2023-08-25T08:55:26.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.391+0000 7f242f7fe700 1 -- 172.21.15.99:0/1587068464 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f2438004560 con 0x7f2440102560 2023-08-25T08:55:26.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.392+0000 7f242f7fe700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f242809ff50 0x7f24280a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:26.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.393+0000 7f242f7fe700 1 -- 172.21.15.99:0/1587068464 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f24380ca9e0 con 0x7f2440102560 2023-08-25T08:55:26.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.393+0000 7f243e59c700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f242809ff50 0x7f24280a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:26.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.393+0000 7f243e59c700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f242809ff50 0x7f24280a2410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2440101a20 tx=0x7f2430009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:55:26.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.395+0000 7f242f7fe700 1 -- 172.21.15.99:0/1587068464 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f2438095060 con 0x7f2440102560 2023-08-25T08:55:26.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:26.605+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f244002cd40 con 0x7f242809ff50 2023-08-25T08:55:27.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:26 smithi099 ceph-mon[97641]: osdmap e60: 9 total, 9 up, 9 in 2023-08-25T08:55:27.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:26 smithi099 ceph-mon[97641]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:26 smithi099 ceph-mon[97641]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:26 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:27.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:26 smithi157 ceph-mon[130879]: osdmap e60: 9 total, 9 up, 9 in 2023-08-25T08:55:27.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:26 smithi157 ceph-mon[130879]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:27.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:26 smithi157 ceph-mon[130879]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:27.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:27.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:27.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:26 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:27.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:26 smithi181 ceph-mon[131296]: osdmap e60: 9 total, 9 up, 9 in 2023-08-25T08:55:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:26 smithi181 ceph-mon[131296]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:26 smithi181 ceph-mon[131296]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:27.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:27.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:27.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:26 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:28.187 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:28 smithi099 ceph-mon[97641]: osdmap e61: 9 total, 9 up, 9 in 2023-08-25T08:55:28.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:28 smithi157 ceph-mon[130879]: osdmap e61: 9 total, 9 up, 9 in 2023-08-25T08:55:28.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:28 smithi181 ceph-mon[131296]: osdmap e61: 9 total, 9 up, 9 in 2023-08-25T08:55:29.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:29 smithi099 ceph-mon[97641]: osdmap e62: 9 total, 9 up, 9 in 2023-08-25T08:55:29.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:29 smithi099 ceph-mon[97641]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:29.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:29 smithi157 ceph-mon[130879]: osdmap e62: 9 total, 9 up, 9 in 2023-08-25T08:55:29.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:29 smithi157 ceph-mon[130879]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:29.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:29 smithi181 ceph-mon[131296]: osdmap e62: 9 total, 9 up, 9 in 2023-08-25T08:55:29.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:29 smithi181 ceph-mon[131296]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:30.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:30 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2160251813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7256a919-8afa-4675-867e-880a61ae8a4f"}]: dispatch 2023-08-25T08:55:30.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:30 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2160251813' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7256a919-8afa-4675-867e-880a61ae8a4f"}]': finished 2023-08-25T08:55:30.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:30 smithi099 ceph-mon[97641]: osdmap e63: 10 total, 9 up, 10 in 2023-08-25T08:55:30.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:30.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:30 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2160251813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7256a919-8afa-4675-867e-880a61ae8a4f"}]: dispatch 2023-08-25T08:55:30.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:30 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2160251813' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7256a919-8afa-4675-867e-880a61ae8a4f"}]': finished 2023-08-25T08:55:30.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:30 smithi157 ceph-mon[130879]: osdmap e63: 10 total, 9 up, 10 in 2023-08-25T08:55:30.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:30.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:30 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2160251813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7256a919-8afa-4675-867e-880a61ae8a4f"}]: dispatch 2023-08-25T08:55:30.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:30 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2160251813' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7256a919-8afa-4675-867e-880a61ae8a4f"}]': finished 2023-08-25T08:55:30.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:30 smithi181 ceph-mon[131296]: osdmap e63: 10 total, 9 up, 10 in 2023-08-25T08:55:30.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:31.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:31 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2326995252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:31.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:31 smithi099 ceph-mon[97641]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-08-25T08:55:31.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:31 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2326995252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:31.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:31 smithi157 ceph-mon[130879]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-08-25T08:55:31.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:31 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2326995252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:31.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:31 smithi181 ceph-mon[131296]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-08-25T08:55:33.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:33 smithi181 ceph-mon[131296]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:55:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:33 smithi099 ceph-mon[97641]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:55:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:33 smithi157 ceph-mon[130879]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:55:35.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:34 smithi099 ceph-mon[97641]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-08-25T08:55:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:34 smithi157 ceph-mon[130879]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-08-25T08:55:35.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:34 smithi181 ceph-mon[131296]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-08-25T08:55:37.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:37 smithi181 ceph-mon[131296]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-08-25T08:55:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:37 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-25T08:55:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:37 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:37.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:37 smithi099 ceph-mon[97641]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-08-25T08:55:37.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:37 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-25T08:55:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:37 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:37 smithi157 ceph-mon[130879]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-08-25T08:55:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:37 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-25T08:55:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:37 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:38.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:38 smithi181 ceph-mon[131296]: Deploying daemon osd.9 on smithi099 2023-08-25T08:55:38.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:38 smithi099 ceph-mon[97641]: Deploying daemon osd.9 on smithi099 2023-08-25T08:55:38.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:38 smithi157 ceph-mon[130879]: Deploying daemon osd.9 on smithi099 2023-08-25T08:55:39.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:39 smithi099 ceph-mon[97641]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-25T08:55:39.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:39 smithi181 ceph-mon[131296]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-25T08:55:39.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:39 smithi157 ceph-mon[130879]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-25T08:55:41.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:41 smithi181 ceph-mon[131296]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:41.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:41 smithi099 ceph-mon[97641]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:41.878 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:41 smithi157 ceph-mon[130879]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:44.203 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:43 smithi099 ceph-mon[97641]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:44.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:44.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:44.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:44.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:43 smithi181 ceph-mon[131296]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:44.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:44.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:44.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:44.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:43 smithi157 ceph-mon[130879]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:44.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:44.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:44.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:45.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:44 smithi099 ceph-mon[97641]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:45.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:44 smithi157 ceph-mon[130879]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:45.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:45 smithi181 ceph-mon[131296]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:46.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.220+0000 7f242f7fe700 1 -- 172.21.15.99:0/1587068464 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f244002cd40 con 0x7f242809ff50 2023-08-25T08:55:46.223 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 9 on host 'smithi099' 2023-08-25T08:55:46.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.223+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f242809ff50 msgr2=0x7f24280a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:46.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.223+0000 7f2444844700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f242809ff50 0x7f24280a2410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2440101a20 tx=0x7f2430009260 comp rx=0 tx=0).stop 2023-08-25T08:55:46.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.223+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2440102560 msgr2=0x7f244019fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:46.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.223+0000 7f2444844700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2440102560 0x7f244019fe70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f243800c3a0 tx=0x7f243800c6b0 comp rx=0 tx=0).stop 2023-08-25T08:55:46.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.224+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 shutdown_connections 2023-08-25T08:55:46.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.224+0000 7f2444844700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2440102560 0x7f244019fe70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:46.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.224+0000 7f2444844700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f242809ff50 0x7f24280a2410 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:46.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.224+0000 7f2444844700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24401009c0 0x7f24401988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:46.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.224+0000 7f2444844700 1 --2- 172.21.15.99:0/1587068464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440101bc0 0x7f2440198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:46.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.224+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 >> 172.21.15.99:0/1587068464 conn(0x7f24400fbf40 msgr2=0x7f24401074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:55:46.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.224+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 shutdown_connections 2023-08-25T08:55:46.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:55:46.224+0000 7f2444844700 1 -- 172.21.15.99:0/1587068464 wait complete. 2023-08-25T08:55:46.684 DEBUG:teuthology.orchestra.run.smithi099:osd.9> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.9.service 2023-08-25T08:55:46.689 INFO:tasks.cephadm:Deploying osd.10 on smithi157 with /dev/vg_nvme/lv_1... 2023-08-25T08:55:46.689 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-25T08:55:46.760 INFO:journalctl@ceph.osd.9.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:55:47.062 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 08:55:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T08:55:46.854+0000 7f552bd39640 -1 osd.9 0 log_to_monitors true 2023-08-25T08:55:47.256 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T08:55:47.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:47 smithi181 ceph-mon[131296]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:47.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:47 smithi181 ceph-mon[131296]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-25T08:55:47.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:47 smithi099 ceph-mon[97641]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:47.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:47 smithi099 ceph-mon[97641]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-25T08:55:47.610 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.611 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.611 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.611 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:47.612 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:47 smithi157 ceph-mon[130879]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:47.612 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:47 smithi157 ceph-mon[130879]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-25T08:55:48.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:48 smithi157 ceph-mon[130879]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-25T08:55:48.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:48 smithi157 ceph-mon[130879]: osdmap e64: 10 total, 9 up, 10 in 2023-08-25T08:55:48.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:48 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:48.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:48 smithi157 ceph-mon[130879]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:55:48.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:48 smithi099 ceph-mon[97641]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-25T08:55:48.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:48 smithi099 ceph-mon[97641]: osdmap e64: 10 total, 9 up, 10 in 2023-08-25T08:55:48.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:48.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:48 smithi099 ceph-mon[97641]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:55:48.563 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 08:55:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T08:55:48.288+0000 7f55213b0700 -1 osd.9 0 waiting for initial osdmap 2023-08-25T08:55:48.564 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 08:55:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T08:55:48.302+0000 7f551919d700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:55:48.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:48 smithi181 ceph-mon[131296]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-25T08:55:48.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:48 smithi181 ceph-mon[131296]: osdmap e64: 10 total, 9 up, 10 in 2023-08-25T08:55:48.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:48.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:48 smithi181 ceph-mon[131296]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-08-25T08:55:48.944 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-25T08:55:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-25T08:55:48.945 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2023-08-25T08:55:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2023-08-25T08:55:48.945 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00826991 s, 1.3 GB/s 2023-08-25T08:55:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2023-08-25T08:55:49.398 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2023-08-25T08:55:49.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:49 smithi099 ceph-mon[97641]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:55:49.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:49 smithi099 ceph-mon[97641]: osdmap e65: 10 total, 9 up, 10 in 2023-08-25T08:55:49.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:49 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:49.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:49 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:49.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:49 smithi099 ceph-mon[97641]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:49 smithi157 ceph-mon[130879]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:55:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:49 smithi157 ceph-mon[130879]: osdmap e65: 10 total, 9 up, 10 in 2023-08-25T08:55:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:49 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:49 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:49 smithi157 ceph-mon[130879]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:49.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:49 smithi181 ceph-mon[131296]: from='osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-08-25T08:55:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:49 smithi181 ceph-mon[131296]: osdmap e65: 10 total, 9 up, 10 in 2023-08-25T08:55:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:49 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:49 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:49 smithi181 ceph-mon[131296]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T08:55:49.955 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T08:55:50.445 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:55:50.445 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:55:50.445 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] boot 2023-08-25T08:55:50.445 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: osdmap e66: 10 total, 10 up, 10 in 2023-08-25T08:55:50.445 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:50.445 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi099 2023-08-25T08:55:50.446 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.446 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.446 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.446 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.446 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.446 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.446 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi099 to 4316M 2023-08-25T08:55:50.447 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.447 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:50.447 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:50.447 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.447 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:50.447 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:50.447 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:50.448 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:55:50.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:55:50.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] boot 2023-08-25T08:55:50.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: osdmap e66: 10 total, 10 up, 10 in 2023-08-25T08:55:50.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:50.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi099 2023-08-25T08:55:50.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi099 to 4316M 2023-08-25T08:55:50.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:50.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:50.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:50.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:50.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:50.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:55:50.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:55:50.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: osd.9 [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] boot 2023-08-25T08:55:50.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: osdmap e66: 10 total, 10 up, 10 in 2023-08-25T08:55:50.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:55:50.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi099 2023-08-25T08:55:50.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:55:50.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi099 to 4316M 2023-08-25T08:55:50.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:50.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:50.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:50.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:55:50.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:50.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:55:50.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:55:51.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 -- 172.21.15.157:0/4269435048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228101bb0 msgr2=0x7fd228102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/4269435048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228101bb0 0x7fd228102010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2240099c0 tx=0x7fd224009cd0 comp rx=0 tx=0).stop 2023-08-25T08:55:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 -- 172.21.15.157:0/4269435048 shutdown_connections 2023-08-25T08:55:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/4269435048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd228102550 0x7fd228108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/4269435048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2281009b0 0x7fd228100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/4269435048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228101bb0 0x7fd228102010 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 -- 172.21.15.157:0/4269435048 >> 172.21.15.157:0/4269435048 conn(0x7fd2280fbf10 msgr2=0x7fd2280fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:55:51.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 -- 172.21.15.157:0/4269435048 shutdown_connections 2023-08-25T08:55:51.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.152+0000 7fd22f2d6700 1 -- 172.21.15.157:0/4269435048 wait complete. 2023-08-25T08:55:51.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22f2d6700 1 Processor -- start 2023-08-25T08:55:51.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22f2d6700 1 -- start start 2023-08-25T08:55:51.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22f2d6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2281009b0 0x7fd2281965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:51.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22f2d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd228101bb0 0x7fd228196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:51.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22f2d6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228102550 0x7fd22819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:51.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22f2d6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22819e170 con 0x7fd2281009b0 2023-08-25T08:55:51.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22f2d6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22819e2b0 con 0x7fd228102550 2023-08-25T08:55:51.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22f2d6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22819e440 con 0x7fd228101bb0 2023-08-25T08:55:51.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228102550 0x7fd22819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:51.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228102550 0x7fd22819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57952/0 (socket says 172.21.15.157:57952) 2023-08-25T08:55:51.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 -- 172.21.15.157:0/3656269464 learned_addr learned my addr 172.21.15.157:0/3656269464 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-08-25T08:55:51.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22c871700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd228101bb0 0x7fd228196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:51.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 -- 172.21.15.157:0/3656269464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd228101bb0 msgr2=0x7fd228196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:51.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d072700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2281009b0 0x7fd2281965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:51.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd228101bb0 0x7fd228196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:51.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 -- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2281009b0 msgr2=0x7fd2281965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:55:51.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2281009b0 0x7fd2281965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:55:51.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 -- 172.21.15.157:0/3656269464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240095f0 con 0x7fd228102550 2023-08-25T08:55:51.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d072700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2281009b0 0x7fd2281965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:55:51.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22c871700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd228101bb0 0x7fd228196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:55:51.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.153+0000 7fd22d873700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228102550 0x7fd22819dba0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd22000b970 tx=0x7fd22000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:55:51.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.154+0000 7fd21e7fc700 1 -- 172.21.15.157:0/3656269464 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22000c9b0 con 0x7fd228102550 2023-08-25T08:55:51.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.154+0000 7fd21e7fc700 1 -- 172.21.15.157:0/3656269464 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd220014440 con 0x7fd228102550 2023-08-25T08:55:51.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.154+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22819e720 con 0x7fd228102550 2023-08-25T08:55:51.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.154+0000 7fd21e7fc700 1 -- 172.21.15.157:0/3656269464 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2200075d0 con 0x7fd228102550 2023-08-25T08:55:51.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.154+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22819ec20 con 0x7fd228102550 2023-08-25T08:55:51.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.155+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd228107850 con 0x7fd228102550 2023-08-25T08:55:51.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.156+0000 7fd21e7fc700 1 -- 172.21.15.157:0/3656269464 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fd2200148d0 con 0x7fd228102550 2023-08-25T08:55:51.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.156+0000 7fd21e7fc700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd21409ffd0 0x7fd2140a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:55:51.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.156+0000 7fd21e7fc700 1 -- 172.21.15.157:0/3656269464 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fd220006e40 con 0x7fd228102550 2023-08-25T08:55:51.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.157+0000 7fd22d072700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd21409ffd0 0x7fd2140a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:55:51.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.157+0000 7fd22d072700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd21409ffd0 0x7fd2140a2490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd218005fd0 tx=0x7fd218005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:55:51.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.160+0000 7fd21e7fc700 1 -- 172.21.15.157:0/3656269464 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd220094a90 con 0x7fd228102550 2023-08-25T08:55:51.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:55:51.355+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd22802ce80 con 0x7fd21409ffd0 2023-08-25T08:55:52.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:51 smithi099 ceph-mon[97641]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:52.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:51 smithi099 ceph-mon[97641]: osdmap e67: 10 total, 10 up, 10 in 2023-08-25T08:55:52.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:51 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:52.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:51 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:52.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:51 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:51 smithi157 ceph-mon[130879]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:51 smithi157 ceph-mon[130879]: osdmap e67: 10 total, 10 up, 10 in 2023-08-25T08:55:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:51 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:52.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:51 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:52.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:51 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:52.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:51 smithi181 ceph-mon[131296]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:52.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:51 smithi181 ceph-mon[131296]: osdmap e67: 10 total, 10 up, 10 in 2023-08-25T08:55:52.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:51 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:55:52.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:51 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:55:52.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:51 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:55:53.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:52 smithi099 ceph-mon[97641]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:53.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:52 smithi157 ceph-mon[130879]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:53.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:52 smithi181 ceph-mon[131296]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:55:54.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:53 smithi099 ceph-mon[97641]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:54.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:53 smithi099 ceph-mon[97641]: from='client.? 172.21.15.157:0/4171185698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]: dispatch 2023-08-25T08:55:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:53 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]: dispatch 2023-08-25T08:55:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:53 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]': finished 2023-08-25T08:55:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:53 smithi099 ceph-mon[97641]: osdmap e68: 11 total, 10 up, 11 in 2023-08-25T08:55:54.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:55:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:53 smithi157 ceph-mon[130879]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:53 smithi157 ceph-mon[130879]: from='client.? 172.21.15.157:0/4171185698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]: dispatch 2023-08-25T08:55:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:53 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]: dispatch 2023-08-25T08:55:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:53 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]': finished 2023-08-25T08:55:54.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:53 smithi157 ceph-mon[130879]: osdmap e68: 11 total, 10 up, 11 in 2023-08-25T08:55:54.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:55:54.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:53 smithi181 ceph-mon[131296]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:53 smithi181 ceph-mon[131296]: from='client.? 172.21.15.157:0/4171185698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]: dispatch 2023-08-25T08:55:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:53 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]: dispatch 2023-08-25T08:55:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:53 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8274d02b-096a-4240-ac33-738e885e7451"}]': finished 2023-08-25T08:55:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:53 smithi181 ceph-mon[131296]: osdmap e68: 11 total, 10 up, 11 in 2023-08-25T08:55:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:55:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:55 smithi099 ceph-mon[97641]: from='client.? 172.21.15.157:0/1821908111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:55 smithi099 ceph-mon[97641]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:55 smithi157 ceph-mon[130879]: from='client.? 172.21.15.157:0/1821908111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:55 smithi157 ceph-mon[130879]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:55.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:55 smithi181 ceph-mon[131296]: from='client.? 172.21.15.157:0/1821908111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:55:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:55 smithi181 ceph-mon[131296]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:57 smithi157 ceph-mon[130879]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:57.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:57 smithi181 ceph-mon[131296]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:57.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:57 smithi099 ceph-mon[97641]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:55:59 smithi181 ceph-mon[131296]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:59.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:55:59 smithi099 ceph-mon[97641]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:55:59.827 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:55:59 smithi157 ceph-mon[130879]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:00.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-25T08:56:00.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:00.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-25T08:56:00.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:00.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-25T08:56:00.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:01.494 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:01 smithi157 ceph-mon[130879]: Deploying daemon osd.10 on smithi157 2023-08-25T08:56:01.495 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:01 smithi157 ceph-mon[130879]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:01.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:01 smithi099 ceph-mon[97641]: Deploying daemon osd.10 on smithi157 2023-08-25T08:56:01.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:01 smithi099 ceph-mon[97641]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:01.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:01 smithi181 ceph-mon[131296]: Deploying daemon osd.10 on smithi157 2023-08-25T08:56:01.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:01 smithi181 ceph-mon[131296]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:03.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:03 smithi099 ceph-mon[97641]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:03 smithi157 ceph-mon[130879]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:03.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:03 smithi181 ceph-mon[131296]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:05 smithi157 ceph-mon[130879]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:05.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:05.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:05 smithi181 ceph-mon[131296]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:05.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:05.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:05.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:05.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:05 smithi099 ceph-mon[97641]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:05.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:05.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.218+0000 7fd21e7fc700 1 -- 172.21.15.157:0/3656269464 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd22802ce80 con 0x7fd21409ffd0 2023-08-25T08:56:07.220 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 10 on host 'smithi157' 2023-08-25T08:56:07.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd21409ffd0 msgr2=0x7fd2140a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:07.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd21409ffd0 0x7fd2140a2490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd218005fd0 tx=0x7fd218005e40 comp rx=0 tx=0).stop 2023-08-25T08:56:07.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228102550 msgr2=0x7fd22819dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:07.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228102550 0x7fd22819dba0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd22000b970 tx=0x7fd22000bd30 comp rx=0 tx=0).stop 2023-08-25T08:56:07.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 shutdown_connections 2023-08-25T08:56:07.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd21409ffd0 0x7fd2140a2490 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:07.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd228102550 0x7fd22819dba0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:07.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2281009b0 0x7fd2281965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:07.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 --2- 172.21.15.157:0/3656269464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd228101bb0 0x7fd228196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:07.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 >> 172.21.15.157:0/3656269464 conn(0x7fd2280fbf10 msgr2=0x7fd2280fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:07.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 shutdown_connections 2023-08-25T08:56:07.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-08-25T08:56:07.220+0000 7fd22f2d6700 1 -- 172.21.15.157:0/3656269464 wait complete. 2023-08-25T08:56:07.661 DEBUG:teuthology.orchestra.run.smithi157:osd.10> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.10.service 2023-08-25T08:56:07.664 INFO:tasks.cephadm:Deploying osd.11 on smithi181 with /dev/vg_nvme/lv_1... 2023-08-25T08:56:07.665 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-25T08:56:07.701 INFO:journalctl@ceph.osd.10.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:56:07.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:07.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:07.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:07.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:07.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:07.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:07.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:07.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:07.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:07 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:08.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:08.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:08.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:08.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.136 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:07 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:56:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:07 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:08.798 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 08:56:08 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T08:56:08.432+0000 7f9a91f74640 -1 osd.10 0 log_to_monitors true 2023-08-25T08:56:09.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:08 smithi157 ceph-mon[130879]: from='osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T08:56:09.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:08 smithi157 ceph-mon[130879]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T08:56:09.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:08 smithi181 ceph-mon[131296]: from='osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T08:56:09.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:08 smithi181 ceph-mon[131296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T08:56:09.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:08 smithi099 ceph-mon[97641]: from='osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T08:56:09.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:08 smithi099 ceph-mon[97641]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T08:56:09.713 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-25T08:56:09.713 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-25T08:56:09.713 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-08-25T08:56:09.713 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-08-25T08:56:09.713 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135455 s, 774 MB/s 2023-08-25T08:56:09.714 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-08-25T08:56:10.081 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:10.081 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-25T08:56:10.081 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: osdmap e69: 11 total, 10 up, 11 in 2023-08-25T08:56:10.082 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:10.082 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:56:10.082 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:56:10.082 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.082 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.082 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.082 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.083 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.083 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.083 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.083 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:10.083 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:10.083 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:09 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2023-08-25T08:56:10.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:10.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-25T08:56:10.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: osdmap e69: 11 total, 10 up, 11 in 2023-08-25T08:56:10.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:10.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:56:10.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:56:10.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:10.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:10.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:09 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:10.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-25T08:56:10.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: osdmap e69: 11 total, 10 up, 11 in 2023-08-25T08:56:10.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:10.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:56:10.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-08-25T08:56:10.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:10.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:10.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:10.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:10.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T08:56:09.897+0000 7f9a875eb700 -1 osd.10 0 waiting for initial osdmap 2023-08-25T08:56:10.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 08:56:09 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T08:56:09.908+0000 7f9a823de700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:56:10.642 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:56:11.118 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:10 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi157 2023-08-25T08:56:11.118 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:10 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi157 to 4292M 2023-08-25T08:56:11.118 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:10 smithi181 ceph-mon[131296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:56:11.119 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:10 smithi181 ceph-mon[131296]: osdmap e70: 11 total, 10 up, 11 in 2023-08-25T08:56:11.119 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:10 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:11.119 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:10 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:11.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:10 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi157 2023-08-25T08:56:11.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:10 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi157 to 4292M 2023-08-25T08:56:11.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:10 smithi099 ceph-mon[97641]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:56:11.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:10 smithi099 ceph-mon[97641]: osdmap e70: 11 total, 10 up, 11 in 2023-08-25T08:56:11.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:11.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:10 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi157 2023-08-25T08:56:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:10 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi157 to 4292M 2023-08-25T08:56:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:10 smithi157 ceph-mon[130879]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-08-25T08:56:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:10 smithi157 ceph-mon[130879]: osdmap e70: 11 total, 10 up, 11 in 2023-08-25T08:56:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:10 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:11.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:10 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:11.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 -- 172.21.15.181:0/2795062919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 msgr2=0x7f36e4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:11.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 --2- 172.21.15.181:0/2795062919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 0x7f36e4100f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009990 tx=0x7f36e0009ca0 comp rx=0 tx=0).stop 2023-08-25T08:56:11.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 -- 172.21.15.181:0/2795062919 shutdown_connections 2023-08-25T08:56:11.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 --2- 172.21.15.181:0/2795062919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101450 0x7f36e4107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:11.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 --2- 172.21.15.181:0/2795062919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e40ff8b0 0x7f36e40ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:11.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 --2- 172.21.15.181:0/2795062919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 0x7f36e4100f10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:11.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 -- 172.21.15.181:0/2795062919 >> 172.21.15.181:0/2795062919 conn(0x7f36e40fae30 msgr2=0x7f36e40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:11.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 -- 172.21.15.181:0/2795062919 shutdown_connections 2023-08-25T08:56:11.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.809+0000 7f36eb764700 1 -- 172.21.15.181:0/2795062919 wait complete. 2023-08-25T08:56:11.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36eb764700 1 Processor -- start 2023-08-25T08:56:11.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36eb764700 1 -- start start 2023-08-25T08:56:11.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36eb764700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e40ff8b0 0x7f36e4068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:11.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36eb764700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 0x7f36e419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:11.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36eb764700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101450 0x7f36e419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:11.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36eb764700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e419e1d0 con 0x7f36e4101450 2023-08-25T08:56:11.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36eb764700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e419e340 con 0x7f36e40ff8b0 2023-08-25T08:56:11.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36eb764700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e419e500 con 0x7f36e4100ab0 2023-08-25T08:56:11.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36e8cff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 0x7f36e419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.810+0000 7f36e8cff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 0x7f36e419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43462/0 (socket says 172.21.15.181:43462) 2023-08-25T08:56:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36e8cff700 1 -- 172.21.15.181:0/808956898 learned_addr learned my addr 172.21.15.181:0/808956898 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:56:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36e9500700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e40ff8b0 0x7f36e4068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36e8cff700 1 -- 172.21.15.181:0/808956898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e40ff8b0 msgr2=0x7f36e4068fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36e8cff700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e40ff8b0 0x7f36e4068fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36e8cff700 1 -- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101450 msgr2=0x7f36e419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36e8cff700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101450 0x7f36e419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36e8cff700 1 -- 172.21.15.181:0/808956898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e00095f0 con 0x7f36e4100ab0 2023-08-25T08:56:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36e8cff700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 0x7f36e419b3a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f36e00036f0 tx=0x7f36e000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36da7fc700 1 -- 172.21.15.181:0/808956898 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e001d070 con 0x7f36e4100ab0 2023-08-25T08:56:11.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36da7fc700 1 -- 172.21.15.181:0/808956898 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36e0003960 con 0x7f36e4100ab0 2023-08-25T08:56:11.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36da7fc700 1 -- 172.21.15.181:0/808956898 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0005690 con 0x7f36e4100ab0 2023-08-25T08:56:11.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e419e780 con 0x7f36e4100ab0 2023-08-25T08:56:11.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.811+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e419ebc0 con 0x7f36e4100ab0 2023-08-25T08:56:11.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.812+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e4106e50 con 0x7f36e4100ab0 2023-08-25T08:56:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.813+0000 7f36da7fc700 1 -- 172.21.15.181:0/808956898 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f36e000fad0 con 0x7f36e4100ab0 2023-08-25T08:56:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.814+0000 7f36da7fc700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f36d009ffd0 0x7f36d00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.814+0000 7f36da7fc700 1 -- 172.21.15.181:0/808956898 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f36e00cc0c0 con 0x7f36e4100ab0 2023-08-25T08:56:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.814+0000 7f36e9500700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f36d009ffd0 0x7f36d00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:11.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.815+0000 7f36e9500700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f36d009ffd0 0x7f36d00a2490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36e4100910 tx=0x7f36d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:11.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:11.817+0000 7f36da7fc700 1 -- 172.21.15.181:0/808956898 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f36e0096470 con 0x7f36e4100ab0 2023-08-25T08:56:12.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:12.016+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f36e402cf70 con 0x7f36d009ffd0 2023-08-25T08:56:12.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:11 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:56:12.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:11 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:56:12.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:11 smithi181 ceph-mon[131296]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:12.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:11 smithi181 ceph-mon[131296]: osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] boot 2023-08-25T08:56:12.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:11 smithi181 ceph-mon[131296]: osdmap e71: 11 total, 11 up, 11 in 2023-08-25T08:56:12.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:12.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:11 smithi181 ceph-mon[131296]: osdmap e72: 11 total, 11 up, 11 in 2023-08-25T08:56:12.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:11 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:56:12.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:11 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:56:12.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:11 smithi099 ceph-mon[97641]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:12.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:11 smithi099 ceph-mon[97641]: osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] boot 2023-08-25T08:56:12.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:11 smithi099 ceph-mon[97641]: osdmap e71: 11 total, 11 up, 11 in 2023-08-25T08:56:12.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:12.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:11 smithi099 ceph-mon[97641]: osdmap e72: 11 total, 11 up, 11 in 2023-08-25T08:56:12.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:11 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:56:12.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:11 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:56:12.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:11 smithi157 ceph-mon[130879]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T08:56:12.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:11 smithi157 ceph-mon[130879]: osd.10 [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] boot 2023-08-25T08:56:12.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:11 smithi157 ceph-mon[130879]: osdmap e71: 11 total, 11 up, 11 in 2023-08-25T08:56:12.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:56:12.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:11 smithi157 ceph-mon[130879]: osdmap e72: 11 total, 11 up, 11 in 2023-08-25T08:56:13.109 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:12 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:56:13.110 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:12 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:56:13.110 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:12 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:13.110 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:12 smithi181 ceph-mon[131296]: osdmap e73: 11 total, 11 up, 11 in 2023-08-25T08:56:13.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:12 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:56:13.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:12 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:56:13.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:12 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:13.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:12 smithi099 ceph-mon[97641]: osdmap e73: 11 total, 11 up, 11 in 2023-08-25T08:56:13.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:12 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T08:56:13.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:12 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T08:56:13.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:12 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:13.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:12 smithi157 ceph-mon[130879]: osdmap e73: 11 total, 11 up, 11 in 2023-08-25T08:56:14.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:13 smithi181 ceph-mon[131296]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:56:14.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:13 smithi181 ceph-mon[131296]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:14.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:13 smithi099 ceph-mon[97641]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:56:14.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:13 smithi099 ceph-mon[97641]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:14.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:13 smithi157 ceph-mon[130879]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:56:14.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:13 smithi157 ceph-mon[130879]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:15.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:14 smithi181 ceph-mon[131296]: osdmap e74: 11 total, 11 up, 11 in 2023-08-25T08:56:15.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:14 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/1099298132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]: dispatch 2023-08-25T08:56:15.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:14 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]: dispatch 2023-08-25T08:56:15.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:14 smithi181 ceph-mon[131296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]': finished 2023-08-25T08:56:15.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:14 smithi181 ceph-mon[131296]: osdmap e75: 12 total, 11 up, 12 in 2023-08-25T08:56:15.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:14 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:15.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:14 smithi099 ceph-mon[97641]: osdmap e74: 11 total, 11 up, 11 in 2023-08-25T08:56:15.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:14 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/1099298132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]: dispatch 2023-08-25T08:56:15.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:14 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]: dispatch 2023-08-25T08:56:15.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:14 smithi099 ceph-mon[97641]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]': finished 2023-08-25T08:56:15.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:14 smithi099 ceph-mon[97641]: osdmap e75: 12 total, 11 up, 12 in 2023-08-25T08:56:15.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:14 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:14 smithi157 ceph-mon[130879]: osdmap e74: 11 total, 11 up, 11 in 2023-08-25T08:56:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:14 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/1099298132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]: dispatch 2023-08-25T08:56:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:14 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]: dispatch 2023-08-25T08:56:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:14 smithi157 ceph-mon[130879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4"}]': finished 2023-08-25T08:56:15.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:14 smithi157 ceph-mon[130879]: osdmap e75: 12 total, 11 up, 12 in 2023-08-25T08:56:15.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:14 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:16.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:15 smithi181 ceph-mon[131296]: pgmap v225: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2023-08-25T08:56:16.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:15 smithi181 ceph-mon[131296]: from='client.? 172.21.15.181:0/1946826092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:56:16.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:15 smithi099 ceph-mon[97641]: pgmap v225: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2023-08-25T08:56:16.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:15 smithi099 ceph-mon[97641]: from='client.? 172.21.15.181:0/1946826092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:56:16.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:15 smithi157 ceph-mon[130879]: pgmap v225: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2023-08-25T08:56:16.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:15 smithi157 ceph-mon[130879]: from='client.? 172.21.15.181:0/1946826092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T08:56:18.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:17 smithi181 ceph-mon[131296]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-08-25T08:56:18.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:17 smithi099 ceph-mon[97641]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-08-25T08:56:18.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:17 smithi157 ceph-mon[130879]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-08-25T08:56:20.227 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:19 smithi181 ceph-mon[131296]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:56:20.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:19 smithi099 ceph-mon[97641]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:56:20.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:19 smithi157 ceph-mon[130879]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T08:56:21.206 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:20 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-25T08:56:21.206 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:20 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:21.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-25T08:56:21.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:20 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:21.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:20 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-25T08:56:21.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:20 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:22.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:21 smithi181 ceph-mon[131296]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-08-25T08:56:22.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:21 smithi181 ceph-mon[131296]: Deploying daemon osd.11 on smithi181 2023-08-25T08:56:22.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:21 smithi099 ceph-mon[97641]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-08-25T08:56:22.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:21 smithi099 ceph-mon[97641]: Deploying daemon osd.11 on smithi181 2023-08-25T08:56:22.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:21 smithi157 ceph-mon[130879]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-08-25T08:56:22.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:21 smithi157 ceph-mon[130879]: Deploying daemon osd.11 on smithi181 2023-08-25T08:56:23.101 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:22 smithi181 ceph-mon[131296]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:23.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:22 smithi099 ceph-mon[97641]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:22 smithi157 ceph-mon[130879]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:25.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:25 smithi181 ceph-mon[131296]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:25.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:25 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:25.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:25 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:25.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:25 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:26.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:25 smithi099 ceph-mon[97641]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:26.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:25 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:25 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:26.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:25 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:25 smithi157 ceph-mon[130879]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:25 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:25 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:25 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:27.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.629+0000 7f36da7fc700 1 -- 172.21.15.181:0/808956898 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f36e402cf70 con 0x7f36d009ffd0 2023-08-25T08:56:27.631 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 11 on host 'smithi181' 2023-08-25T08:56:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.631+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f36d009ffd0 msgr2=0x7f36d00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f36d009ffd0 0x7f36d00a2490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36e4100910 tx=0x7f36d400b410 comp rx=0 tx=0).stop 2023-08-25T08:56:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 msgr2=0x7f36e419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 0x7f36e419b3a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f36e00036f0 tx=0x7f36e000f690 comp rx=0 tx=0).stop 2023-08-25T08:56:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 shutdown_connections 2023-08-25T08:56:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101450 0x7f36e419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f36d009ffd0 0x7f36d00a2490 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e40ff8b0 0x7f36e4068fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 --2- 172.21.15.181:0/808956898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4100ab0 0x7f36e419b3a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 >> 172.21.15.181:0/808956898 conn(0x7f36e40fae30 msgr2=0x7f36e4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 shutdown_connections 2023-08-25T08:56:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:27.632+0000 7f36eb764700 1 -- 172.21.15.181:0/808956898 wait complete. 2023-08-25T08:56:28.020 DEBUG:teuthology.orchestra.run.smithi181:osd.11> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.11.service 2023-08-25T08:56:28.023 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-08-25T08:56:28.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd stat -f json 2023-08-25T08:56:28.056 INFO:journalctl@ceph.osd.11.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:56:28.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:28.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:28.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:28.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:27 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:28.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:28.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:27 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:28.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:28.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:28.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:28.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:28.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:27 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:28.750 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:56:29.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:28 smithi099 ceph-mon[97641]: from='osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T08:56:29.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:28 smithi099 ceph-mon[97641]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T08:56:29.231 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 08:56:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T08:56:28.828+0000 7f6fc0425640 -1 osd.11 0 log_to_monitors true 2023-08-25T08:56:29.231 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:28 smithi181 ceph-mon[131296]: from='osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T08:56:29.231 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:28 smithi181 ceph-mon[131296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T08:56:29.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:28 smithi157 ceph-mon[130879]: from='osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T08:56:29.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:28 smithi157 ceph-mon[130879]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T08:56:30.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:29 smithi099 ceph-mon[97641]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:30.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:29 smithi099 ceph-mon[97641]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-25T08:56:30.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:29 smithi099 ceph-mon[97641]: osdmap e76: 12 total, 11 up, 12 in 2023-08-25T08:56:30.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:29 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:30.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:29 smithi099 ceph-mon[97641]: from='osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:56:30.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:29 smithi099 ceph-mon[97641]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:56:30.092 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 08:56:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T08:56:29.924+0000 7f6fb5a9c700 -1 osd.11 0 waiting for initial osdmap 2023-08-25T08:56:30.093 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 08:56:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T08:56:29.938+0000 7f6fae08a700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T08:56:30.093 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:29 smithi181 ceph-mon[131296]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:30.093 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:29 smithi181 ceph-mon[131296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-25T08:56:30.093 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:29 smithi181 ceph-mon[131296]: osdmap e76: 12 total, 11 up, 12 in 2023-08-25T08:56:30.093 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:29 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:30.093 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:29 smithi181 ceph-mon[131296]: from='osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:56:30.094 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:29 smithi181 ceph-mon[131296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:56:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 -- 172.21.15.99:0/655736424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f115c102c10 msgr2=0x7f115c103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 --2- 172.21.15.99:0/655736424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f115c102c10 0x7f115c103030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11500098f0 tx=0x7f1150009c00 comp rx=0 tx=0).stop 2023-08-25T08:56:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 -- 172.21.15.99:0/655736424 shutdown_connections 2023-08-25T08:56:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 --2- 172.21.15.99:0/655736424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c1047b0 0x7f115c10b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 --2- 172.21.15.99:0/655736424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f115c102c10 0x7f115c103030 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 --2- 172.21.15.99:0/655736424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c103e10 0x7f115c104270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 -- 172.21.15.99:0/655736424 >> 172.21.15.99:0/655736424 conn(0x7f115c0fe1b0 msgr2=0x7f115c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:30.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 -- 172.21.15.99:0/655736424 shutdown_connections 2023-08-25T08:56:30.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.262+0000 7f11630a1700 1 -- 172.21.15.99:0/655736424 wait complete. 2023-08-25T08:56:30.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.263+0000 7f11630a1700 1 Processor -- start 2023-08-25T08:56:30.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f11630a1700 1 -- start start 2023-08-25T08:56:30.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f11630a1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c102c10 0x7f115c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:30.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f11630a1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c103e10 0x7f115c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:30.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f11630a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f115c1047b0 0x7f115c19ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:30.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f11630a1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1a04d0 con 0x7f115c102c10 2023-08-25T08:56:30.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f11630a1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1a0610 con 0x7f115c103e10 2023-08-25T08:56:30.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f11630a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1a07a0 con 0x7f115c1047b0 2023-08-25T08:56:30.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f1160e3d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c102c10 0x7f115c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:30.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f1160e3d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c102c10 0x7f115c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49376/0 (socket says 172.21.15.99:49376) 2023-08-25T08:56:30.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.264+0000 7f1160e3d700 1 -- 172.21.15.99:0/2905683700 learned_addr learned my addr 172.21.15.99:0/2905683700 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:30.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.265+0000 7f116163e700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f115c1047b0 0x7f115c19ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:30.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.265+0000 7f1160e3d700 1 -- 172.21.15.99:0/2905683700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f115c1047b0 msgr2=0x7f115c19ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:30.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.265+0000 7f115bfff700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c103e10 0x7f115c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:30.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.265+0000 7f1160e3d700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f115c1047b0 0x7f115c19ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.265+0000 7f1160e3d700 1 -- 172.21.15.99:0/2905683700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c103e10 msgr2=0x7f115c198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:30.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.265+0000 7f1160e3d700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c103e10 0x7f115c198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.265+0000 7f1160e3d700 1 -- 172.21.15.99:0/2905683700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480095f0 con 0x7f115c102c10 2023-08-25T08:56:30.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.265+0000 7f115bfff700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c103e10 0x7f115c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:56:30.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.266+0000 7f1160e3d700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c102c10 0x7f115c198940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1150009230 tx=0x7f115000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:30.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.266+0000 7f1159ffb700 1 -- 172.21.15.99:0/2905683700 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115001c070 con 0x7f115c102c10 2023-08-25T08:56:30.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.266+0000 7f1159ffb700 1 -- 172.21.15.99:0/2905683700 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115000ba10 con 0x7f115c102c10 2023-08-25T08:56:30.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.266+0000 7f1159ffb700 1 -- 172.21.15.99:0/2905683700 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150003ca0 con 0x7f115c102c10 2023-08-25T08:56:30.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.266+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1150009520 con 0x7f115c102c10 2023-08-25T08:56:30.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.266+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c1a0c50 con 0x7f115c102c10 2023-08-25T08:56:30.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.267+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115c10e460 con 0x7f115c102c10 2023-08-25T08:56:30.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.270+0000 7f1159ffb700 1 -- 172.21.15.99:0/2905683700 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f1150005040 con 0x7f115c102c10 2023-08-25T08:56:30.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.271+0000 7f1159ffb700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f114c09ffd0 0x7f114c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:30.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.271+0000 7f1159ffb700 1 -- 172.21.15.99:0/2905683700 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f11500cbb70 con 0x7f115c102c10 2023-08-25T08:56:30.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.271+0000 7f115bfff700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f114c09ffd0 0x7f114c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:30.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.272+0000 7f115bfff700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f114c09ffd0 0x7f114c0a2490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1148005fd0 tx=0x7f1148009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:30.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.274+0000 7f1159ffb700 1 -- 172.21.15.99:0/2905683700 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1150095eb0 con 0x7f115c102c10 2023-08-25T08:56:30.378 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:29 smithi157 ceph-mon[130879]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:30.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:29 smithi157 ceph-mon[130879]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-25T08:56:30.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:29 smithi157 ceph-mon[130879]: osdmap e76: 12 total, 11 up, 12 in 2023-08-25T08:56:30.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:29 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:30.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:29 smithi157 ceph-mon[130879]: from='osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:56:30.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:29 smithi157 ceph-mon[130879]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-08-25T08:56:30.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.482+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f115c10e6a0 con 0x7f115c102c10 2023-08-25T08:56:30.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.482+0000 7f1159ffb700 1 -- 172.21.15.99:0/2905683700 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f11500994d0 con 0x7f115c102c10 2023-08-25T08:56:30.484 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:30.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f114c09ffd0 msgr2=0x7f114c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:30.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f114c09ffd0 0x7f114c0a2490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1148005fd0 tx=0x7f1148009310 comp rx=0 tx=0).stop 2023-08-25T08:56:30.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c102c10 msgr2=0x7f115c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:30.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c102c10 0x7f115c198940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1150009230 tx=0x7f115000b560 comp rx=0 tx=0).stop 2023-08-25T08:56:30.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 shutdown_connections 2023-08-25T08:56:30.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f115c1047b0 0x7f115c19ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c102c10 0x7f115c198940 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f114c09ffd0 0x7f114c0a2490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 --2- 172.21.15.99:0/2905683700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c103e10 0x7f115c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:30.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 >> 172.21.15.99:0/2905683700 conn(0x7f115c0fe1b0 msgr2=0x7f115c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:30.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 shutdown_connections 2023-08-25T08:56:30.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:30.485+0000 7f11630a1700 1 -- 172.21.15.99:0/2905683700 wait complete. 2023-08-25T08:56:31.021 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1692953770,"num_in_osds":12,"osd_in_since":1692953774,"num_remapped_pgs":0} 2023-08-25T08:56:31.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:56:31.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: osdmap e77: 12 total, 11 up, 12 in 2023-08-25T08:56:31.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:31.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:31.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:31.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:31.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.245 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:30 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2905683700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T08:56:31.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:56:31.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: osdmap e77: 12 total, 11 up, 12 in 2023-08-25T08:56:31.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:31.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:31.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:31.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:31.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:30 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2905683700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T08:56:31.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-08-25T08:56:31.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: osdmap e77: 12 total, 11 up, 12 in 2023-08-25T08:56:31.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:31.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:31.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:56:31.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:31.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:31.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:31.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:30 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2905683700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T08:56:32.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd stat -f json 2023-08-25T08:56:32.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:31 smithi181 ceph-mon[131296]: purged_snaps scrub starts 2023-08-25T08:56:32.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:31 smithi181 ceph-mon[131296]: purged_snaps scrub ok 2023-08-25T08:56:32.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:31 smithi181 ceph-mon[131296]: Detected new or changed devices on smithi181 2023-08-25T08:56:32.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:31 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi181 to 4292M 2023-08-25T08:56:32.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:31 smithi181 ceph-mon[131296]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:32.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:31 smithi181 ceph-mon[131296]: osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] boot 2023-08-25T08:56:32.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:31 smithi181 ceph-mon[131296]: osdmap e78: 12 total, 12 up, 12 in 2023-08-25T08:56:32.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:31 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:32.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:31 smithi099 ceph-mon[97641]: purged_snaps scrub starts 2023-08-25T08:56:32.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:31 smithi099 ceph-mon[97641]: purged_snaps scrub ok 2023-08-25T08:56:32.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:31 smithi099 ceph-mon[97641]: Detected new or changed devices on smithi181 2023-08-25T08:56:32.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:31 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi181 to 4292M 2023-08-25T08:56:32.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:31 smithi099 ceph-mon[97641]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:32.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:31 smithi099 ceph-mon[97641]: osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] boot 2023-08-25T08:56:32.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:31 smithi099 ceph-mon[97641]: osdmap e78: 12 total, 12 up, 12 in 2023-08-25T08:56:32.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:31 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:32.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:31 smithi157 ceph-mon[130879]: purged_snaps scrub starts 2023-08-25T08:56:32.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:31 smithi157 ceph-mon[130879]: purged_snaps scrub ok 2023-08-25T08:56:32.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:31 smithi157 ceph-mon[130879]: Detected new or changed devices on smithi181 2023-08-25T08:56:32.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:31 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi181 to 4292M 2023-08-25T08:56:32.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:31 smithi157 ceph-mon[130879]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T08:56:32.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:31 smithi157 ceph-mon[130879]: osd.11 [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] boot 2023-08-25T08:56:32.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:31 smithi157 ceph-mon[130879]: osdmap e78: 12 total, 12 up, 12 in 2023-08-25T08:56:32.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:31 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:56:32.745 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:56:33.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:33 smithi099 ceph-mon[97641]: osdmap e79: 12 total, 12 up, 12 in 2023-08-25T08:56:33.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:33 smithi099 ceph-mon[97641]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:33.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:33 smithi181 ceph-mon[131296]: osdmap e79: 12 total, 12 up, 12 in 2023-08-25T08:56:33.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:33 smithi181 ceph-mon[131296]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:33 smithi157 ceph-mon[130879]: osdmap e79: 12 total, 12 up, 12 in 2023-08-25T08:56:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:33 smithi157 ceph-mon[130879]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:34.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 -- 172.21.15.99:0/2621200022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64102550 msgr2=0x7faa64108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:34.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 --2- 172.21.15.99:0/2621200022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64102550 0x7faa64108e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00b0d0 tx=0x7faa5c00b3e0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 -- 172.21.15.99:0/2621200022 shutdown_connections 2023-08-25T08:56:34.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 --2- 172.21.15.99:0/2621200022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64102550 0x7faa64108e00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 --2- 172.21.15.99:0/2621200022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa641009b0 0x7faa64100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 --2- 172.21.15.99:0/2621200022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa64101bb0 0x7faa64102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 -- 172.21.15.99:0/2621200022 >> 172.21.15.99:0/2621200022 conn(0x7faa640fbf10 msgr2=0x7faa640fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:34.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 -- 172.21.15.99:0/2621200022 shutdown_connections 2023-08-25T08:56:34.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.251+0000 7faa6a56e700 1 -- 172.21.15.99:0/2621200022 wait complete. 2023-08-25T08:56:34.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.252+0000 7faa6a56e700 1 Processor -- start 2023-08-25T08:56:34.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa6a56e700 1 -- start start 2023-08-25T08:56:34.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa6a56e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa641009b0 0x7faa640746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:34.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa6a56e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64101bb0 0x7faa64072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:34.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa6a56e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa64102550 0x7faa64073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:34.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa6a56e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa640739d0 con 0x7faa64101bb0 2023-08-25T08:56:34.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa6a56e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa64073b40 con 0x7faa641009b0 2023-08-25T08:56:34.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa6a56e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa641a0810 con 0x7faa64102550 2023-08-25T08:56:34.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa63fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa641009b0 0x7faa640746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:34.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa637fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64101bb0 0x7faa64072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:34.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa63fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa641009b0 0x7faa640746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:56218/0 (socket says 172.21.15.99:56218) 2023-08-25T08:56:34.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.253+0000 7faa63fff700 1 -- 172.21.15.99:0/3923148441 learned_addr learned my addr 172.21.15.99:0/3923148441 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:34.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa68b0b700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa64102550 0x7faa64073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:34.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa63fff700 1 -- 172.21.15.99:0/3923148441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa64102550 msgr2=0x7faa64073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:34.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa63fff700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa64102550 0x7faa64073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa63fff700 1 -- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64101bb0 msgr2=0x7faa64072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:34.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa63fff700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64101bb0 0x7faa64072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa63fff700 1 -- 172.21.15.99:0/3923148441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c009d00 con 0x7faa641009b0 2023-08-25T08:56:34.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa637fe700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64101bb0 0x7faa64072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:56:34.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa63fff700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa641009b0 0x7faa640746c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e970 tx=0x7faa5400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:34.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa617fa700 1 -- 172.21.15.99:0/3923148441 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54009790 con 0x7faa641009b0 2023-08-25T08:56:34.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa617fa700 1 -- 172.21.15.99:0/3923148441 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa5400c7e0 con 0x7faa641009b0 2023-08-25T08:56:34.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa641a0af0 con 0x7faa641009b0 2023-08-25T08:56:34.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.254+0000 7faa617fa700 1 -- 172.21.15.99:0/3923148441 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54003c70 con 0x7faa641009b0 2023-08-25T08:56:34.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.255+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa641a0ff0 con 0x7faa641009b0 2023-08-25T08:56:34.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.256+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa64107ef0 con 0x7faa641009b0 2023-08-25T08:56:34.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.257+0000 7faa617fa700 1 -- 172.21.15.99:0/3923148441 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7faa5400cc50 con 0x7faa641009b0 2023-08-25T08:56:34.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.259+0000 7faa617fa700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faa4c09ff50 0x7faa4c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:34.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.259+0000 7faa617fa700 1 -- 172.21.15.99:0/3923148441 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faa54014070 con 0x7faa641009b0 2023-08-25T08:56:34.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.259+0000 7faa637fe700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faa4c09ff50 0x7faa4c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:34.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.260+0000 7faa637fe700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faa4c09ff50 0x7faa4c0a2410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa58007fd0 tx=0x7faa580073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:34.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.261+0000 7faa617fa700 1 -- 172.21.15.99:0/3923148441 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7faa54095310 con 0x7faa641009b0 2023-08-25T08:56:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.463+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7faa6410c210 con 0x7faa641009b0 2023-08-25T08:56:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.464+0000 7faa617fa700 1 -- 172.21.15.99:0/3923148441 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7faa54098930 con 0x7faa641009b0 2023-08-25T08:56:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:34.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faa4c09ff50 msgr2=0x7faa4c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:34.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faa4c09ff50 0x7faa4c0a2410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa58007fd0 tx=0x7faa580073d0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa641009b0 msgr2=0x7faa640746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:34.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa641009b0 0x7faa640746c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e970 tx=0x7faa5400ec80 comp rx=0 tx=0).stop 2023-08-25T08:56:34.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 shutdown_connections 2023-08-25T08:56:34.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa64102550 0x7faa64073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faa4c09ff50 0x7faa4c0a2410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa641009b0 0x7faa640746c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 --2- 172.21.15.99:0/3923148441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa64101bb0 0x7faa64072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:34.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.466+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 >> 172.21.15.99:0/3923148441 conn(0x7faa640fbf10 msgr2=0x7faa64105030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:34.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.467+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 shutdown_connections 2023-08-25T08:56:34.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:34.467+0000 7faa6a56e700 1 -- 172.21.15.99:0/3923148441 wait complete. 2023-08-25T08:56:34.913 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1692953790,"num_in_osds":12,"osd_in_since":1692953774,"num_remapped_pgs":0} 2023-08-25T08:56:34.914 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd dump --format=json 2023-08-25T08:56:35.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:35 smithi099 ceph-mon[97641]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:35.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:35 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3923148441' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T08:56:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:35 smithi157 ceph-mon[130879]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:35 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3923148441' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T08:56:35.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:35 smithi181 ceph-mon[131296]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:35 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3923148441' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T08:56:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:56:37.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 -- 172.21.15.99:0/1285274149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d3410d180 msgr2=0x7f7d3410f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:37.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 --2- 172.21.15.99:0/1285274149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d3410d180 0x7f7d3410f5f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00b0a0 tx=0x7f7d2c00b3b0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 -- 172.21.15.99:0/1285274149 shutdown_connections 2023-08-25T08:56:37.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 --2- 172.21.15.99:0/1285274149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d3410d180 0x7f7d3410f5f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 --2- 172.21.15.99:0/1285274149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d34072d50 0x7f7d34073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 --2- 172.21.15.99:0/1285274149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d34073670 0x7f7d3410cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 -- 172.21.15.99:0/1285274149 >> 172.21.15.99:0/1285274149 conn(0x7f7d340fc410 msgr2=0x7f7d340fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:37.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 -- 172.21.15.99:0/1285274149 shutdown_connections 2023-08-25T08:56:37.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.218+0000 7f7d38b76700 1 -- 172.21.15.99:0/1285274149 wait complete. 2023-08-25T08:56:37.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.219+0000 7f7d38b76700 1 Processor -- start 2023-08-25T08:56:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.219+0000 7f7d38b76700 1 -- start start 2023-08-25T08:56:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.219+0000 7f7d38b76700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d34072d50 0x7f7d34198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.219+0000 7f7d38b76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d34073670 0x7f7d34199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.219+0000 7f7d38b76700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3410d180 0x7f7d3419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.219+0000 7f7d38b76700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3419dd80 con 0x7f7d34072d50 2023-08-25T08:56:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.219+0000 7f7d38b76700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3419def0 con 0x7f7d3410d180 2023-08-25T08:56:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.219+0000 7f7d38b76700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3419e0b0 con 0x7f7d34073670 2023-08-25T08:56:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d34073670 0x7f7d34199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d34072d50 0x7f7d34198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d34072d50 0x7f7d34198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45440/0 (socket says 172.21.15.99:45440) 2023-08-25T08:56:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d34073670 0x7f7d34199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:54218/0 (socket says 172.21.15.99:54218) 2023-08-25T08:56:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 -- 172.21.15.99:0/279199315 learned_addr learned my addr 172.21.15.99:0/279199315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d32d9d700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3410d180 0x7f7d3419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 -- 172.21.15.99:0/279199315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d34073670 msgr2=0x7f7d34199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d34073670 0x7f7d34199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 -- 172.21.15.99:0/279199315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3410d180 msgr2=0x7f7d3419d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:37.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3410d180 0x7f7d3419d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 -- 172.21.15.99:0/279199315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c009d00 con 0x7f7d34072d50 2023-08-25T08:56:37.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d32d9d700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3410d180 0x7f7d3419d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:56:37.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d3259c700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d34072d50 0x7f7d34198dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e9a0 tx=0x7f7d2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:37.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.220+0000 7f7d237fe700 1 -- 172.21.15.99:0/279199315 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24009790 con 0x7f7d34072d50 2023-08-25T08:56:37.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.221+0000 7f7d237fe700 1 -- 172.21.15.99:0/279199315 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2400cbd0 con 0x7f7d34072d50 2023-08-25T08:56:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.221+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3419e390 con 0x7f7d34072d50 2023-08-25T08:56:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.221+0000 7f7d237fe700 1 -- 172.21.15.99:0/279199315 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24018680 con 0x7f7d34072d50 2023-08-25T08:56:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.221+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d341a5bf0 con 0x7f7d34072d50 2023-08-25T08:56:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.222+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3404e620 con 0x7f7d34072d50 2023-08-25T08:56:37.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.224+0000 7f7d237fe700 1 -- 172.21.15.99:0/279199315 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f7d2400cd40 con 0x7f7d34072d50 2023-08-25T08:56:37.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.225+0000 7f7d237fe700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:37.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.225+0000 7f7d237fe700 1 -- 172.21.15.99:0/279199315 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7d24014070 con 0x7f7d34072d50 2023-08-25T08:56:37.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.225+0000 7f7d31d9b700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:37.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.226+0000 7f7d31d9b700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d3419a3e0 tx=0x7f7d28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.229+0000 7f7d237fe700 1 -- 172.21.15.99:0/279199315 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7d24051e00 con 0x7f7d34072d50 2023-08-25T08:56:37.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.425+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d34112c50 con 0x7f7d34072d50 2023-08-25T08:56:37.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.426+0000 7f7d237fe700 1 -- 172.21.15.99:0/279199315 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18422 (secure 0 0 0) 0x7f7d24095270 con 0x7f7d34072d50 2023-08-25T08:56:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:37.428 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":79,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:56:32.216885+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:56:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7d1c09ff50 msgr2=0x7f7d1c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d3419a3e0 tx=0x7f7d28009260 comp rx=0 tx=0).stop 2023-08-25T08:56:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d34072d50 msgr2=0x7f7d34198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d34072d50 0x7f7d34198dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e9a0 tx=0x7f7d2400ecb0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 shutdown_connections 2023-08-25T08:56:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3410d180 0x7f7d3419d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d34072d50 0x7f7d34198dc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 --2- 172.21.15.99:0/279199315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d34073670 0x7f7d34199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:37.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 >> 172.21.15.99:0/279199315 conn(0x7f7d340fc410 msgr2=0x7f7d3410f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:37.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 shutdown_connections 2023-08-25T08:56:37.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:37.429+0000 7f7d38b76700 1 -- 172.21.15.99:0/279199315 wait complete. 2023-08-25T08:56:37.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:37 smithi181 ceph-mon[131296]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:37 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/279199315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:56:37.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:37 smithi099 ceph-mon[97641]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:37 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/279199315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:56:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:37 smithi157 ceph-mon[130879]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:37 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/279199315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:56:37.915 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T08:56:37.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-08-25T08:56:38.632 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:56:39.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:39 smithi181 ceph-mon[131296]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:39 smithi099 ceph-mon[97641]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:39 smithi157 ceph-mon[130879]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 -- 172.21.15.99:0/227332867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 msgr2=0x7f8ce41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/227332867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 0x7f8ce41031b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009990 tx=0x7f8cd4009ca0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 -- 172.21.15.99:0/227332867 shutdown_connections 2023-08-25T08:56:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/227332867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce4107eb0 0x7f8ce410fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/227332867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 0x7f8ce41031b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/227332867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ce4103780 0x7f8ce41077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 -- 172.21.15.99:0/227332867 >> 172.21.15.99:0/227332867 conn(0x7f8ce40fe620 msgr2=0x7f8ce4100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 -- 172.21.15.99:0/227332867 shutdown_connections 2023-08-25T08:56:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.136+0000 7f8ce91ba700 1 -- 172.21.15.99:0/227332867 wait complete. 2023-08-25T08:56:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.137+0000 7f8ce91ba700 1 Processor -- start 2023-08-25T08:56:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.137+0000 7f8ce91ba700 1 -- start start 2023-08-25T08:56:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce91ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 0x7f8ce419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce91ba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ce4103780 0x7f8ce419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce91ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce4107eb0 0x7f8ce4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:40.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce91ba700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4199140 con 0x7f8ce4102dd0 2023-08-25T08:56:40.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce91ba700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce41992b0 con 0x7f8ce4103780 2023-08-25T08:56:40.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce91ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4199470 con 0x7f8ce4107eb0 2023-08-25T08:56:40.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 0x7f8ce419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:40.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 0x7f8ce419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45454/0 (socket says 172.21.15.99:45454) 2023-08-25T08:56:40.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce2d9d700 1 -- 172.21.15.99:0/3037846998 learned_addr learned my addr 172.21.15.99:0/3037846998 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:40.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce359e700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce4107eb0 0x7f8ce4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce259c700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ce4103780 0x7f8ce419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce2d9d700 1 -- 172.21.15.99:0/3037846998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce4107eb0 msgr2=0x7f8ce4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce2d9d700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce4107eb0 0x7f8ce4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce2d9d700 1 -- 172.21.15.99:0/3037846998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ce4103780 msgr2=0x7f8ce419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce2d9d700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ce4103780 0x7f8ce419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.138+0000 7f8ce2d9d700 1 -- 172.21.15.99:0/3037846998 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd40095f0 con 0x7f8ce4102dd0 2023-08-25T08:56:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.139+0000 7f8ce2d9d700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 0x7f8ce419ea50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40095c0 tx=0x7f8cd400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.139+0000 7f8ccbfff700 1 -- 172.21.15.99:0/3037846998 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd401d070 con 0x7f8ce4102dd0 2023-08-25T08:56:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.139+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce41996f0 con 0x7f8ce4102dd0 2023-08-25T08:56:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.139+0000 7f8ccbfff700 1 -- 172.21.15.99:0/3037846998 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cd4005050 con 0x7f8ce4102dd0 2023-08-25T08:56:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.139+0000 7f8ccbfff700 1 -- 172.21.15.99:0/3037846998 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4004a90 con 0x7f8ce4102dd0 2023-08-25T08:56:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.139+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce41a5a40 con 0x7f8ce4102dd0 2023-08-25T08:56:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.140+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce404e620 con 0x7f8ce4102dd0 2023-08-25T08:56:40.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.142+0000 7f8ccbfff700 1 -- 172.21.15.99:0/3037846998 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f8cd4005430 con 0x7f8ce4102dd0 2023-08-25T08:56:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.144+0000 7f8ccbfff700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8ccc09ff50 0x7f8ccc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.144+0000 7f8ccbfff700 1 -- 172.21.15.99:0/3037846998 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8cd40cbce0 con 0x7f8ce4102dd0 2023-08-25T08:56:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.144+0000 7f8ce259c700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8ccc09ff50 0x7f8ccc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.145+0000 7f8ce259c700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8ccc09ff50 0x7f8ccc0a2410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40febe0 tx=0x7f8cd800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:40.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.148+0000 7f8ccbfff700 1 -- 172.21.15.99:0/3037846998 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8cd4095f00 con 0x7f8ce4102dd0 2023-08-25T08:56:40.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.346+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ce4199fe0 con 0x7f8ce4102dd0 2023-08-25T08:56:40.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.346+0000 7f8ccbfff700 1 -- 172.21.15.99:0/3037846998 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cd4099520 con 0x7f8ce4102dd0 2023-08-25T08:56:40.348 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T08:56:40.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.348+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8ccc09ff50 msgr2=0x7f8ccc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:40.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.348+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8ccc09ff50 0x7f8ccc0a2410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40febe0 tx=0x7f8cd800b410 comp rx=0 tx=0).stop 2023-08-25T08:56:40.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.348+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 msgr2=0x7f8ce419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:40.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.348+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 0x7f8ce419ea50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40095c0 tx=0x7f8cd400f690 comp rx=0 tx=0).stop 2023-08-25T08:56:40.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.349+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 shutdown_connections 2023-08-25T08:56:40.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.349+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce4107eb0 0x7f8ce4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.349+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce4102dd0 0x7f8ce419ea50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.349+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8ccc09ff50 0x7f8ccc0a2410 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.349+0000 7f8ce91ba700 1 --2- 172.21.15.99:0/3037846998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ce4103780 0x7f8ce419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:40.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.349+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 >> 172.21.15.99:0/3037846998 conn(0x7f8ce40fe620 msgr2=0x7f8ce40ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:40.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.349+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 shutdown_connections 2023-08-25T08:56:40.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:40.349+0000 7f8ce91ba700 1 -- 172.21.15.99:0/3037846998 wait complete. 2023-08-25T08:56:40.772 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:40 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3037846998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:56:40.802 INFO:tasks.cephadm:Adding mds.a on smithi099 2023-08-25T08:56:40.803 INFO:tasks.cephadm:Adding mds.d on smithi099 2023-08-25T08:56:40.803 INFO:tasks.cephadm:Adding mds.g on smithi099 2023-08-25T08:56:40.803 INFO:tasks.cephadm:Adding mds.j on smithi099 2023-08-25T08:56:40.804 INFO:tasks.cephadm:Adding mds.b on smithi157 2023-08-25T08:56:40.804 INFO:tasks.cephadm:Adding mds.e on smithi157 2023-08-25T08:56:40.804 INFO:tasks.cephadm:Adding mds.h on smithi157 2023-08-25T08:56:40.804 INFO:tasks.cephadm:Adding mds.k on smithi157 2023-08-25T08:56:40.804 INFO:tasks.cephadm:Adding mds.c on smithi181 2023-08-25T08:56:40.805 INFO:tasks.cephadm:Adding mds.f on smithi181 2023-08-25T08:56:40.805 INFO:tasks.cephadm:Adding mds.i on smithi181 2023-08-25T08:56:40.805 INFO:tasks.cephadm:Adding mds.l on smithi181 2023-08-25T08:56:40.805 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph orch apply mds all '12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l' 2023-08-25T08:56:40.838 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:40 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3037846998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:56:40.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:40 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3037846998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:56:41.374 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T08:56:41.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:41 smithi099 ceph-mon[97641]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:41.855 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:41 smithi181 ceph-mon[131296]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:41 smithi157 ceph-mon[130879]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:42.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1097358613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4100ab0 msgr2=0x7fc9f4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:42.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1097358613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4100ab0 0x7fc9f4100f10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009990 tx=0x7fc9f0009ca0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1097358613 shutdown_connections 2023-08-25T08:56:42.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1097358613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f4101450 0x7fc9f4107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1097358613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f40ff8b0 0x7fc9f40ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1097358613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4100ab0 0x7fc9f4100f10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1097358613 >> 172.21.15.181:0/1097358613 conn(0x7fc9f40fae30 msgr2=0x7fc9f40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:42.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1097358613 shutdown_connections 2023-08-25T08:56:42.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.561+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1097358613 wait complete. 2023-08-25T08:56:42.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9fb98e700 1 Processor -- start 2023-08-25T08:56:42.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9fb98e700 1 -- start start 2023-08-25T08:56:42.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9fb98e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f40ff8b0 0x7fc9f4103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:42.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9fb98e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f4100ab0 0x7fc9f4101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:42.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9fb98e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4101450 0x7fc9f41023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9fb98e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f4102b50 con 0x7fc9f4100ab0 2023-08-25T08:56:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9fb98e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f4102cc0 con 0x7fc9f40ff8b0 2023-08-25T08:56:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9fb98e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f41a0670 con 0x7fc9f4101450 2023-08-25T08:56:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9f9f2b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4101450 0x7fc9f41023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9f9f2b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4101450 0x7fc9f41023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43586/0 (socket says 172.21.15.181:43586) 2023-08-25T08:56:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9f9f2b700 1 -- 172.21.15.181:0/1730287384 learned_addr learned my addr 172.21.15.181:0/1730287384 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-08-25T08:56:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.562+0000 7fc9f972a700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f40ff8b0 0x7fc9f4103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9f8f29700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f4100ab0 0x7fc9f4101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9f9f2b700 1 -- 172.21.15.181:0/1730287384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f40ff8b0 msgr2=0x7fc9f4103840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9f9f2b700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f40ff8b0 0x7fc9f4103840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9f9f2b700 1 -- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f4100ab0 msgr2=0x7fc9f4101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9f9f2b700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f4100ab0 0x7fc9f4101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9f9f2b700 1 -- 172.21.15.181:0/1730287384 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00095f0 con 0x7fc9f4101450 2023-08-25T08:56:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9f9f2b700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4101450 0x7fc9f41023d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00ada0 tx=0x7fc9ec00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9ea7fc700 1 -- 172.21.15.181:0/1730287384 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00f070 con 0x7fc9f4101450 2023-08-25T08:56:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9ea7fc700 1 -- 172.21.15.181:0/1730287384 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9ec007b30 con 0x7fc9f4101450 2023-08-25T08:56:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f41a0950 con 0x7fc9f4101450 2023-08-25T08:56:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9ea7fc700 1 -- 172.21.15.181:0/1730287384 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec005440 con 0x7fc9f4101450 2023-08-25T08:56:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.563+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f41a0e50 con 0x7fc9f4101450 2023-08-25T08:56:42.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.564+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f410b0b0 con 0x7fc9f4101450 2023-08-25T08:56:42.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.565+0000 7fc9ea7fc700 1 -- 172.21.15.181:0/1730287384 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fc9ec004ad0 con 0x7fc9f4101450 2023-08-25T08:56:42.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.566+0000 7fc9ea7fc700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc9e009ffd0 0x7fc9e00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:42.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.566+0000 7fc9ea7fc700 1 -- 172.21.15.181:0/1730287384 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc9ec0caa40 con 0x7fc9f4101450 2023-08-25T08:56:42.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.566+0000 7fc9f972a700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc9e009ffd0 0x7fc9e00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:42.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.567+0000 7fc9f972a700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc9e009ffd0 0x7fc9e00a2490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4005fd0 tx=0x7fc9e4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:42.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.569+0000 7fc9ea7fc700 1 -- 172.21.15.181:0/1730287384 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc9ec094c60 con 0x7fc9f4101450 2023-08-25T08:56:42.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.766+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9f4068cd0 con 0x7fc9e009ffd0 2023-08-25T08:56:42.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.776+0000 7fc9ea7fc700 1 -- 172.21.15.181:0/1730287384 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc9f4068cd0 con 0x7fc9e009ffd0 2023-08-25T08:56:42.777 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mds.all update... 2023-08-25T08:56:42.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc9e009ffd0 msgr2=0x7fc9e00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc9e009ffd0 0x7fc9e00a2490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4005fd0 tx=0x7fc9e4005e50 comp rx=0 tx=0).stop 2023-08-25T08:56:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4101450 msgr2=0x7fc9f41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4101450 0x7fc9f41023d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00ada0 tx=0x7fc9ec00c330 comp rx=0 tx=0).stop 2023-08-25T08:56:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 shutdown_connections 2023-08-25T08:56:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f4101450 0x7fc9f41023d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc9e009ffd0 0x7fc9e00a2490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f40ff8b0 0x7fc9f4103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 --2- 172.21.15.181:0/1730287384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f4100ab0 0x7fc9f4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 >> 172.21.15.181:0/1730287384 conn(0x7fc9f40fae30 msgr2=0x7fc9f40684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 shutdown_connections 2023-08-25T08:56:42.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-08-25T08:56:42.779+0000 7fc9fb98e700 1 -- 172.21.15.181:0/1730287384 wait complete. 2023-08-25T08:56:43.221 DEBUG:teuthology.orchestra.run.smithi099:mds.a> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.a.service 2023-08-25T08:56:43.226 DEBUG:teuthology.orchestra.run.smithi099:mds.d> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.d.service 2023-08-25T08:56:43.271 DEBUG:teuthology.orchestra.run.smithi099:mds.g> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.g.service 2023-08-25T08:56:43.306 INFO:journalctl@ceph.mds.a.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:56:43.309 DEBUG:teuthology.orchestra.run.smithi099:mds.j> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.j.service 2023-08-25T08:56:43.343 INFO:journalctl@ceph.mds.d.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:56:43.346 DEBUG:teuthology.orchestra.run.smithi157:mds.b> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.b.service 2023-08-25T08:56:43.349 DEBUG:teuthology.orchestra.run.smithi157:mds.e> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.e.service 2023-08-25T08:56:43.380 INFO:journalctl@ceph.mds.g.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:56:43.386 INFO:journalctl@ceph.mds.b.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:56:43.388 DEBUG:teuthology.orchestra.run.smithi157:mds.h> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.h.service 2023-08-25T08:56:43.414 INFO:journalctl@ceph.mds.j.smithi099.stdout:-- Logs begin at Fri 2023-08-25 08:32:57 UTC. -- 2023-08-25T08:56:43.417 INFO:journalctl@ceph.mds.e.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:56:43.418 DEBUG:teuthology.orchestra.run.smithi157:mds.k> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.k.service 2023-08-25T08:56:43.446 INFO:journalctl@ceph.mds.h.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:56:43.446 DEBUG:teuthology.orchestra.run.smithi181:mds.c> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.c.service 2023-08-25T08:56:43.449 DEBUG:teuthology.orchestra.run.smithi181:mds.f> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.f.service 2023-08-25T08:56:43.473 INFO:journalctl@ceph.mds.k.smithi157.stdout:-- Logs begin at Fri 2023-08-25 08:29:35 UTC. -- 2023-08-25T08:56:43.479 INFO:journalctl@ceph.mds.c.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:56:43.481 DEBUG:teuthology.orchestra.run.smithi181:mds.i> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.i.service 2023-08-25T08:56:43.509 INFO:journalctl@ceph.mds.f.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:56:43.510 DEBUG:teuthology.orchestra.run.smithi181:mds.l> sudo journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.l.service 2023-08-25T08:56:43.536 INFO:journalctl@ceph.mds.i.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:56:43.537 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-08-25T08:56:43.538 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:56:43.562 INFO:journalctl@ceph.mds.l.smithi181.stdout:-- Logs begin at Fri 2023-08-25 08:29:03 UTC. -- 2023-08-25T08:56:43.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 -- 172.21.15.99:0/2771428710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 msgr2=0x7efc70101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 --2- 172.21.15.99:0/2771428710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 0x7efc70101f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc600099c0 tx=0x7efc60009cd0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 -- 172.21.15.99:0/2771428710 shutdown_connections 2023-08-25T08:56:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 --2- 172.21.15.99:0/2771428710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc701024c0 0x7efc70108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 --2- 172.21.15.99:0/2771428710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc70100930 0x7efc70100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 --2- 172.21.15.99:0/2771428710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 0x7efc70101f80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 -- 172.21.15.99:0/2771428710 >> 172.21.15.99:0/2771428710 conn(0x7efc700fbe60 msgr2=0x7efc700fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 -- 172.21.15.99:0/2771428710 shutdown_connections 2023-08-25T08:56:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.749+0000 7efc773ff700 1 -- 172.21.15.99:0/2771428710 wait complete. 2023-08-25T08:56:43.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.750+0000 7efc773ff700 1 Processor -- start 2023-08-25T08:56:43.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.750+0000 7efc773ff700 1 -- start start 2023-08-25T08:56:43.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc773ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc70100930 0x7efc701987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:43.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc773ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 0x7efc70198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:43.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc773ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc701024c0 0x7efc7019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:43.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc773ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc701a0360 con 0x7efc70101b20 2023-08-25T08:56:43.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc773ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc701a04a0 con 0x7efc701024c0 2023-08-25T08:56:43.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc773ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc701a0630 con 0x7efc70100930 2023-08-25T08:56:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7499a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 0x7efc70198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7499a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 0x7efc70198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45478/0 (socket says 172.21.15.99:45478) 2023-08-25T08:56:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7499a700 1 -- 172.21.15.99:0/1774859539 learned_addr learned my addr 172.21.15.99:0/1774859539 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7519b700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc70100930 0x7efc701987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7599c700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc701024c0 0x7efc7019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7499a700 1 -- 172.21.15.99:0/1774859539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc70100930 msgr2=0x7efc701987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7499a700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc70100930 0x7efc701987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7499a700 1 -- 172.21.15.99:0/1774859539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc701024c0 msgr2=0x7efc7019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7499a700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc701024c0 0x7efc7019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7499a700 1 -- 172.21.15.99:0/1774859539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc600095f0 con 0x7efc70101b20 2023-08-25T08:56:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7599c700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc701024c0 0x7efc7019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:56:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.751+0000 7efc7519b700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc70100930 0x7efc701987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:56:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.752+0000 7efc7499a700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 0x7efc70198d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc6000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.752+0000 7efc667fc700 1 -- 172.21.15.99:0/1774859539 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6001d070 con 0x7efc70101b20 2023-08-25T08:56:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.752+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc701a08b0 con 0x7efc70101b20 2023-08-25T08:56:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.752+0000 7efc667fc700 1 -- 172.21.15.99:0/1774859539 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc60005510 con 0x7efc70101b20 2023-08-25T08:56:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.752+0000 7efc667fc700 1 -- 172.21.15.99:0/1774859539 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60004ae0 con 0x7efc70101b20 2023-08-25T08:56:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.752+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc701a0cd0 con 0x7efc70101b20 2023-08-25T08:56:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.753+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7010c190 con 0x7efc70101b20 2023-08-25T08:56:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.755+0000 7efc667fc700 1 -- 172.21.15.99:0/1774859539 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7efc60004cc0 con 0x7efc70101b20 2023-08-25T08:56:43.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.756+0000 7efc667fc700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7efc5c09ffa0 0x7efc5c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:43.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.756+0000 7efc667fc700 1 -- 172.21.15.99:0/1774859539 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efc600cbcf0 con 0x7efc70101b20 2023-08-25T08:56:43.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.756+0000 7efc7519b700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7efc5c09ffa0 0x7efc5c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:43.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.757+0000 7efc7519b700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7efc5c09ffa0 0x7efc5c0a2460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc70101980 tx=0x7efc6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:43.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.760+0000 7efc667fc700 1 -- 172.21.15.99:0/1774859539 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7efc60095f10 con 0x7efc70101b20 2023-08-25T08:56:43.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.961+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc701a0ef0 con 0x7efc70101b20 2023-08-25T08:56:43.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.962+0000 7efc667fc700 1 -- 172.21.15.99:0/1774859539 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18422 (secure 0 0 0) 0x7efc60099530 con 0x7efc70101b20 2023-08-25T08:56:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:43.963 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":79,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:56:32.216885+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:56:43.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7efc5c09ffa0 msgr2=0x7efc5c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:43.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7efc5c09ffa0 0x7efc5c0a2460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc70101980 tx=0x7efc6c009260 comp rx=0 tx=0).stop 2023-08-25T08:56:43.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 msgr2=0x7efc70198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:43.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 0x7efc70198d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc6000f690 comp rx=0 tx=0).stop 2023-08-25T08:56:43.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 shutdown_connections 2023-08-25T08:56:43.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7efc5c09ffa0 0x7efc5c0a2460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc701024c0 0x7efc7019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc70100930 0x7efc701987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 --2- 172.21.15.99:0/1774859539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc70101b20 0x7efc70198d20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:43.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 >> 172.21.15.99:0/1774859539 conn(0x7efc700fbe60 msgr2=0x7efc700fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:43.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 shutdown_connections 2023-08-25T08:56:43.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:43.965+0000 7efc773ff700 1 -- 172.21.15.99:0/1774859539 wait complete. 2023-08-25T08:56:43.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:43.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:56:43.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-08-25T08:56:43.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:43.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:43.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:43.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:43.984 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:43.984 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:43.984 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:43.984 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:43.985 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:43 smithi099 ceph-mon[97641]: Deploying daemon mds.l on smithi181 2023-08-25T08:56:43.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T08:56:43.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T08:56:44.106 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:44.107 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:56:44.107 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-08-25T08:56:44.107 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:44.107 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:44.107 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:44.108 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:44.108 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:44.108 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:44.108 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:44.108 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:44.108 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:43 smithi181 ceph-mon[131296]: Deploying daemon mds.l on smithi181 2023-08-25T08:56:44.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:44.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:56:44.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-08-25T08:56:44.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:44.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:56:44.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:44.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:56:44.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:44.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:44.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:44.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:44.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:43 smithi157 ceph-mon[130879]: Deploying daemon mds.l on smithi181 2023-08-25T08:56:44.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 -- 172.21.15.99:0/712825712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 msgr2=0x7f6564100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 --2- 172.21.15.99:0/712825712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 0x7f6564100d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6554009990 tx=0x7f6554009ca0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 -- 172.21.15.99:0/712825712 shutdown_connections 2023-08-25T08:56:44.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 --2- 172.21.15.99:0/712825712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65641024c0 0x7f6564108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 --2- 172.21.15.99:0/712825712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 0x7f6564100d50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 --2- 172.21.15.99:0/712825712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6564101b20 0x7f6564101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 -- 172.21.15.99:0/712825712 >> 172.21.15.99:0/712825712 conn(0x7f65640fbea0 msgr2=0x7f65640fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:44.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 -- 172.21.15.99:0/712825712 shutdown_connections 2023-08-25T08:56:44.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.193+0000 7f656a3ff700 1 -- 172.21.15.99:0/712825712 wait complete. 2023-08-25T08:56:44.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.194+0000 7f656a3ff700 1 Processor -- start 2023-08-25T08:56:44.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f656a3ff700 1 -- start start 2023-08-25T08:56:44.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f656a3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 0x7f65640746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:44.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f656a3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6564101b20 0x7f6564072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:44.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f656a3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65641024c0 0x7f6564073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:44.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f656a3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640739f0 con 0x7f6564101b20 2023-08-25T08:56:44.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f656a3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6564073b60 con 0x7f65641024c0 2023-08-25T08:56:44.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f656a3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65641a06a0 con 0x7f6564100930 2023-08-25T08:56:44.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f6563fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 0x7f65640746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:44.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f6563fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 0x7f65640746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:54280/0 (socket says 172.21.15.99:54280) 2023-08-25T08:56:44.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.195+0000 7f6563fff700 1 -- 172.21.15.99:0/3211404406 learned_addr learned my addr 172.21.15.99:0/3211404406 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:44.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f65637fe700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6564101b20 0x7f6564072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:44.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f656899c700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65641024c0 0x7f6564073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:44.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f6563fff700 1 -- 172.21.15.99:0/3211404406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65641024c0 msgr2=0x7f6564073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f6563fff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65641024c0 0x7f6564073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f6563fff700 1 -- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6564101b20 msgr2=0x7f6564072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f6563fff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6564101b20 0x7f6564072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f6563fff700 1 -- 172.21.15.99:0/3211404406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6558009520 con 0x7f6564100930 2023-08-25T08:56:44.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f6563fff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 0x7f65640746d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6554005720 tx=0x7f6554009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:44.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f65617fa700 1 -- 172.21.15.99:0/3211404406 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655401d070 con 0x7f6564100930 2023-08-25T08:56:44.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f65617fa700 1 -- 172.21.15.99:0/3211404406 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6554021410 con 0x7f6564100930 2023-08-25T08:56:44.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65540095f0 con 0x7f6564100930 2023-08-25T08:56:44.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f65617fa700 1 -- 172.21.15.99:0/3211404406 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554017650 con 0x7f6564100930 2023-08-25T08:56:44.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.196+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65641a0bb0 con 0x7f6564100930 2023-08-25T08:56:44.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.197+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6564107ec0 con 0x7f6564100930 2023-08-25T08:56:44.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.199+0000 7f65617fa700 1 -- 172.21.15.99:0/3211404406 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f6554021580 con 0x7f6564100930 2023-08-25T08:56:44.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.201+0000 7f65617fa700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f654c09ff50 0x7f654c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:44.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.201+0000 7f65617fa700 1 -- 172.21.15.99:0/3211404406 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f65540cc0a0 con 0x7f6564100930 2023-08-25T08:56:44.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.201+0000 7f65637fe700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f654c09ff50 0x7f654c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:44.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.201+0000 7f65637fe700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f654c09ff50 0x7f654c0a2410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6558009df0 tx=0x7f6558009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:44.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.203+0000 7f65617fa700 1 -- 172.21.15.99:0/3211404406 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f65540962c0 con 0x7f6564100930 2023-08-25T08:56:44.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.406+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f656410c320 con 0x7f6564100930 2023-08-25T08:56:44.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.407+0000 7f65617fa700 1 -- 172.21.15.99:0/3211404406 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65540998e0 con 0x7f6564100930 2023-08-25T08:56:44.407 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T08:56:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f654c09ff50 msgr2=0x7f654c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f654c09ff50 0x7f654c0a2410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6558009df0 tx=0x7f6558009260 comp rx=0 tx=0).stop 2023-08-25T08:56:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 msgr2=0x7f65640746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 0x7f65640746d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6554005720 tx=0x7f6554009580 comp rx=0 tx=0).stop 2023-08-25T08:56:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 shutdown_connections 2023-08-25T08:56:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f654c09ff50 0x7f654c0a2410 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65641024c0 0x7f6564073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100930 0x7f65640746d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 --2- 172.21.15.99:0/3211404406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6564101b20 0x7f6564072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 >> 172.21.15.99:0/3211404406 conn(0x7f65640fbea0 msgr2=0x7f6564105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:44.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 shutdown_connections 2023-08-25T08:56:44.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.410+0000 7f656a3ff700 1 -- 172.21.15.99:0/3211404406 wait complete. 2023-08-25T08:56:44.428 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:56:44.493 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 08:56:44 smithi181 systemd[1]: Starting Ceph mds.l for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:56:44.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 -- 172.21.15.99:0/36703614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c101b20 msgr2=0x7f814c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 --2- 172.21.15.99:0/36703614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c101b20 0x7f814c101f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81400099c0 tx=0x7f8140009cd0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 -- 172.21.15.99:0/36703614 shutdown_connections 2023-08-25T08:56:44.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 --2- 172.21.15.99:0/36703614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f814c1024c0 0x7f814c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 --2- 172.21.15.99:0/36703614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c100930 0x7f814c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 --2- 172.21.15.99:0/36703614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c101b20 0x7f814c101f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 -- 172.21.15.99:0/36703614 >> 172.21.15.99:0/36703614 conn(0x7f814c0fbe60 msgr2=0x7f814c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:44.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 -- 172.21.15.99:0/36703614 shutdown_connections 2023-08-25T08:56:44.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.626+0000 7f81517ff700 1 -- 172.21.15.99:0/36703614 wait complete. 2023-08-25T08:56:44.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.627+0000 7f81517ff700 1 Processor -- start 2023-08-25T08:56:44.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.627+0000 7f81517ff700 1 -- start start 2023-08-25T08:56:44.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.627+0000 7f81517ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c100930 0x7f814c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:44.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.627+0000 7f81517ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f814c101b20 0x7f814c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:44.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.627+0000 7f81517ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c1024c0 0x7f814c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:44.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.627+0000 7f81517ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c1a0310 con 0x7f814c101b20 2023-08-25T08:56:44.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.627+0000 7f81517ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c1a0450 con 0x7f814c100930 2023-08-25T08:56:44.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.627+0000 7f81517ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c1a05e0 con 0x7f814c1024c0 2023-08-25T08:56:44.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f814c101b20 0x7f814c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:44.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c100930 0x7f814c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:44.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c100930 0x7f814c198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:44904/0 (socket says 172.21.15.99:44904) 2023-08-25T08:56:44.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c1024c0 0x7f814c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:44.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 -- 172.21.15.99:0/605041142 learned_addr learned my addr 172.21.15.99:0/605041142 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:44.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f814c101b20 0x7f814c198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45514/0 (socket says 172.21.15.99:45514) 2023-08-25T08:56:44.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 -- 172.21.15.99:0/605041142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c1024c0 msgr2=0x7f814c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c1024c0 0x7f814c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 -- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f814c101b20 msgr2=0x7f814c198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f814c101b20 0x7f814c198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 -- 172.21.15.99:0/605041142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81400095f0 con 0x7f814c100930 2023-08-25T08:56:44.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814a7fc700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f814c101b20 0x7f814c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:56:44.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.628+0000 7f814affd700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c100930 0x7f814c198790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f813c00e970 tx=0x7f813c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:44.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.629+0000 7f8133fff700 1 -- 172.21.15.99:0/605041142 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c009790 con 0x7f814c100930 2023-08-25T08:56:44.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.629+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c1a08c0 con 0x7f814c100930 2023-08-25T08:56:44.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.629+0000 7f8133fff700 1 -- 172.21.15.99:0/605041142 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f813c00c7e0 con 0x7f814c100930 2023-08-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.629+0000 7f8133fff700 1 -- 172.21.15.99:0/605041142 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c003c70 con 0x7f814c100930 2023-08-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.629+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c1a0dd0 con 0x7f814c100930 2023-08-25T08:56:44.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.630+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c10c140 con 0x7f814c100930 2023-08-25T08:56:44.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.632+0000 7f8133fff700 1 -- 172.21.15.99:0/605041142 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f813c00cc50 con 0x7f814c100930 2023-08-25T08:56:44.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.633+0000 7f8133fff700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f813409ffa0 0x7f81340a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:44.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.633+0000 7f8133fff700 1 -- 172.21.15.99:0/605041142 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f813c014070 con 0x7f814c100930 2023-08-25T08:56:44.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.633+0000 7f814a7fc700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f813409ffa0 0x7f81340a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:44.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.634+0000 7f814a7fc700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f813409ffa0 0x7f81340a2460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f814000b580 tx=0x7f8140005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:44.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.635+0000 7f8133fff700 1 -- 172.21.15.99:0/605041142 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f813c0952e0 con 0x7f814c100930 2023-08-25T08:56:44.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.885+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f814c10c330 con 0x7f814c100930 2023-08-25T08:56:44.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.886+0000 7f8133fff700 1 -- 172.21.15.99:0/605041142 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f813c003dd0 con 0x7f814c100930 2023-08-25T08:56:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:44.886 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-25T08:56:44.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.888+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f813409ffa0 msgr2=0x7f81340a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.888+0000 7f81517ff700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f813409ffa0 0x7f81340a2460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f814000b580 tx=0x7f8140005fb0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.888+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c100930 msgr2=0x7f814c198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:44.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.888+0000 7f81517ff700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c100930 0x7f814c198790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f813c00e970 tx=0x7f813c00ec80 comp rx=0 tx=0).stop 2023-08-25T08:56:44.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.889+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 shutdown_connections 2023-08-25T08:56:44.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.889+0000 7f81517ff700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c1024c0 0x7f814c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.889+0000 7f81517ff700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f813409ffa0 0x7f81340a2460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.889+0000 7f81517ff700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c100930 0x7f814c198790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.889+0000 7f81517ff700 1 --2- 172.21.15.99:0/605041142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f814c101b20 0x7f814c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:44.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.889+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 >> 172.21.15.99:0/605041142 conn(0x7f814c0fbe60 msgr2=0x7f814c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:44.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.889+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 shutdown_connections 2023-08-25T08:56:44.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:44.889+0000 7f81517ff700 1 -- 172.21.15.99:0/605041142 wait complete. 2023-08-25T08:56:44.890 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 2 2023-08-25T08:56:44.924 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:44 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1774859539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:56:44.924 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:44 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3211404406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:56:44.925 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 08:56:44 smithi181 podman[156375]: 2023-08-25T08:56:44.925 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 08:56:44 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-l[156391]: starting mds.l at 2023-08-25T08:56:44.926 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 08:56:44 smithi181 bash[156375]: 3deff3cf2ff4a8dd3e64bf1136bd25e7ebfb1fa26b4cd0ab94454800d80c0292 2023-08-25T08:56:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:44 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1774859539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:56:45.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:44 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3211404406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:56:45.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:44 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1774859539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:56:45.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:44 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3211404406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:56:45.242 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 08:56:44 smithi181 systemd[1]: Started Ceph mds.l for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:56:45.935 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:45.936 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6832/829780458,v1:172.21.15.181:6833/829780458] up:boot 2023-08-25T08:56:45.936 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: fsmap 1 up:standby 2023-08-25T08:56:45.936 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T08:56:45.936 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/605041142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:45.936 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:45.936 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:45.937 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:45.937 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:45.937 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:45.937 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:45 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:46.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6832/829780458,v1:172.21.15.181:6833/829780458] up:boot 2023-08-25T08:56:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: fsmap 1 up:standby 2023-08-25T08:56:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T08:56:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/605041142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:46.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:46.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:46.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:46.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:45 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:46.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:46.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6832/829780458,v1:172.21.15.181:6833/829780458] up:boot 2023-08-25T08:56:46.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: fsmap 1 up:standby 2023-08-25T08:56:46.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T08:56:46.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/605041142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:46.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:46.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:46.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:46.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:46.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:46.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:45 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:46.629 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 08:56:46 smithi157 systemd[1]: Starting Ceph mds.k for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:56:46.907 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:56:47.075 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:46 smithi157 ceph-mon[130879]: Deploying daemon mds.k on smithi157 2023-08-25T08:56:47.075 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 08:56:46 smithi157 podman[154869]: 2023-08-25T08:56:47.075 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mds[154909]: -- 172.21.15.157:0/3217321666 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5626d52a51e0 con 0x5626d527ac00 2023-08-25T08:56:47.076 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 08:56:47 smithi157 bash[154869]: fbf0b74d2a2284916bfeb3ad1f7cd7df7fa7b9c4136d8eaacf5de98249273c46 2023-08-25T08:56:47.076 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-k[154885]: starting mds.k at 2023-08-25T08:56:47.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 -- 172.21.15.99:0/1756110478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c101b50 msgr2=0x7f033c101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:47.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 --2- 172.21.15.99:0/1756110478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c101b50 0x7f033c101fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0330009990 tx=0x7f0330009ca0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 -- 172.21.15.99:0/1756110478 shutdown_connections 2023-08-25T08:56:47.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 --2- 172.21.15.99:0/1756110478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f033c1024f0 0x7f033c108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 --2- 172.21.15.99:0/1756110478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c100960 0x7f033c100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 --2- 172.21.15.99:0/1756110478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c101b50 0x7f033c101fb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 -- 172.21.15.99:0/1756110478 >> 172.21.15.99:0/1756110478 conn(0x7f033c0fbed0 msgr2=0x7f033c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:47.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 -- 172.21.15.99:0/1756110478 shutdown_connections 2023-08-25T08:56:47.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.117+0000 7f03413ff700 1 -- 172.21.15.99:0/1756110478 wait complete. 2023-08-25T08:56:47.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.118+0000 7f03413ff700 1 Processor -- start 2023-08-25T08:56:47.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.118+0000 7f03413ff700 1 -- start start 2023-08-25T08:56:47.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.118+0000 7f03413ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f033c100960 0x7f033c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:47.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.118+0000 7f03413ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c101b50 0x7f033c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:47.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.118+0000 7f03413ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c1024f0 0x7f033c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:47.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.118+0000 7f03413ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c1a03a0 con 0x7f033c1024f0 2023-08-25T08:56:47.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.118+0000 7f03413ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c1a04e0 con 0x7f033c100960 2023-08-25T08:56:47.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.118+0000 7f03413ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c1a0670 con 0x7f033c101b50 2023-08-25T08:56:47.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f033c100960 0x7f033c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:47.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c101b50 0x7f033c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:47.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f033c100960 0x7f033c198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50850/0 (socket says 172.21.15.99:50850) 2023-08-25T08:56:47.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 -- 172.21.15.99:0/2588112889 learned_addr learned my addr 172.21.15.99:0/2588112889 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:47.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033b7fe700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c1024f0 0x7f033c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:47.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 -- 172.21.15.99:0/2588112889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c101b50 msgr2=0x7f033c198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:47.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c101b50 0x7f033c198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 -- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c1024f0 msgr2=0x7f033c19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:47.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c1024f0 0x7f033c19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 -- 172.21.15.99:0/2588112889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334009c40 con 0x7f033c100960 2023-08-25T08:56:47.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f033affd700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f033c100960 0x7f033c198820 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f032c00e970 tx=0x7f032c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:47.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.119+0000 7f0323fff700 1 -- 172.21.15.99:0/2588112889 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c009790 con 0x7f033c100960 2023-08-25T08:56:47.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.120+0000 7f0323fff700 1 -- 172.21.15.99:0/2588112889 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f032c00c7c0 con 0x7f033c100960 2023-08-25T08:56:47.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.120+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03300095f0 con 0x7f033c100960 2023-08-25T08:56:47.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.120+0000 7f0323fff700 1 -- 172.21.15.99:0/2588112889 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c003c20 con 0x7f033c100960 2023-08-25T08:56:47.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.120+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c1a0d20 con 0x7f033c100960 2023-08-25T08:56:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.121+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c192fb0 con 0x7f033c100960 2023-08-25T08:56:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.123+0000 7f0323fff700 1 -- 172.21.15.99:0/2588112889 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f032c004220 con 0x7f033c100960 2023-08-25T08:56:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.124+0000 7f0323fff700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f03240a0020 0x7f03240a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.124+0000 7f0323fff700 1 -- 172.21.15.99:0/2588112889 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f032c014070 con 0x7f033c100960 2023-08-25T08:56:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.124+0000 7f033a7fc700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f03240a0020 0x7f03240a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.125+0000 7f033a7fc700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f03240a0020 0x7f03240a24e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f033000b2e0 tx=0x7f0330005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.129+0000 7f0323fff700 1 -- 172.21.15.99:0/2588112889 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f032c095110 con 0x7f033c100960 2023-08-25T08:56:47.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:46 smithi181 ceph-mon[131296]: Deploying daemon mds.k on smithi157 2023-08-25T08:56:47.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:46 smithi099 ceph-mon[97641]: Deploying daemon mds.k on smithi157 2023-08-25T08:56:47.379 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 08:56:47 smithi157 systemd[1]: Started Ceph mds.k for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:56:47.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.386+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f033c02cc70 con 0x7f033c100960 2023-08-25T08:56:47.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.387+0000 7f0323fff700 1 -- 172.21.15.99:0/2588112889 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f032c098730 con 0x7f033c100960 2023-08-25T08:56:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:47.388 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-25T08:56:47.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.389+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f03240a0020 msgr2=0x7f03240a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:47.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.389+0000 7f03413ff700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f03240a0020 0x7f03240a24e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f033000b2e0 tx=0x7f0330005fb0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.389+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f033c100960 msgr2=0x7f033c198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.389+0000 7f03413ff700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f033c100960 0x7f033c198820 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f032c00e970 tx=0x7f032c00ec80 comp rx=0 tx=0).stop 2023-08-25T08:56:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.390+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 shutdown_connections 2023-08-25T08:56:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.390+0000 7f03413ff700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c1024f0 0x7f033c19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.390+0000 7f03413ff700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f03240a0020 0x7f03240a24e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.390+0000 7f03413ff700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f033c100960 0x7f033c198820 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.390+0000 7f03413ff700 1 --2- 172.21.15.99:0/2588112889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c101b50 0x7f033c198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.390+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 >> 172.21.15.99:0/2588112889 conn(0x7f033c0fbed0 msgr2=0x7f033c107430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.390+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 shutdown_connections 2023-08-25T08:56:47.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:47.390+0000 7f03413ff700 1 -- 172.21.15.99:0/2588112889 wait complete. 2023-08-25T08:56:47.392 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 2 2023-08-25T08:56:48.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mon[130879]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:48.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:48.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:48.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:48.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:47 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2588112889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:48.162 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:47 smithi099 ceph-mon[97641]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:48.162 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.162 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.163 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.163 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:48.163 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:48.163 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:47 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:48.163 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:47 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2588112889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:48.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:47 smithi181 ceph-mon[131296]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:48.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:48.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:48.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:48.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:47 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:48.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:47 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2588112889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:49.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:48 smithi157 ceph-mon[130879]: Deploying daemon mds.j on smithi099 2023-08-25T08:56:49.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:48 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6832/3715294590,v1:172.21.15.157:6833/3715294590] up:boot 2023-08-25T08:56:49.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:48 smithi157 ceph-mon[130879]: fsmap 2 up:standby 2023-08-25T08:56:49.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:48 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T08:56:49.222 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:48 smithi099 ceph-mon[97641]: Deploying daemon mds.j on smithi099 2023-08-25T08:56:49.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:48 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6832/3715294590,v1:172.21.15.157:6833/3715294590] up:boot 2023-08-25T08:56:49.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:48 smithi099 ceph-mon[97641]: fsmap 2 up:standby 2023-08-25T08:56:49.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:48 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T08:56:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:48 smithi181 ceph-mon[131296]: Deploying daemon mds.j on smithi099 2023-08-25T08:56:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:48 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6832/3715294590,v1:172.21.15.157:6833/3715294590] up:boot 2023-08-25T08:56:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:48 smithi181 ceph-mon[131296]: fsmap 2 up:standby 2023-08-25T08:56:49.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:48 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T08:56:49.410 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:56:49.479 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 08:56:49 smithi099 systemd[1]: Starting Ceph mds.j for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:56:49.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 -- 172.21.15.99:0/3337198576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8854100930 msgr2=0x7f8854100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:49.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 --2- 172.21.15.99:0/3337198576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8854100930 0x7f8854100d50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88440098f0 tx=0x7f8844009c00 comp rx=0 tx=0).stop 2023-08-25T08:56:49.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 -- 172.21.15.99:0/3337198576 shutdown_connections 2023-08-25T08:56:49.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 --2- 172.21.15.99:0/3337198576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88541024c0 0x7f8854108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 --2- 172.21.15.99:0/3337198576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8854100930 0x7f8854100d50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 --2- 172.21.15.99:0/3337198576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854101b20 0x7f8854101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 -- 172.21.15.99:0/3337198576 >> 172.21.15.99:0/3337198576 conn(0x7f88540fbe60 msgr2=0x7f88540fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:49.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 -- 172.21.15.99:0/3337198576 shutdown_connections 2023-08-25T08:56:49.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.631+0000 7f88597ff700 1 -- 172.21.15.99:0/3337198576 wait complete. 2023-08-25T08:56:49.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.633+0000 7f88597ff700 1 Processor -- start 2023-08-25T08:56:49.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.633+0000 7f88597ff700 1 -- start start 2023-08-25T08:56:49.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f88597ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854100930 0x7f8854196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:49.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f88597ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854101b20 0x7f8854196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f88597ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88541024c0 0x7f885419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f88597ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885419e1e0 con 0x7f8854100930 2023-08-25T08:56:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f88597ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885419e320 con 0x7f8854101b20 2023-08-25T08:56:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f88597ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885419e4b0 con 0x7f88541024c0 2023-08-25T08:56:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854100930 0x7f8854196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854100930 0x7f8854196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34724/0 (socket says 172.21.15.99:34724) 2023-08-25T08:56:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 -- 172.21.15.99:0/1961095497 learned_addr learned my addr 172.21.15.99:0/1961095497 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f88527fc700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854101b20 0x7f8854196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f88537fe700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88541024c0 0x7f885419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 -- 172.21.15.99:0/1961095497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88541024c0 msgr2=0x7f885419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88541024c0 0x7f885419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 -- 172.21.15.99:0/1961095497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854101b20 msgr2=0x7f8854196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854101b20 0x7f8854196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 -- 172.21.15.99:0/1961095497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8844009520 con 0x7f8854100930 2023-08-25T08:56:49.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.634+0000 7f8852ffd700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854100930 0x7f8854196660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8844009fd0 tx=0x7f884400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:49.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.635+0000 7f883bfff700 1 -- 172.21.15.99:0/1961095497 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884401c070 con 0x7f8854100930 2023-08-25T08:56:49.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.635+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8854072b50 con 0x7f8854100930 2023-08-25T08:56:49.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.635+0000 7f883bfff700 1 -- 172.21.15.99:0/1961095497 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f884400bc10 con 0x7f8854100930 2023-08-25T08:56:49.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.635+0000 7f883bfff700 1 -- 172.21.15.99:0/1961095497 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844003990 con 0x7f8854100930 2023-08-25T08:56:49.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.635+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8854073000 con 0x7f8854100930 2023-08-25T08:56:49.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.636+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8854107f20 con 0x7f8854100930 2023-08-25T08:56:49.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.638+0000 7f883bfff700 1 -- 172.21.15.99:0/1961095497 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f8844020030 con 0x7f8854100930 2023-08-25T08:56:49.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.639+0000 7f883bfff700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f883c09ffa0 0x7f883c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:49.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.639+0000 7f883bfff700 1 -- 172.21.15.99:0/1961095497 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f88440cbe00 con 0x7f8854100930 2023-08-25T08:56:49.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.640+0000 7f88527fc700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f883c09ffa0 0x7f883c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:49.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.640+0000 7f88527fc700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f883c09ffa0 0x7f883c0a2460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8848009c60 tx=0x7f8848009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:49.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.644+0000 7f883bfff700 1 -- 172.21.15.99:0/1961095497 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8844096020 con 0x7f8854100930 2023-08-25T08:56:49.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.921+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f885402cc70 con 0x7f8854100930 2023-08-25T08:56:49.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.921+0000 7f883bfff700 1 -- 172.21.15.99:0/1961095497 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f8844005040 con 0x7f8854100930 2023-08-25T08:56:49.922 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:49.923 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-25T08:56:49.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f883c09ffa0 msgr2=0x7f883c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:49.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f883c09ffa0 0x7f883c0a2460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8848009c60 tx=0x7f8848009260 comp rx=0 tx=0).stop 2023-08-25T08:56:49.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854100930 msgr2=0x7f8854196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:49.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854100930 0x7f8854196660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8844009fd0 tx=0x7f884400b560 comp rx=0 tx=0).stop 2023-08-25T08:56:49.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 shutdown_connections 2023-08-25T08:56:49.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88541024c0 0x7f885419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854100930 0x7f8854196660 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f883c09ffa0 0x7f883c0a2460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 --2- 172.21.15.99:0/1961095497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8854101b20 0x7f8854196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:49.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 >> 172.21.15.99:0/1961095497 conn(0x7f88540fbe60 msgr2=0x7f8854105060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:49.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 shutdown_connections 2023-08-25T08:56:49.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:49.924+0000 7f88597ff700 1 -- 172.21.15.99:0/1961095497 wait complete. 2023-08-25T08:56:49.926 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 3 2023-08-25T08:56:50.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:49 smithi157 ceph-mon[130879]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:50.161 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:49 smithi099 ceph-mon[97641]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:50.161 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 08:56:49 smithi099 podman[131032]: 2023-08-25T08:56:50.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:49 smithi181 ceph-mon[131296]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:50.562 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 08:56:50 smithi099 bash[131032]: 370be917da72ac74ce34a1124168682b8516cd5a79c9eeb93d91210f1c500c49 2023-08-25T08:56:50.562 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-mds[131074]: -- 172.21.15.99:0/1092047321 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5634adefb1e0 con 0x5634aded0800 2023-08-25T08:56:50.563 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-j[131050]: starting mds.j at 2023-08-25T08:56:50.563 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 08:56:50 smithi099 systemd[1]: Started Ceph mds.j for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:56:51.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:50 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1961095497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:51.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:51.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:51.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:50 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:51.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1961095497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:51.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:51.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:51.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:50 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:51.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:50 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1961095497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:51.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:51.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:50 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:51.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:56:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 -- 172.21.15.99:0/3055916578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 msgr2=0x7f40d8108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/3055916578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 0x7f40d8108d70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f40d000b0d0 tx=0x7f40d000b3e0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 -- 172.21.15.99:0/3055916578 shutdown_connections 2023-08-25T08:56:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/3055916578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 0x7f40d8108d70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/3055916578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100930 0x7f40d8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/3055916578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d8101b20 0x7f40d8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 -- 172.21.15.99:0/3055916578 >> 172.21.15.99:0/3055916578 conn(0x7f40d80fbe60 msgr2=0x7f40d80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 -- 172.21.15.99:0/3055916578 shutdown_connections 2023-08-25T08:56:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.162+0000 7f40dd3ff700 1 -- 172.21.15.99:0/3055916578 wait complete. 2023-08-25T08:56:52.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.163+0000 7f40dd3ff700 1 Processor -- start 2023-08-25T08:56:52.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.163+0000 7f40dd3ff700 1 -- start start 2023-08-25T08:56:52.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.163+0000 7f40dd3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100930 0x7f40d81966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:52.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40dd3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d8101b20 0x7f40d8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:52.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40dd3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 0x7f40d819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:52.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40dd3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819e260 con 0x7f40d81024c0 2023-08-25T08:56:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40dd3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819e3a0 con 0x7f40d8100930 2023-08-25T08:56:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40dd3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819e530 con 0x7f40d8101b20 2023-08-25T08:56:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 0x7f40d819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 0x7f40d819dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34770/0 (socket says 172.21.15.99:34770) 2023-08-25T08:56:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d77fe700 1 -- 172.21.15.99:0/2644832561 learned_addr learned my addr 172.21.15.99:0/2644832561 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d67fc700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d8101b20 0x7f40d8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d6ffd700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100930 0x7f40d81966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d77fe700 1 -- 172.21.15.99:0/2644832561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d8101b20 msgr2=0x7f40d8196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d77fe700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d8101b20 0x7f40d8196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d77fe700 1 -- 172.21.15.99:0/2644832561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100930 msgr2=0x7f40d81966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d77fe700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100930 0x7f40d81966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d77fe700 1 -- 172.21.15.99:0/2644832561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8009520 con 0x7f40d81024c0 2023-08-25T08:56:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.164+0000 7f40d67fc700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d8101b20 0x7f40d8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:56:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.165+0000 7f40d77fe700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 0x7f40d819dc90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f40d0008eb0 tx=0x7f40d0008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.165+0000 7f40bffff700 1 -- 172.21.15.99:0/2644832561 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000e040 con 0x7f40d81024c0 2023-08-25T08:56:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.165+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d0009d00 con 0x7f40d81024c0 2023-08-25T08:56:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.165+0000 7f40bffff700 1 -- 172.21.15.99:0/2644832561 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40d000bdb0 con 0x7f40d81024c0 2023-08-25T08:56:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.165+0000 7f40bffff700 1 -- 172.21.15.99:0/2644832561 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d001d470 con 0x7f40d81024c0 2023-08-25T08:56:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.165+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d819eaa0 con 0x7f40d81024c0 2023-08-25T08:56:52.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.167+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d8067780 con 0x7f40d81024c0 2023-08-25T08:56:52.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.168+0000 7f40bffff700 1 -- 172.21.15.99:0/2644832561 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f40d0005020 con 0x7f40d81024c0 2023-08-25T08:56:52.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.169+0000 7f40bffff700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f40c009ffa0 0x7f40c00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:52.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.170+0000 7f40bffff700 1 -- 172.21.15.99:0/2644832561 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40d00cbdb0 con 0x7f40d81024c0 2023-08-25T08:56:52.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.170+0000 7f40d6ffd700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f40c009ffa0 0x7f40c00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:52.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.170+0000 7f40d6ffd700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f40c009ffa0 0x7f40c00a2460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40d8101980 tx=0x7f40c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:52.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.173+0000 7f40bffff700 1 -- 172.21.15.99:0/2644832561 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f40d8067780 con 0x7f40d81024c0 2023-08-25T08:56:52.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:51 smithi181 ceph-mon[131296]: Deploying daemon mds.d on smithi099 2023-08-25T08:56:52.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:51 smithi181 ceph-mon[131296]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:52.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:51 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.99:6834/2749583644,v1:172.21.15.99:6835/2749583644] up:boot 2023-08-25T08:56:52.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:51 smithi181 ceph-mon[131296]: fsmap 3 up:standby 2023-08-25T08:56:52.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:51 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T08:56:52.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:51 smithi099 ceph-mon[97641]: Deploying daemon mds.d on smithi099 2023-08-25T08:56:52.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:51 smithi099 ceph-mon[97641]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:52.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:51 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.99:6834/2749583644,v1:172.21.15.99:6835/2749583644] up:boot 2023-08-25T08:56:52.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:51 smithi099 ceph-mon[97641]: fsmap 3 up:standby 2023-08-25T08:56:52.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:51 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T08:56:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:51 smithi157 ceph-mon[130879]: Deploying daemon mds.d on smithi099 2023-08-25T08:56:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:51 smithi157 ceph-mon[130879]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:51 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.99:6834/2749583644,v1:172.21.15.99:6835/2749583644] up:boot 2023-08-25T08:56:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:51 smithi157 ceph-mon[130879]: fsmap 3 up:standby 2023-08-25T08:56:52.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:51 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T08:56:52.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.471+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40d810c250 con 0x7f40d81024c0 2023-08-25T08:56:52.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.471+0000 7f40bffff700 1 -- 172.21.15.99:0/2644832561 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f40d00d0070 con 0x7f40d81024c0 2023-08-25T08:56:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:52.473 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-25T08:56:52.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f40c009ffa0 msgr2=0x7f40c00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:52.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f40c009ffa0 0x7f40c00a2460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40d8101980 tx=0x7f40c8009260 comp rx=0 tx=0).stop 2023-08-25T08:56:52.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 msgr2=0x7f40d819dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:52.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 0x7f40d819dc90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f40d0008eb0 tx=0x7f40d0008ee0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 shutdown_connections 2023-08-25T08:56:52.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40d81024c0 0x7f40d819dc90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f40c009ffa0 0x7f40c00a2460 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100930 0x7f40d81966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 --2- 172.21.15.99:0/2644832561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d8101b20 0x7f40d8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:52.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 >> 172.21.15.99:0/2644832561 conn(0x7f40d80fbe60 msgr2=0x7f40d80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:52.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 shutdown_connections 2023-08-25T08:56:52.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:52.474+0000 7f40dd3ff700 1 -- 172.21.15.99:0/2644832561 wait complete. 2023-08-25T08:56:52.478 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 4 2023-08-25T08:56:53.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:52 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2644832561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:53.063 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 08:56:52 smithi099 systemd[1]: Starting Ceph mds.d for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:56:53.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:52 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2644832561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:53.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:52 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2644832561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:53.477 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 08:56:53 smithi099 podman[131420]: 2023-08-25T08:56:53.812 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 08:56:53 smithi099 bash[131420]: aa62f45c0b6ccb628d6f139122da0dfe03bc978f09792b211bc2a1b51c20fbf3 2023-08-25T08:56:53.812 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-mds[131460]: -- 172.21.15.99:0/3911184875 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568f30951e0 con 0x5568f306b000 2023-08-25T08:56:53.813 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-d[131436]: starting mds.d at 2023-08-25T08:56:53.813 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 08:56:53 smithi099 systemd[1]: Started Ceph mds.d for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:56:54.199 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:53 smithi157 ceph-mon[130879]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:54.199 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.199 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.200 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.200 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:54.200 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:54.200 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:53 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:54.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:53 smithi181 ceph-mon[131296]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:53 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:54.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-mon[97641]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:54.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:54.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:54.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:54.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:53 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:54.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:56:54.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.702+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/2236218721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 msgr2=0x7f4d24102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:54.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.702+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/2236218721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 0x7f4d24102f40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d140099c0 tx=0x7f4d14009cd0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.703+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/2236218721 shutdown_connections 2023-08-25T08:56:54.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.703+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/2236218721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d241046b0 0x7f4d2410af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.703+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/2236218721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 0x7f4d24102f40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.703+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/2236218721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d24103d10 0x7f4d24104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.703+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/2236218721 >> 172.21.15.99:0/2236218721 conn(0x7f4d240fe070 msgr2=0x7f4d241004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:54.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.703+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/2236218721 shutdown_connections 2023-08-25T08:56:54.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.703+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/2236218721 wait complete. 2023-08-25T08:56:54.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2a3ff700 1 Processor -- start 2023-08-25T08:56:54.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2a3ff700 1 -- start start 2023-08-25T08:56:54.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2a3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 0x7f4d24198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:54.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2a3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d24103d10 0x7f4d24198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:54.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2a3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d241046b0 0x7f4d2419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:54.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2a3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d241a0300 con 0x7f4d241046b0 2023-08-25T08:56:54.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2a3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d241a0440 con 0x7f4d24102b20 2023-08-25T08:56:54.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2a3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d241a05d0 con 0x7f4d24103d10 2023-08-25T08:56:54.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 0x7f4d24198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:54.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 0x7f4d24198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50944/0 (socket says 172.21.15.99:50944) 2023-08-25T08:56:54.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d23fff700 1 -- 172.21.15.99:0/1670319403 learned_addr learned my addr 172.21.15.99:0/1670319403 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:54.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d2899c700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d241046b0 0x7f4d2419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:54.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.704+0000 7f4d237fe700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d24103d10 0x7f4d24198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:54.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d23fff700 1 -- 172.21.15.99:0/1670319403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d24103d10 msgr2=0x7f4d24198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d23fff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d24103d10 0x7f4d24198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d23fff700 1 -- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d241046b0 msgr2=0x7f4d2419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d23fff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d241046b0 0x7f4d2419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d23fff700 1 -- 172.21.15.99:0/1670319403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18009520 con 0x7f4d24102b20 2023-08-25T08:56:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d23fff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 0x7f4d24198780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009fd0 tx=0x7f4d14009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d217fa700 1 -- 172.21.15.99:0/1670319403 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1401d070 con 0x7f4d24102b20 2023-08-25T08:56:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d140095f0 con 0x7f4d24102b20 2023-08-25T08:56:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d217fa700 1 -- 172.21.15.99:0/1670319403 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d14005030 con 0x7f4d24102b20 2023-08-25T08:56:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d217fa700 1 -- 172.21.15.99:0/1670319403 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14004a90 con 0x7f4d24102b20 2023-08-25T08:56:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.705+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d241a0ae0 con 0x7f4d24102b20 2023-08-25T08:56:54.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.706+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2410e330 con 0x7f4d24102b20 2023-08-25T08:56:54.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.708+0000 7f4d217fa700 1 -- 172.21.15.99:0/1670319403 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f4d140051a0 con 0x7f4d24102b20 2023-08-25T08:56:54.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.709+0000 7f4d217fa700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d0c09ffa0 0x7f4d0c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:54.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.710+0000 7f4d217fa700 1 -- 172.21.15.99:0/1670319403 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d140cbb70 con 0x7f4d24102b20 2023-08-25T08:56:54.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.710+0000 7f4d237fe700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d0c09ffa0 0x7f4d0c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.710+0000 7f4d237fe700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d0c09ffa0 0x7f4d0c0a2460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009df0 tx=0x7f4d18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.712+0000 7f4d217fa700 1 -- 172.21.15.99:0/1670319403 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4d14095d90 con 0x7f4d24102b20 2023-08-25T08:56:54.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.958+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d24109a00 con 0x7f4d24102b20 2023-08-25T08:56:54.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.958+0000 7f4d217fa700 1 -- 172.21.15.99:0/1670319403 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7f4d140993b0 con 0x7f4d24102b20 2023-08-25T08:56:54.959 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:54.959 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2023-08-25T08:56:54.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d0c09ffa0 msgr2=0x7f4d0c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:54.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d0c09ffa0 0x7f4d0c0a2460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009df0 tx=0x7f4d18009260 comp rx=0 tx=0).stop 2023-08-25T08:56:54.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 msgr2=0x7f4d24198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:54.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 0x7f4d24198780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009fd0 tx=0x7f4d14009580 comp rx=0 tx=0).stop 2023-08-25T08:56:54.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 shutdown_connections 2023-08-25T08:56:54.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d241046b0 0x7f4d2419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d0c09ffa0 0x7f4d0c0a2460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d24102b20 0x7f4d24198780 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 --2- 172.21.15.99:0/1670319403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d24103d10 0x7f4d24198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:54.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 >> 172.21.15.99:0/1670319403 conn(0x7f4d240fe070 msgr2=0x7f4d241002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:54.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 shutdown_connections 2023-08-25T08:56:54.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:54.961+0000 7f4d2a3ff700 1 -- 172.21.15.99:0/1670319403 wait complete. 2023-08-25T08:56:54.964 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 5 2023-08-25T08:56:55.129 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 08:56:55 smithi157 systemd[1]: Starting Ceph mds.e for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:56:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:54 smithi157 ceph-mon[130879]: Deploying daemon mds.e on smithi157 2023-08-25T08:56:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:54 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.99:6836/654200682,v1:172.21.15.99:6837/654200682] up:boot 2023-08-25T08:56:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:54 smithi157 ceph-mon[130879]: fsmap 4 up:standby 2023-08-25T08:56:55.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:54 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T08:56:55.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:54 smithi181 ceph-mon[131296]: Deploying daemon mds.e on smithi157 2023-08-25T08:56:55.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:54 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.99:6836/654200682,v1:172.21.15.99:6837/654200682] up:boot 2023-08-25T08:56:55.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:54 smithi181 ceph-mon[131296]: fsmap 4 up:standby 2023-08-25T08:56:55.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:54 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T08:56:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:54 smithi099 ceph-mon[97641]: Deploying daemon mds.e on smithi157 2023-08-25T08:56:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:54 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.99:6836/654200682,v1:172.21.15.99:6837/654200682] up:boot 2023-08-25T08:56:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:54 smithi099 ceph-mon[97641]: fsmap 4 up:standby 2023-08-25T08:56:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:54 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T08:56:55.629 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 08:56:55 smithi157 podman[155209]: 2023-08-25T08:56:55.629 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mds[155249]: -- 172.21.15.157:0/2271176134 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cda1e7f1e0 con 0x55cda1e54800 2023-08-25T08:56:55.629 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-e[155225]: starting mds.e at 2023-08-25T08:56:55.629 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 08:56:55 smithi157 bash[155209]: 458f75894c419ede512d96822ea7afb71ed2c6788ef5c159abb613bbf2a0502d 2023-08-25T08:56:56.129 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 08:56:55 smithi157 systemd[1]: Started Ceph mds.e for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:56:56.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mon[130879]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:56.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1670319403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:56.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:56.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:56.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:56.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:55 smithi181 ceph-mon[131296]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:56.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:55 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1670319403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:56.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:56.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:56.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:56.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:55 smithi099 ceph-mon[97641]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:55 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1670319403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:56.981 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:56:56.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:56 smithi181 ceph-mon[131296]: Deploying daemon mds.f on smithi181 2023-08-25T08:56:56.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:56 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6834/1963370106,v1:172.21.15.157:6835/1963370106] up:boot 2023-08-25T08:56:56.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:56 smithi181 ceph-mon[131296]: fsmap 5 up:standby 2023-08-25T08:56:56.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T08:56:57.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 -- 172.21.15.99:0/2160499230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10102970 msgr2=0x7f7c10102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:57.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 --2- 172.21.15.99:0/2160499230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10102970 0x7f7c10102d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009990 tx=0x7f7c00009ca0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 -- 172.21.15.99:0/2160499230 shutdown_connections 2023-08-25T08:56:57.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 --2- 172.21.15.99:0/2160499230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c10104500 0x7f7c1010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 --2- 172.21.15.99:0/2160499230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10102970 0x7f7c10102d90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 --2- 172.21.15.99:0/2160499230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c10103b60 0x7f7c10103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 -- 172.21.15.99:0/2160499230 >> 172.21.15.99:0/2160499230 conn(0x7f7c100fdec0 msgr2=0x7f7c10100320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:57.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 -- 172.21.15.99:0/2160499230 shutdown_connections 2023-08-25T08:56:57.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.191+0000 7f7c1539d700 1 -- 172.21.15.99:0/2160499230 wait complete. 2023-08-25T08:56:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c1539d700 1 Processor -- start 2023-08-25T08:56:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c1539d700 1 -- start start 2023-08-25T08:56:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c1539d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c10102970 0x7f7c10198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c1539d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10103b60 0x7f7c10198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c1539d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c10104500 0x7f7c1019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c1539d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101a00d0 con 0x7f7c10102970 2023-08-25T08:56:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c1539d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101a0210 con 0x7f7c10104500 2023-08-25T08:56:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c1539d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101a03a0 con 0x7f7c10103b60 2023-08-25T08:56:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.192+0000 7f7c0ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c10102970 0x7f7c10198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c10102970 0x7f7c10198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47862/0 (socket says 172.21.15.99:47862) 2023-08-25T08:56:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10103b60 0x7f7c10198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0ffff700 1 -- 172.21.15.99:0/4157702063 learned_addr learned my addr 172.21.15.99:0/4157702063 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c14b9c700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c10104500 0x7f7c1019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0ffff700 1 -- 172.21.15.99:0/4157702063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10103b60 msgr2=0x7f7c10198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0ffff700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10103b60 0x7f7c10198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0ffff700 1 -- 172.21.15.99:0/4157702063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c10104500 msgr2=0x7f7c1019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0ffff700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c10104500 0x7f7c1019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0ffff700 1 -- 172.21.15.99:0/4157702063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c000095f0 con 0x7f7c10102970 2023-08-25T08:56:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c07fff700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10103b60 0x7f7c10198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:56:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0ffff700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c10102970 0x7f7c10198550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c000098e0 tx=0x7f7c00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0dffb700 1 -- 172.21.15.99:0/4157702063 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0001d070 con 0x7f7c10102970 2023-08-25T08:56:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0dffb700 1 -- 172.21.15.99:0/4157702063 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c000054f0 con 0x7f7c10102970 2023-08-25T08:56:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c101a0620 con 0x7f7c10102970 2023-08-25T08:56:57.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.193+0000 7f7c0dffb700 1 -- 172.21.15.99:0/4157702063 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00004be0 con 0x7f7c10102970 2023-08-25T08:56:57.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.194+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c101a0a40 con 0x7f7c10102970 2023-08-25T08:56:57.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.195+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c101097b0 con 0x7f7c10102970 2023-08-25T08:56:57.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.196+0000 7f7c0dffb700 1 -- 172.21.15.99:0/4157702063 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f7c0000fbc0 con 0x7f7c10102970 2023-08-25T08:56:57.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.199+0000 7f7c0dffb700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7bfc0a00c0 0x7f7bfc0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.199+0000 7f7c0dffb700 1 -- 172.21.15.99:0/4157702063 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7c000cbff0 con 0x7f7c10102970 2023-08-25T08:56:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.199+0000 7f7c07fff700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7bfc0a00c0 0x7f7bfc0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.200+0000 7f7c07fff700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7bfc0a00c0 0x7f7bfc0a2580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8005950 tx=0x7f7bf800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:57.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.203+0000 7f7c0dffb700 1 -- 172.21.15.99:0/4157702063 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7c0005ae00 con 0x7f7c10102970 2023-08-25T08:56:57.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:56 smithi099 ceph-mon[97641]: Deploying daemon mds.f on smithi181 2023-08-25T08:56:57.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:56 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6834/1963370106,v1:172.21.15.157:6835/1963370106] up:boot 2023-08-25T08:56:57.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:56 smithi099 ceph-mon[97641]: fsmap 5 up:standby 2023-08-25T08:56:57.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T08:56:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:56 smithi157 ceph-mon[130879]: Deploying daemon mds.f on smithi181 2023-08-25T08:56:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:56 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6834/1963370106,v1:172.21.15.157:6835/1963370106] up:boot 2023-08-25T08:56:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:56 smithi157 ceph-mon[130879]: fsmap 5 up:standby 2023-08-25T08:56:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T08:56:57.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.444+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c1010e1d0 con 0x7f7c10102970 2023-08-25T08:56:57.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.444+0000 7f7c0dffb700 1 -- 172.21.15.99:0/4157702063 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7f7c00096230 con 0x7f7c10102970 2023-08-25T08:56:57.445 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:57.445 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-08-25T08:56:57.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7bfc0a00c0 msgr2=0x7f7bfc0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:57.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7bfc0a00c0 0x7f7bfc0a2580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8005950 tx=0x7f7bf800b410 comp rx=0 tx=0).stop 2023-08-25T08:56:57.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c10102970 msgr2=0x7f7c10198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c10102970 0x7f7c10198550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c000098e0 tx=0x7f7c00004730 comp rx=0 tx=0).stop 2023-08-25T08:56:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 shutdown_connections 2023-08-25T08:56:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7bfc0a00c0 0x7f7bfc0a2580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c10104500 0x7f7c1019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c10102970 0x7f7c10198550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 --2- 172.21.15.99:0/4157702063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c10103b60 0x7f7c10198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 >> 172.21.15.99:0/4157702063 conn(0x7f7c100fdec0 msgr2=0x7f7c101000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:57.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 shutdown_connections 2023-08-25T08:56:57.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:57.447+0000 7f7c1539d700 1 -- 172.21.15.99:0/4157702063 wait complete. 2023-08-25T08:56:57.449 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2023-08-25T08:56:57.493 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 08:56:57 smithi181 systemd[1]: Starting Ceph mds.f for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:56:57.792 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 08:56:57 smithi181 podman[156719]: 2023-08-25T08:56:58.242 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 08:56:57 smithi181 ceph-mds[156759]: -- 172.21.15.181:0/1627203274 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559a649ed1e0 con 0x559a649c2c00 2023-08-25T08:56:58.242 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 08:56:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-f[156735]: starting mds.f at 2023-08-25T08:56:58.243 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 08:56:57 smithi181 bash[156719]: 1da113e7caa024037853c96d86883519477363c2d4908cbc6b0eb9f05703f496 2023-08-25T08:56:58.243 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 08:56:57 smithi181 systemd[1]: Started Ceph mds.f for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:56:58.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:57 smithi181 ceph-mon[131296]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:58.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:57 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4157702063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:58.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:57 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:58.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:57 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:58.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:57 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:58.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:57 smithi099 ceph-mon[97641]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:58.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:57 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4157702063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:58.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:58.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:58.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:57 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:58.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:57 smithi157 ceph-mon[130879]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:58.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:57 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4157702063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:56:58.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:57 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:58.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:57 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:58.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:57 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:56:59.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:59.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:59.231 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:59.231 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:58 smithi181 ceph-mon[131296]: Deploying daemon mds.c on smithi181 2023-08-25T08:56:59.231 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:58 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] up:boot 2023-08-25T08:56:59.231 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:58 smithi181 ceph-mon[131296]: fsmap 6 up:standby 2023-08-25T08:56:59.231 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:58 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T08:56:59.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:59.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:59.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:59.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:58 smithi099 ceph-mon[97641]: Deploying daemon mds.c on smithi181 2023-08-25T08:56:59.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:58 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] up:boot 2023-08-25T08:56:59.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:58 smithi099 ceph-mon[97641]: fsmap 6 up:standby 2023-08-25T08:56:59.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:58 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T08:56:59.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:56:59.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:56:59.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:56:59.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:58 smithi157 ceph-mon[130879]: Deploying daemon mds.c on smithi181 2023-08-25T08:56:59.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:58 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] up:boot 2023-08-25T08:56:59.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:58 smithi157 ceph-mon[130879]: fsmap 6 up:standby 2023-08-25T08:56:59.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:58 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T08:56:59.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:56:59.492 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 08:56:59 smithi181 systemd[1]: Starting Ceph mds.c for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:56:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 -- 172.21.15.99:0/2421902787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c100930 msgr2=0x7f3f4c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2421902787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c100930 0x7f3f4c100d50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009990 tx=0x7f3f48009ca0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 -- 172.21.15.99:0/2421902787 shutdown_connections 2023-08-25T08:56:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2421902787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f4c1024c0 0x7f3f4c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2421902787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c100930 0x7f3f4c100d50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2421902787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c101b20 0x7f3f4c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 -- 172.21.15.99:0/2421902787 >> 172.21.15.99:0/2421902787 conn(0x7f3f4c0fbe60 msgr2=0x7f3f4c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 -- 172.21.15.99:0/2421902787 shutdown_connections 2023-08-25T08:56:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.675+0000 7f3f543ff700 1 -- 172.21.15.99:0/2421902787 wait complete. 2023-08-25T08:56:59.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.676+0000 7f3f543ff700 1 Processor -- start 2023-08-25T08:56:59.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.676+0000 7f3f543ff700 1 -- start start 2023-08-25T08:56:59.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f543ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c100930 0x7f3f4c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f543ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f4c101b20 0x7f3f4c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f543ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c1024c0 0x7f3f4c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f543ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0739f0 con 0x7f3f4c101b20 2023-08-25T08:56:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f543ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c073b60 con 0x7f3f4c100930 2023-08-25T08:56:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f543ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c1a0680 con 0x7f3f4c1024c0 2023-08-25T08:56:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5199a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f4c101b20 0x7f3f4c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5199a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f4c101b20 0x7f3f4c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47876/0 (socket says 172.21.15.99:47876) 2023-08-25T08:56:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5199a700 1 -- 172.21.15.99:0/2400262705 learned_addr learned my addr 172.21.15.99:0/2400262705 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:56:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5299c700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c1024c0 0x7f3f4c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5219b700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c100930 0x7f3f4c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5199a700 1 -- 172.21.15.99:0/2400262705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c1024c0 msgr2=0x7f3f4c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5199a700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c1024c0 0x7f3f4c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5199a700 1 -- 172.21.15.99:0/2400262705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c100930 msgr2=0x7f3f4c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5199a700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c100930 0x7f3f4c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.677+0000 7f3f5199a700 1 -- 172.21.15.99:0/2400262705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480095f0 con 0x7f3f4c101b20 2023-08-25T08:56:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.678+0000 7f3f5199a700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f4c101b20 0x7f3f4c072d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00e9a0 tx=0x7f3f3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.678+0000 7f3f437fe700 1 -- 172.21.15.99:0/2400262705 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c009790 con 0x7f3f4c101b20 2023-08-25T08:56:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.678+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c1a0960 con 0x7f3f4c101b20 2023-08-25T08:56:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.678+0000 7f3f437fe700 1 -- 172.21.15.99:0/2400262705 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3c010430 con 0x7f3f4c101b20 2023-08-25T08:56:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.678+0000 7f3f437fe700 1 -- 172.21.15.99:0/2400262705 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c018680 con 0x7f3f4c101b20 2023-08-25T08:56:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.678+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c1a0ec0 con 0x7f3f4c101b20 2023-08-25T08:56:59.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.679+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c107ed0 con 0x7f3f4c101b20 2023-08-25T08:56:59.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.681+0000 7f3f437fe700 1 -- 172.21.15.99:0/2400262705 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f3f3c0105a0 con 0x7f3f4c101b20 2023-08-25T08:56:59.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.683+0000 7f3f437fe700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f3809ffa0 0x7f3f380a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:56:59.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.683+0000 7f3f437fe700 1 -- 172.21.15.99:0/2400262705 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f3c014070 con 0x7f3f4c101b20 2023-08-25T08:56:59.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.683+0000 7f3f5219b700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f3809ffa0 0x7f3f380a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:56:59.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.684+0000 7f3f5219b700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f3809ffa0 0x7f3f380a2460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800b680 tx=0x7f3f48005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:56:59.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.688+0000 7f3f437fe700 1 -- 172.21.15.99:0/2400262705 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3f3c051e00 con 0x7f3f4c101b20 2023-08-25T08:56:59.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.934+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f4c10c140 con 0x7f3f4c101b20 2023-08-25T08:56:59.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.934+0000 7f3f437fe700 1 -- 172.21.15.99:0/2400262705 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7f3f3c095180 con 0x7f3f4c101b20 2023-08-25T08:56:59.935 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:56:59.936 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-08-25T08:56:59.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f3809ffa0 msgr2=0x7f3f380a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:59.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f3809ffa0 0x7f3f380a2460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800b680 tx=0x7f3f48005fd0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f4c101b20 msgr2=0x7f3f4c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:56:59.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f4c101b20 0x7f3f4c072d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00e9a0 tx=0x7f3f3c00ecb0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 shutdown_connections 2023-08-25T08:56:59.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c1024c0 0x7f3f4c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f3809ffa0 0x7f3f380a2460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c100930 0x7f3f4c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 --2- 172.21.15.99:0/2400262705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f4c101b20 0x7f3f4c072d10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:56:59.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 >> 172.21.15.99:0/2400262705 conn(0x7f3f4c0fbe60 msgr2=0x7f3f4c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:56:59.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 shutdown_connections 2023-08-25T08:56:59.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:56:59.937+0000 7f3f543ff700 1 -- 172.21.15.99:0/2400262705 wait complete. 2023-08-25T08:56:59.940 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 7 2023-08-25T08:56:59.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:56:59 smithi181 ceph-mon[131296]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:56:59.994 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 08:56:59 smithi181 podman[157062]: 2023-08-25T08:57:00.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:56:59 smithi099 ceph-mon[97641]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:00.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:56:59 smithi157 ceph-mon[130879]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:00.492 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-mds[157102]: -- 172.21.15.181:0/974339468 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621570ed1e0 con 0x5621570c3000 2023-08-25T08:57:00.493 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-c[157078]: starting mds.c at 2023-08-25T08:57:00.493 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 08:57:00 smithi181 bash[157062]: f341dae8b1114a8a063f157e6cc9a6a1c824abcc735538b6a208b1f770f4dc54 2023-08-25T08:57:00.493 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 08:57:00 smithi181 systemd[1]: Started Ceph mds.c for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:57:01.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:00 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2400262705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:01.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:01.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:01.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:00 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:01.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2400262705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:01.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:01.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:01.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:00 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:00 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2400262705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:01.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:01.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:01.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:01.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:00 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:01.879 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 08:57:01 smithi157 systemd[1]: Starting Ceph mds.h for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:57:01.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:57:02.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 -- 172.21.15.99:0/2768332704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 msgr2=0x7f3fd8101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:02.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2768332704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 0x7f3fd8101f70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40099c0 tx=0x7f3fc4009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 -- 172.21.15.99:0/2768332704 shutdown_connections 2023-08-25T08:57:02.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2768332704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd81024b0 0x7f3fd8108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2768332704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fd8100920 0x7f3fd8100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2768332704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 0x7f3fd8101f70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 -- 172.21.15.99:0/2768332704 >> 172.21.15.99:0/2768332704 conn(0x7f3fd80fbe50 msgr2=0x7f3fd80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:02.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 -- 172.21.15.99:0/2768332704 shutdown_connections 2023-08-25T08:57:02.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.164+0000 7f3fdedff700 1 -- 172.21.15.99:0/2768332704 wait complete. 2023-08-25T08:57:02.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.166+0000 7f3fdedff700 1 Processor -- start 2023-08-25T08:57:02.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.166+0000 7f3fdedff700 1 -- start start 2023-08-25T08:57:02.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.166+0000 7f3fdedff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8100920 0x7f3fd80746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:02.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fdedff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 0x7f3fd8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:02.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fdedff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fd81024b0 0x7f3fd8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:02.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fdedff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80739f0 con 0x7f3fd8101b10 2023-08-25T08:57:02.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fdedff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd8073b60 con 0x7f3fd8100920 2023-08-25T08:57:02.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fdedff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd81a06c0 con 0x7f3fd81024b0 2023-08-25T08:57:02.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 0x7f3fd8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:02.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 0x7f3fd8072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47882/0 (socket says 172.21.15.99:47882) 2023-08-25T08:57:02.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fdcb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8100920 0x7f3fd80746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:02.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 -- 172.21.15.99:0/2609913376 learned_addr learned my addr 172.21.15.99:0/2609913376 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:02.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fdd39c700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fd81024b0 0x7f3fd8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:02.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 -- 172.21.15.99:0/2609913376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fd81024b0 msgr2=0x7f3fd8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:02.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fd81024b0 0x7f3fd8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 -- 172.21.15.99:0/2609913376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8100920 msgr2=0x7f3fd80746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:02.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8100920 0x7f3fd80746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 -- 172.21.15.99:0/2609913376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40095f0 con 0x7f3fd8101b10 2023-08-25T08:57:02.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fdcb9b700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8100920 0x7f3fd80746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:02.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.167+0000 7f3fd7fff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 0x7f3fd8072d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400bf40 tx=0x7f3fc400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:02.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.168+0000 7f3fd5ffb700 1 -- 172.21.15.99:0/2609913376 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc401d070 con 0x7f3fd8101b10 2023-08-25T08:57:02.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.168+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd81a0940 con 0x7f3fd8101b10 2023-08-25T08:57:02.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.168+0000 7f3fd5ffb700 1 -- 172.21.15.99:0/2609913376 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc4005510 con 0x7f3fd8101b10 2023-08-25T08:57:02.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.168+0000 7f3fd5ffb700 1 -- 172.21.15.99:0/2609913376 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4004c00 con 0x7f3fd8101b10 2023-08-25T08:57:02.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.168+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd81a0dc0 con 0x7f3fd8101b10 2023-08-25T08:57:02.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.169+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd8107ec0 con 0x7f3fd8101b10 2023-08-25T08:57:02.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.171+0000 7f3fd5ffb700 1 -- 172.21.15.99:0/2609913376 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f3fc400fb80 con 0x7f3fd8101b10 2023-08-25T08:57:02.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.172+0000 7f3fd5ffb700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fc809ffa0 0x7f3fc80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:02.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.172+0000 7f3fd5ffb700 1 -- 172.21.15.99:0/2609913376 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3fc40cc110 con 0x7f3fd8101b10 2023-08-25T08:57:02.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.173+0000 7f3fdcb9b700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fc809ffa0 0x7f3fc80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:02.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.173+0000 7f3fdcb9b700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fc809ffa0 0x7f3fc80a2460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8101970 tx=0x7f3fcc007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:02.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.174+0000 7f3fd5ffb700 1 -- 172.21.15.99:0/2609913376 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3fc4096330 con 0x7f3fd8101b10 2023-08-25T08:57:02.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:01 smithi181 ceph-mon[131296]: Deploying daemon mds.h on smithi157 2023-08-25T08:57:02.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:01 smithi181 ceph-mon[131296]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:02.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:01 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6836/2291370984,v1:172.21.15.181:6837/2291370984] up:boot 2023-08-25T08:57:02.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:01 smithi181 ceph-mon[131296]: fsmap 7 up:standby 2023-08-25T08:57:02.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:01 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T08:57:02.264 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 08:57:02 smithi157 podman[155548]: 2023-08-25T08:57:02.264 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:01 smithi157 ceph-mon[130879]: Deploying daemon mds.h on smithi157 2023-08-25T08:57:02.265 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:01 smithi157 ceph-mon[130879]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:02.265 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:01 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6836/2291370984,v1:172.21.15.181:6837/2291370984] up:boot 2023-08-25T08:57:02.265 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:01 smithi157 ceph-mon[130879]: fsmap 7 up:standby 2023-08-25T08:57:02.265 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:01 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T08:57:02.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:01 smithi099 ceph-mon[97641]: Deploying daemon mds.h on smithi157 2023-08-25T08:57:02.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:01 smithi099 ceph-mon[97641]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:02.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:01 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6836/2291370984,v1:172.21.15.181:6837/2291370984] up:boot 2023-08-25T08:57:02.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:01 smithi099 ceph-mon[97641]: fsmap 7 up:standby 2023-08-25T08:57:02.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:01 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T08:57:02.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.423+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fd810c180 con 0x7f3fd8101b10 2023-08-25T08:57:02.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.424+0000 7f3fd5ffb700 1 -- 172.21.15.99:0/2609913376 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7f3fc4099950 con 0x7f3fd8101b10 2023-08-25T08:57:02.425 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:02.425 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-08-25T08:57:02.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fc809ffa0 msgr2=0x7f3fc80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:02.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fc809ffa0 0x7f3fc80a2460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8101970 tx=0x7f3fcc007400 comp rx=0 tx=0).stop 2023-08-25T08:57:02.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 msgr2=0x7f3fd8072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:02.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 0x7f3fd8072d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400bf40 tx=0x7f3fc400f690 comp rx=0 tx=0).stop 2023-08-25T08:57:02.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 shutdown_connections 2023-08-25T08:57:02.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fd81024b0 0x7f3fd8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fc809ffa0 0x7f3fc80a2460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8100920 0x7f3fd80746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 --2- 172.21.15.99:0/2609913376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd8101b10 0x7f3fd8072d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:02.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 >> 172.21.15.99:0/2609913376 conn(0x7f3fd80fbe50 msgr2=0x7f3fd8105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:02.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 shutdown_connections 2023-08-25T08:57:02.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:02.426+0000 7f3fdedff700 1 -- 172.21.15.99:0/2609913376 wait complete. 2023-08-25T08:57:02.429 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-08-25T08:57:02.629 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 08:57:02 smithi157 ceph-mds[155590]: -- 172.21.15.157:0/808972912 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557203d3b1e0 con 0x557203d11000 2023-08-25T08:57:02.629 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 08:57:02 smithi157 bash[155548]: 20a140486cbe208248c44a1ae94f3ee54cf2f594ec9a29bea68579642519d571 2023-08-25T08:57:02.629 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 08:57:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: starting mds.h at 2023-08-25T08:57:02.630 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 08:57:02 smithi157 systemd[1]: Started Ceph mds.h for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:57:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:03.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:03.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:03.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: Deploying daemon mds.i on smithi181 2023-08-25T08:57:03.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2609913376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:03.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:03.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6836/135740173,v1:172.21.15.157:6837/135740173] up:boot 2023-08-25T08:57:03.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: fsmap 8 up:standby 2023-08-25T08:57:03.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:03 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T08:57:03.632 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.632 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.632 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.632 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:03.632 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:03.633 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:03.633 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: Deploying daemon mds.i on smithi181 2023-08-25T08:57:03.633 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2609913376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:03.633 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:03.633 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6836/135740173,v1:172.21.15.157:6837/135740173] up:boot 2023-08-25T08:57:03.633 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: fsmap 8 up:standby 2023-08-25T08:57:03.633 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:03 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T08:57:03.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: Deploying daemon mds.i on smithi181 2023-08-25T08:57:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2609913376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:03.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:03.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6836/135740173,v1:172.21.15.157:6837/135740173] up:boot 2023-08-25T08:57:03.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: fsmap 8 up:standby 2023-08-25T08:57:03.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:03 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T08:57:03.913 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 08:57:03 smithi181 systemd[1]: Starting Ceph mds.i for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:57:04.443 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 08:57:04 smithi181 podman[157403]: 2023-08-25T08:57:04.443 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 08:57:04 smithi181 ceph-mds[157443]: -- 172.21.15.181:0/2683624088 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563974cd11e0 con 0x563974ca6c00 2023-08-25T08:57:04.447 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:57:04.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 -- 172.21.15.99:0/5217910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc100930 msgr2=0x7f45bc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:04.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 --2- 172.21.15.99:0/5217910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc100930 0x7f45bc100d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009990 tx=0x7f45b8009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 -- 172.21.15.99:0/5217910 shutdown_connections 2023-08-25T08:57:04.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 --2- 172.21.15.99:0/5217910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45bc1024c0 0x7f45bc108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 --2- 172.21.15.99:0/5217910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc100930 0x7f45bc100d50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 --2- 172.21.15.99:0/5217910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc101b20 0x7f45bc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 -- 172.21.15.99:0/5217910 >> 172.21.15.99:0/5217910 conn(0x7f45bc0fbe60 msgr2=0x7f45bc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:04.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 -- 172.21.15.99:0/5217910 shutdown_connections 2023-08-25T08:57:04.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.659+0000 7f45c47ff700 1 -- 172.21.15.99:0/5217910 wait complete. 2023-08-25T08:57:04.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.660+0000 7f45c47ff700 1 Processor -- start 2023-08-25T08:57:04.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.660+0000 7f45c47ff700 1 -- start start 2023-08-25T08:57:04.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.660+0000 7f45c47ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45bc100930 0x7f45bc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:04.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.660+0000 7f45c47ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc101b20 0x7f45bc198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:04.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c47ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc1024c0 0x7f45bc19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:04.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c47ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc1a03a0 con 0x7f45bc100930 2023-08-25T08:57:04.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c47ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc1a04e0 con 0x7f45bc1024c0 2023-08-25T08:57:04.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c47ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc1a0670 con 0x7f45bc101b20 2023-08-25T08:57:04.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c1d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc101b20 0x7f45bc198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:04.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c259b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45bc100930 0x7f45bc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:04.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c1d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc101b20 0x7f45bc198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:32826/0 (socket says 172.21.15.99:32826) 2023-08-25T08:57:04.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c1d9a700 1 -- 172.21.15.99:0/1741887051 learned_addr learned my addr 172.21.15.99:0/1741887051 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:04.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c2d9c700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc1024c0 0x7f45bc19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:04.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c259b700 1 -- 172.21.15.99:0/1741887051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc101b20 msgr2=0x7f45bc198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:04.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c259b700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc101b20 0x7f45bc198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c259b700 1 -- 172.21.15.99:0/1741887051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc1024c0 msgr2=0x7f45bc19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:04.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c259b700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc1024c0 0x7f45bc19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c259b700 1 -- 172.21.15.99:0/1741887051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80095f0 con 0x7f45bc100930 2023-08-25T08:57:04.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c1d9a700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc101b20 0x7f45bc198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:04.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.661+0000 7f45c259b700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45bc100930 0x7f45bc198820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45b80098e0 tx=0x7f45b8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:04.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.662+0000 7f45b37fe700 1 -- 172.21.15.99:0/1741887051 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b801d070 con 0x7f45bc100930 2023-08-25T08:57:04.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.662+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc1a08f0 con 0x7f45bc100930 2023-08-25T08:57:04.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.662+0000 7f45b37fe700 1 -- 172.21.15.99:0/1741887051 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b8021410 con 0x7f45bc100930 2023-08-25T08:57:04.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.662+0000 7f45b37fe700 1 -- 172.21.15.99:0/1741887051 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8005720 con 0x7f45bc100930 2023-08-25T08:57:04.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.662+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc1a0d70 con 0x7f45bc100930 2023-08-25T08:57:04.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.663+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc10c190 con 0x7f45bc100930 2023-08-25T08:57:04.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.665+0000 7f45b37fe700 1 -- 172.21.15.99:0/1741887051 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f45b8021580 con 0x7f45bc100930 2023-08-25T08:57:04.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.666+0000 7f45b37fe700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f45a80a0020 0x7f45a80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:04.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.666+0000 7f45b37fe700 1 -- 172.21.15.99:0/1741887051 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f45b80cc100 con 0x7f45bc100930 2023-08-25T08:57:04.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.666+0000 7f45c1d9a700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f45a80a0020 0x7f45a80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:04.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.667+0000 7f45c1d9a700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f45a80a0020 0x7f45a80a24e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45ac011550 tx=0x7f45ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:04.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.671+0000 7f45b37fe700 1 -- 172.21.15.99:0/1741887051 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f45b8096320 con 0x7f45bc100930 2023-08-25T08:57:04.742 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 08:57:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-i[157419]: starting mds.i at 2023-08-25T08:57:04.743 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 08:57:04 smithi181 bash[157403]: 1e50289198081ba79551d4cbf92a194e49a531a2ae7dede46248e96b67c1c380 2023-08-25T08:57:04.743 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 08:57:04 smithi181 systemd[1]: Started Ceph mds.i for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:57:04.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.937+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45bc10c3a0 con 0x7f45bc100930 2023-08-25T08:57:04.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.938+0000 7f45b37fe700 1 -- 172.21.15.99:0/1741887051 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f45b8099940 con 0x7f45bc100930 2023-08-25T08:57:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:04.939 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24545,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-08-25T08:57:04.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f45a80a0020 msgr2=0x7f45a80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:04.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f45a80a0020 0x7f45a80a24e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45ac011550 tx=0x7f45ac009260 comp rx=0 tx=0).stop 2023-08-25T08:57:04.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45bc100930 msgr2=0x7f45bc198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:04.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45bc100930 0x7f45bc198820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45b80098e0 tx=0x7f45b8009580 comp rx=0 tx=0).stop 2023-08-25T08:57:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 shutdown_connections 2023-08-25T08:57:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f45a80a0020 0x7f45a80a24e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc1024c0 0x7f45bc19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45bc100930 0x7f45bc198820 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 --2- 172.21.15.99:0/1741887051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45bc101b20 0x7f45bc198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.941+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 >> 172.21.15.99:0/1741887051 conn(0x7f45bc0fbe60 msgr2=0x7f45bc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:04.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.942+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 shutdown_connections 2023-08-25T08:57:04.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:04.942+0000 7f45c47ff700 1 -- 172.21.15.99:0/1741887051 wait complete. 2023-08-25T08:57:04.944 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 9 2023-08-25T08:57:05.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:05.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:05.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:05.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: Deploying daemon mds.g on smithi099 2023-08-25T08:57:05.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1741887051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:05.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] up:boot 2023-08-25T08:57:05.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: fsmap 9 up:standby 2023-08-25T08:57:05.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:05 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T08:57:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:05.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:05.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:05.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: Deploying daemon mds.g on smithi099 2023-08-25T08:57:05.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1741887051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:05.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] up:boot 2023-08-25T08:57:05.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: fsmap 9 up:standby 2023-08-25T08:57:05.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:05 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T08:57:05.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:05.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:05.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:05.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:05.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:05.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: Deploying daemon mds.g on smithi099 2023-08-25T08:57:05.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1741887051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:05.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] up:boot 2023-08-25T08:57:05.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: fsmap 9 up:standby 2023-08-25T08:57:05.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:05 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T08:57:06.840 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 08:57:06 smithi099 systemd[1]: Starting Ceph mds.g for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:57:06.970 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:57:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 -- 172.21.15.99:0/980773228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 msgr2=0x7f2290108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 --2- 172.21.15.99:0/980773228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 0x7f2290108d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f228800b0d0 tx=0x7f228800b3e0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 -- 172.21.15.99:0/980773228 shutdown_connections 2023-08-25T08:57:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 --2- 172.21.15.99:0/980773228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 0x7f2290108d70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 --2- 172.21.15.99:0/980773228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2290100930 0x7f2290100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 --2- 172.21.15.99:0/980773228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290101b20 0x7f2290101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 -- 172.21.15.99:0/980773228 >> 172.21.15.99:0/980773228 conn(0x7f22900fbe60 msgr2=0x7f22900fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 -- 172.21.15.99:0/980773228 shutdown_connections 2023-08-25T08:57:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.199+0000 7f2295fff700 1 -- 172.21.15.99:0/980773228 wait complete. 2023-08-25T08:57:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.200+0000 7f2295fff700 1 Processor -- start 2023-08-25T08:57:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.200+0000 7f2295fff700 1 -- start start 2023-08-25T08:57:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f2295fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2290100930 0x7f2290198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f2295fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290101b20 0x7f2290198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f2295fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 0x7f229019ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f2295fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22901a04f0 con 0x7f22901024c0 2023-08-25T08:57:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f2295fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22901a0630 con 0x7f2290101b20 2023-08-25T08:57:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f2295fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22901a07c0 con 0x7f2290100930 2023-08-25T08:57:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 0x7f229019ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2290100930 0x7f2290198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 0x7f229019ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54700/0 (socket says 172.21.15.99:54700) 2023-08-25T08:57:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228ffff700 1 -- 172.21.15.99:0/3913649376 learned_addr learned my addr 172.21.15.99:0/3913649376 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228effd700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290101b20 0x7f2290198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228f7fe700 1 -- 172.21.15.99:0/3913649376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290101b20 msgr2=0x7f2290198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228f7fe700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290101b20 0x7f2290198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228f7fe700 1 -- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 msgr2=0x7f229019ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228f7fe700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 0x7f229019ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228f7fe700 1 -- 172.21.15.99:0/3913649376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288009d00 con 0x7f2290100930 2023-08-25T08:57:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228effd700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290101b20 0x7f2290198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.201+0000 7f228ffff700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 0x7f229019ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.202+0000 7f228f7fe700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2290100930 0x7f2290198970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f228000e9c0 tx=0x7f228000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.202+0000 7f228cff9700 1 -- 172.21.15.99:0/3913649376 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000cbd0 con 0x7f2290100930 2023-08-25T08:57:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.202+0000 7f228cff9700 1 -- 172.21.15.99:0/3913649376 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f228000eeb0 con 0x7f2290100930 2023-08-25T08:57:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.202+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22901a0aa0 con 0x7f2290100930 2023-08-25T08:57:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.202+0000 7f228cff9700 1 -- 172.21.15.99:0/3913649376 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280003c70 con 0x7f2290100930 2023-08-25T08:57:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.202+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22901a1000 con 0x7f2290100930 2023-08-25T08:57:07.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.203+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229010c1a0 con 0x7f2290100930 2023-08-25T08:57:07.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.205+0000 7f228cff9700 1 -- 172.21.15.99:0/3913649376 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f2280003dd0 con 0x7f2290100930 2023-08-25T08:57:07.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.206+0000 7f228cff9700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f22780a0020 0x7f22780a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:07.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.207+0000 7f228cff9700 1 -- 172.21.15.99:0/3913649376 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2280014070 con 0x7f2290100930 2023-08-25T08:57:07.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.207+0000 7f228effd700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f22780a0020 0x7f22780a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:07.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.207+0000 7f228effd700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f22780a0020 0x7f22780a24e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2284009c90 tx=0x7f2284009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:07.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.211+0000 7f228cff9700 1 -- 172.21.15.99:0/3913649376 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f22800951f0 con 0x7f2290100930 2023-08-25T08:57:07.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.481+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f229010c3e0 con 0x7f2290100930 2023-08-25T08:57:07.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.482+0000 7f228cff9700 1 -- 172.21.15.99:0/3913649376 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7f2280098810 con 0x7f2290100930 2023-08-25T08:57:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:07.483 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24545,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-25T08:57:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f22780a0020 msgr2=0x7f22780a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f22780a0020 0x7f22780a24e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2284009c90 tx=0x7f2284009260 comp rx=0 tx=0).stop 2023-08-25T08:57:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2290100930 msgr2=0x7f2290198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2290100930 0x7f2290198970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f228000e9c0 tx=0x7f228000ecd0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 shutdown_connections 2023-08-25T08:57:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22901024c0 0x7f229019ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2290100930 0x7f2290198970 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f228000e9c0 tx=0x7f228000ecd0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f22780a0020 0x7f22780a24e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 --2- 172.21.15.99:0/3913649376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2290101b20 0x7f2290198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 >> 172.21.15.99:0/3913649376 conn(0x7f22900fbe60 msgr2=0x7f2290107380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 shutdown_connections 2023-08-25T08:57:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:07.485+0000 7f2295fff700 1 -- 172.21.15.99:0/3913649376 wait complete. 2023-08-25T08:57:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 10 2023-08-25T08:57:07.615 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:07 smithi099 ceph-mon[97641]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:07.616 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:07 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3913649376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:07.616 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 08:57:07 smithi099 podman[132018]: 2023-08-25T08:57:07.616 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 08:57:07 smithi099 bash[132018]: 4fccd29d3f228d2546616bb45f97a68f2eebfaa94133cc4d58ccae74fe8b2ca6 2023-08-25T08:57:07.617 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 08:57:07 smithi099 ceph-mds[132080]: -- 172.21.15.99:0/3206407530 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5651d835b1e0 con 0x5651d8330800 2023-08-25T08:57:07.617 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 08:57:07 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-g[132054]: starting mds.g at 2023-08-25T08:57:07.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:07 smithi157 ceph-mon[130879]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:07.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:07 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3913649376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:07.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:07 smithi181 ceph-mon[131296]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:07.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:07 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3913649376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:08.062 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 08:57:07 smithi099 systemd[1]: Started Ceph mds.g for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:57:08.868 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:08.869 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:08.869 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:08.869 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:08.869 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:08.869 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:08.869 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: Deploying daemon mds.a on smithi099 2023-08-25T08:57:08.870 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.99:6838/2687363843,v1:172.21.15.99:6839/2687363843] up:boot 2023-08-25T08:57:08.870 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: fsmap 10 up:standby 2023-08-25T08:57:08.870 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:08 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T08:57:08.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:08.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: Deploying daemon mds.a on smithi099 2023-08-25T08:57:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.99:6838/2687363843,v1:172.21.15.99:6839/2687363843] up:boot 2023-08-25T08:57:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: fsmap 10 up:standby 2023-08-25T08:57:08.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:08 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T08:57:09.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:09.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:09.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:09.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:09.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:09.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:09.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: Deploying daemon mds.a on smithi099 2023-08-25T08:57:09.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.99:6838/2687363843,v1:172.21.15.99:6839/2687363843] up:boot 2023-08-25T08:57:09.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: fsmap 10 up:standby 2023-08-25T08:57:09.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:08 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T08:57:09.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:57:09.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 -- 172.21.15.99:0/495687974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4828100930 msgr2=0x7f4828100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:09.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 --2- 172.21.15.99:0/495687974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4828100930 0x7f4828100d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4818009920 tx=0x7f4818009c30 comp rx=0 tx=0).stop 2023-08-25T08:57:09.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 -- 172.21.15.99:0/495687974 shutdown_connections 2023-08-25T08:57:09.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 --2- 172.21.15.99:0/495687974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48281024c0 0x7f4828108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:09.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 --2- 172.21.15.99:0/495687974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4828100930 0x7f4828100d50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:09.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 --2- 172.21.15.99:0/495687974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4828101b20 0x7f4828101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:09.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 -- 172.21.15.99:0/495687974 >> 172.21.15.99:0/495687974 conn(0x7f48280fbe60 msgr2=0x7f48280fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:09.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 -- 172.21.15.99:0/495687974 shutdown_connections 2023-08-25T08:57:09.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.766+0000 7f482d3ff700 1 -- 172.21.15.99:0/495687974 wait complete. 2023-08-25T08:57:09.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.768+0000 7f482d3ff700 1 Processor -- start 2023-08-25T08:57:09.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.768+0000 7f482d3ff700 1 -- start start 2023-08-25T08:57:09.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.768+0000 7f482d3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4828100930 0x7f48280746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:09.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.768+0000 7f482d3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4828101b20 0x7f4828072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:09.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.768+0000 7f482d3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48281024c0 0x7f4828073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:09.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.768+0000 7f482d3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48280739f0 con 0x7f48281024c0 2023-08-25T08:57:09.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.768+0000 7f482d3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828073b60 con 0x7f4828100930 2023-08-25T08:57:09.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.768+0000 7f482d3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48281a06e0 con 0x7f4828101b20 2023-08-25T08:57:09.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4828100930 0x7f48280746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:09.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4828100930 0x7f48280746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:46802/0 (socket says 172.21.15.99:46802) 2023-08-25T08:57:09.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f48277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48281024c0 0x7f4828073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:09.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 -- 172.21.15.99:0/1651074176 learned_addr learned my addr 172.21.15.99:0/1651074176 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:09.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f48267fc700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4828101b20 0x7f4828072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:09.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 -- 172.21.15.99:0/1651074176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4828101b20 msgr2=0x7f4828072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:09.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4828101b20 0x7f4828072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:09.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 -- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48281024c0 msgr2=0x7f4828073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:09.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48281024c0 0x7f4828073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:09.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 -- 172.21.15.99:0/1651074176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c009520 con 0x7f4828100930 2023-08-25T08:57:09.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f48277fe700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48281024c0 0x7f4828073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:09.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.769+0000 7f4826ffd700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4828100930 0x7f48280746d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4818009230 tx=0x7f481800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:09.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.770+0000 7f480ffff700 1 -- 172.21.15.99:0/1651074176 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481801c070 con 0x7f4828100930 2023-08-25T08:57:09.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.770+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4818009520 con 0x7f4828100930 2023-08-25T08:57:09.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.770+0000 7f480ffff700 1 -- 172.21.15.99:0/1651074176 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481800f7d0 con 0x7f4828100930 2023-08-25T08:57:09.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.770+0000 7f480ffff700 1 -- 172.21.15.99:0/1651074176 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818004930 con 0x7f4828100930 2023-08-25T08:57:09.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.770+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48281a0ba0 con 0x7f4828100930 2023-08-25T08:57:09.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.771+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482810c190 con 0x7f4828100930 2023-08-25T08:57:09.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.773+0000 7f480ffff700 1 -- 172.21.15.99:0/1651074176 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f4818020070 con 0x7f4828100930 2023-08-25T08:57:09.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.774+0000 7f480ffff700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f481009ffa0 0x7f48100a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:09.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.774+0000 7f480ffff700 1 -- 172.21.15.99:0/1651074176 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48180cba20 con 0x7f4828100930 2023-08-25T08:57:09.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.774+0000 7f48267fc700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f481009ffa0 0x7f48100a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:09.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.775+0000 7f48267fc700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f481009ffa0 0x7f48100a2460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f481c009df0 tx=0x7f481c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:09.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:09.776+0000 7f480ffff700 1 -- 172.21.15.99:0/1651074176 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4818095cc0 con 0x7f4828100930 2023-08-25T08:57:09.860 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:09 smithi099 ceph-mon[97641]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:09.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:09 smithi181 ceph-mon[131296]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:10.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.038+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48281078c0 con 0x7f4828100930 2023-08-25T08:57:10.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.039+0000 7f480ffff700 1 -- 172.21.15.99:0/1651074176 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8459 (secure 0 0 0) 0x7f48180992e0 con 0x7f4828100930 2023-08-25T08:57:10.040 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:10.040 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24545,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-25T08:57:10.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f481009ffa0 msgr2=0x7f48100a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f481009ffa0 0x7f48100a2460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f481c009df0 tx=0x7f481c009260 comp rx=0 tx=0).stop 2023-08-25T08:57:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4828100930 msgr2=0x7f48280746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4828100930 0x7f48280746d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4818009230 tx=0x7f481800b790 comp rx=0 tx=0).stop 2023-08-25T08:57:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 shutdown_connections 2023-08-25T08:57:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48281024c0 0x7f4828073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f481009ffa0 0x7f48100a2460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4828100930 0x7f48280746d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:10.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 --2- 172.21.15.99:0/1651074176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4828101b20 0x7f4828072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:10.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 >> 172.21.15.99:0/1651074176 conn(0x7f48280fbe60 msgr2=0x7f48280fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:10.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 shutdown_connections 2023-08-25T08:57:10.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:10.042+0000 7f482d3ff700 1 -- 172.21.15.99:0/1651074176 wait complete. 2023-08-25T08:57:10.044 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 11 2023-08-25T08:57:10.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:09 smithi157 ceph-mon[130879]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:10.312 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 08:57:09 smithi099 systemd[1]: Starting Ceph mds.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:57:10.631 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 08:57:10 smithi099 podman[132421]: 2023-08-25T08:57:10.632 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 08:57:10 smithi099 bash[132421]: 5e8997ffbf0c19158a24aed8fa7d9c2e6c895fd5eefcdb521e91435ba9c3aac9 2023-08-25T08:57:10.632 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 08:57:10 smithi099 ceph-mds[132461]: -- 172.21.15.99:0/2804280380 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f692e591e0 con 0x55f692e2f000 2023-08-25T08:57:10.632 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 08:57:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-a[132437]: starting mds.a at 2023-08-25T08:57:10.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:10 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1651074176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:10.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:10 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:10.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:10 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:11.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:10 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1651074176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:11.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:11.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:10 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:11.063 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 08:57:10 smithi099 systemd[1]: Started Ceph mds.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:57:11.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:10 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1651074176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:11.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:10 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:11.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:10 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:11.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:11.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.99:6840/1596550306,v1:172.21.15.99:6841/1596550306] up:boot 2023-08-25T08:57:11.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: fsmap 11 up:standby 2023-08-25T08:57:11.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T08:57:11.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:11.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:11.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:11.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:11.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:11 smithi181 ceph-mon[131296]: Deploying daemon mds.b on smithi157 2023-08-25T08:57:12.002 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:12.003 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.99:6840/1596550306,v1:172.21.15.99:6841/1596550306] up:boot 2023-08-25T08:57:12.003 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: fsmap 11 up:standby 2023-08-25T08:57:12.003 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T08:57:12.003 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:12.003 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:12.003 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:12.003 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:12.004 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:11 smithi157 ceph-mon[130879]: Deploying daemon mds.b on smithi157 2023-08-25T08:57:12.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:12.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.99:6840/1596550306,v1:172.21.15.99:6841/1596550306] up:boot 2023-08-25T08:57:12.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: fsmap 11 up:standby 2023-08-25T08:57:12.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T08:57:12.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:12.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T08:57:12.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T08:57:12.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:12.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:11 smithi099 ceph-mon[97641]: Deploying daemon mds.b on smithi157 2023-08-25T08:57:12.064 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:57:12.252 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 08:57:12 smithi157 systemd[1]: Starting Ceph mds.b for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T08:57:12.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.274+0000 7f14ec5ff700 1 -- 172.21.15.99:0/590826721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e4101b10 msgr2=0x7f14e4101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:12.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.274+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/590826721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e4101b10 0x7f14e4101f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f14d40099c0 tx=0x7f14d4009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.274+0000 7f14ec5ff700 1 -- 172.21.15.99:0/590826721 shutdown_connections 2023-08-25T08:57:12.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.274+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/590826721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14e41024b0 0x7f14e4108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.274+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/590826721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14e4100920 0x7f14e4100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.274+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/590826721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e4101b10 0x7f14e4101f70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.275+0000 7f14ec5ff700 1 -- 172.21.15.99:0/590826721 >> 172.21.15.99:0/590826721 conn(0x7f14e40fbe50 msgr2=0x7f14e40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:12.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.275+0000 7f14ec5ff700 1 -- 172.21.15.99:0/590826721 shutdown_connections 2023-08-25T08:57:12.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.275+0000 7f14ec5ff700 1 -- 172.21.15.99:0/590826721 wait complete. 2023-08-25T08:57:12.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ec5ff700 1 Processor -- start 2023-08-25T08:57:12.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ec5ff700 1 -- start start 2023-08-25T08:57:12.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ec5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14e4100920 0x7f14e4196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:12.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ec5ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14e4101b10 0x7f14e4196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:12.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ec5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41024b0 0x7f14e419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:12.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ec5ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e419e190 con 0x7f14e4101b10 2023-08-25T08:57:12.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ec5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e419e2d0 con 0x7f14e41024b0 2023-08-25T08:57:12.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ec5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e419e460 con 0x7f14e4100920 2023-08-25T08:57:12.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14e9b9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14e4101b10 0x7f14e4196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:12.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14eab9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41024b0 0x7f14e419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:12.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14eab9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41024b0 0x7f14e419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:46852/0 (socket says 172.21.15.99:46852) 2023-08-25T08:57:12.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14ea39b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14e4100920 0x7f14e4196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:12.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.276+0000 7f14eab9c700 1 -- 172.21.15.99:0/2848823427 learned_addr learned my addr 172.21.15.99:0/2848823427 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:12.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14e9b9a700 1 -- 172.21.15.99:0/2848823427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14e4100920 msgr2=0x7f14e4196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:12.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14e9b9a700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14e4100920 0x7f14e4196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14e9b9a700 1 -- 172.21.15.99:0/2848823427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41024b0 msgr2=0x7f14e419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:12.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14e9b9a700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41024b0 0x7f14e419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14e9b9a700 1 -- 172.21.15.99:0/2848823427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e0009520 con 0x7f14e4101b10 2023-08-25T08:57:12.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14eab9c700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41024b0 0x7f14e419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:12.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14e9b9a700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14e4101b10 0x7f14e4196ba0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14d40098e0 tx=0x7f14d401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:12.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14db7fe700 1 -- 172.21.15.99:0/2848823427 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4021070 con 0x7f14e4101b10 2023-08-25T08:57:12.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d40095f0 con 0x7f14e4101b10 2023-08-25T08:57:12.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14db7fe700 1 -- 172.21.15.99:0/2848823427 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14d40054e0 con 0x7f14e4101b10 2023-08-25T08:57:12.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14db7fe700 1 -- 172.21.15.99:0/2848823427 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4004c10 con 0x7f14e4101b10 2023-08-25T08:57:12.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.277+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e4072e40 con 0x7f14e4101b10 2023-08-25T08:57:12.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.278+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e4107f60 con 0x7f14e4101b10 2023-08-25T08:57:12.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.280+0000 7f14db7fe700 1 -- 172.21.15.99:0/2848823427 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f14d4025070 con 0x7f14e4101b10 2023-08-25T08:57:12.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.282+0000 7f14db7fe700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f14d009ffa0 0x7f14d00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:12.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.282+0000 7f14db7fe700 1 -- 172.21.15.99:0/2848823427 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14d40cfde0 con 0x7f14e4101b10 2023-08-25T08:57:12.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.282+0000 7f14ea39b700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f14d009ffa0 0x7f14d00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:12.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.282+0000 7f14ea39b700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f14d009ffa0 0x7f14d00a2460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14e4101970 tx=0x7f14e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:12.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.286+0000 7f14db7fe700 1 -- 172.21.15.99:0/2848823427 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f14d409a000 con 0x7f14e4101b10 2023-08-25T08:57:12.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.527+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14e402ccc0 con 0x7f14e4101b10 2023-08-25T08:57:12.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.528+0000 7f14db7fe700 1 -- 172.21.15.99:0/2848823427 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7f14d409d620 con 0x7f14e4101b10 2023-08-25T08:57:12.529 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24545,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-25T08:57:12.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.530+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f14d009ffa0 msgr2=0x7f14d00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:12.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.530+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f14d009ffa0 0x7f14d00a2460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14e4101970 tx=0x7f14e0009260 comp rx=0 tx=0).stop 2023-08-25T08:57:12.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.530+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14e4101b10 msgr2=0x7f14e4196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:12.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.530+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14e4101b10 0x7f14e4196ba0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14d40098e0 tx=0x7f14d401ca40 comp rx=0 tx=0).stop 2023-08-25T08:57:12.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.530+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 shutdown_connections 2023-08-25T08:57:12.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.531+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f14d009ffa0 0x7f14d00a2460 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.531+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41024b0 0x7f14e419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.531+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14e4100920 0x7f14e4196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.531+0000 7f14ec5ff700 1 --2- 172.21.15.99:0/2848823427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14e4101b10 0x7f14e4196ba0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:12.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.531+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 >> 172.21.15.99:0/2848823427 conn(0x7f14e40fbe50 msgr2=0x7f14e41050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:12.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.531+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 shutdown_connections 2023-08-25T08:57:12.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:12.531+0000 7f14ec5ff700 1 -- 172.21.15.99:0/2848823427 wait complete. 2023-08-25T08:57:12.533 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 12 2023-08-25T08:57:12.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:12 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2848823427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:12.841 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:12 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2848823427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:12.842 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 08:57:12 smithi157 podman[155891]: 2023-08-25T08:57:12.842 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 08:57:12 smithi157 ceph-mds[155931]: -- 172.21.15.157:0/3919367749 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556c06a191e0 con 0x556c069eec00 2023-08-25T08:57:12.842 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 08:57:12 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-b[155907]: starting mds.b at 2023-08-25T08:57:12.842 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 08:57:12 smithi157 bash[155891]: 2d1d798cc84740b26607b4b93944dc70ef93968a926b6d8489aab135b75c40dc 2023-08-25T08:57:12.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:12 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2848823427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:13.129 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 08:57:12 smithi157 systemd[1]: Started Ceph mds.b for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T08:57:14.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:14.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:57:14.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] up:boot 2023-08-25T08:57:14.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: fsmap 12 up:standby 2023-08-25T08:57:14.244 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:13 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T08:57:14.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:14.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:57:14.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] up:boot 2023-08-25T08:57:14.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: fsmap 12 up:standby 2023-08-25T08:57:14.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:13 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T08:57:14.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:14.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:14.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:57:14.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] up:boot 2023-08-25T08:57:14.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: fsmap 12 up:standby 2023-08-25T08:57:14.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:13 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T08:57:14.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:57:14.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 -- 172.21.15.99:0/1958181516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc108800 msgr2=0x7fb1bc10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:14.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/1958181516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc108800 0x7fb1bc10ac70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b0a0 tx=0x7fb1b400b3b0 comp rx=0 tx=0).stop 2023-08-25T08:57:14.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 -- 172.21.15.99:0/1958181516 shutdown_connections 2023-08-25T08:57:14.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/1958181516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc108800 0x7fb1bc10ac70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:14.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/1958181516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1bc074900 0x7fb1bc072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:14.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/1958181516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1bc073320 0x7fb1bc0737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:14.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 -- 172.21.15.99:0/1958181516 >> 172.21.15.99:0/1958181516 conn(0x7fb1bc0fbe60 msgr2=0x7fb1bc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:14.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 -- 172.21.15.99:0/1958181516 shutdown_connections 2023-08-25T08:57:14.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.759+0000 7fb1c2dff700 1 -- 172.21.15.99:0/1958181516 wait complete. 2023-08-25T08:57:14.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.760+0000 7fb1c2dff700 1 Processor -- start 2023-08-25T08:57:14.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.760+0000 7fb1c2dff700 1 -- start start 2023-08-25T08:57:14.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.760+0000 7fb1c2dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc073320 0x7fb1bc1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:14.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.760+0000 7fb1c2dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1bc074900 0x7fb1bc198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:14.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c2dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1bc108800 0x7fb1bc19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:14.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c2dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a0450 con 0x7fb1bc073320 2023-08-25T08:57:14.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c2dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a0590 con 0x7fb1bc108800 2023-08-25T08:57:14.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c2dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a0720 con 0x7fb1bc074900 2023-08-25T08:57:14.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc073320 0x7fb1bc1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:14.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc073320 0x7fb1bc1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54780/0 (socket says 172.21.15.99:54780) 2023-08-25T08:57:14.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 -- 172.21.15.99:0/69990206 learned_addr learned my addr 172.21.15.99:0/69990206 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c139c700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1bc108800 0x7fb1bc19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1bbfff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1bc074900 0x7fb1bc198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 -- 172.21.15.99:0/69990206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1bc074900 msgr2=0x7fb1bc198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1bc074900 0x7fb1bc198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 -- 172.21.15.99:0/69990206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1bc108800 msgr2=0x7fb1bc19fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1bc108800 0x7fb1bc19fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 -- 172.21.15.99:0/69990206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b4009d00 con 0x7fb1bc073320 2023-08-25T08:57:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c139c700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1bc108800 0x7fb1bc19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:14.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.761+0000 7fb1c0b9b700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc073320 0x7fb1bc1988d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000e9a0 tx=0x7fb1b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:14.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.762+0000 7fb1b9ffb700 1 -- 172.21.15.99:0/69990206 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0009790 con 0x7fb1bc073320 2023-08-25T08:57:14.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.762+0000 7fb1b9ffb700 1 -- 172.21.15.99:0/69990206 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b0010430 con 0x7fb1bc073320 2023-08-25T08:57:14.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.762+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc1a0a00 con 0x7fb1bc073320 2023-08-25T08:57:14.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.762+0000 7fb1b9ffb700 1 -- 172.21.15.99:0/69990206 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0018680 con 0x7fb1bc073320 2023-08-25T08:57:14.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.762+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc1a0f60 con 0x7fb1bc073320 2023-08-25T08:57:14.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.763+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc106f00 con 0x7fb1bc073320 2023-08-25T08:57:14.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.765+0000 7fb1b9ffb700 1 -- 172.21.15.99:0/69990206 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fb1b00105a0 con 0x7fb1bc073320 2023-08-25T08:57:14.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.766+0000 7fb1b9ffb700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb1ac0a0020 0x7fb1ac0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:14.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.766+0000 7fb1b9ffb700 1 -- 172.21.15.99:0/69990206 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb1b0014070 con 0x7fb1bc073320 2023-08-25T08:57:14.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.766+0000 7fb1bbfff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb1ac0a0020 0x7fb1ac0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:14.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.767+0000 7fb1bbfff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb1ac0a0020 0x7fb1ac0a24e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009c60 tx=0x7fb1a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:14.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:14.770+0000 7fb1b9ffb700 1 -- 172.21.15.99:0/69990206 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb1b0051e00 con 0x7fb1bc073320 2023-08-25T08:57:15.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.031+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1bc10e310 con 0x7fb1bc073320 2023-08-25T08:57:15.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.032+0000 7fb1b9ffb700 1 -- 172.21.15.99:0/69990206 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7fb1b0095040 con 0x7fb1bc073320 2023-08-25T08:57:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:15.033 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24545,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24593,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-08-25T08:57:15.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.034+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb1ac0a0020 msgr2=0x7fb1ac0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.034+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb1ac0a0020 0x7fb1ac0a24e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009c60 tx=0x7fb1a8009260 comp rx=0 tx=0).stop 2023-08-25T08:57:15.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.034+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc073320 msgr2=0x7fb1bc1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc073320 0x7fb1bc1988d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000e9a0 tx=0x7fb1b000ecb0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 shutdown_connections 2023-08-25T08:57:15.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb1ac0a0020 0x7fb1ac0a24e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1bc108800 0x7fb1bc19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1bc073320 0x7fb1bc1988d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 --2- 172.21.15.99:0/69990206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1bc074900 0x7fb1bc198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 >> 172.21.15.99:0/69990206 conn(0x7fb1bc0fbe60 msgr2=0x7fb1bc0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:15.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 shutdown_connections 2023-08-25T08:57:15.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.035+0000 7fb1c2dff700 1 -- 172.21.15.99:0/69990206 wait complete. 2023-08-25T08:57:15.037 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 13 2023-08-25T08:57:15.055 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:57:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.261+0000 7f3965fff700 1 -- 172.21.15.99:0/2865555785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 msgr2=0x7f3960100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.261+0000 7f3965fff700 1 --2- 172.21.15.99:0/2865555785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 0x7f3960100d40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3950009990 tx=0x7f3950009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.262+0000 7f3965fff700 1 -- 172.21.15.99:0/2865555785 shutdown_connections 2023-08-25T08:57:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.262+0000 7f3965fff700 1 --2- 172.21.15.99:0/2865555785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39601024b0 0x7f3960108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.262+0000 7f3965fff700 1 --2- 172.21.15.99:0/2865555785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 0x7f3960100d40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.262+0000 7f3965fff700 1 --2- 172.21.15.99:0/2865555785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3960101b10 0x7f3960101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.262+0000 7f3965fff700 1 -- 172.21.15.99:0/2865555785 >> 172.21.15.99:0/2865555785 conn(0x7f39600fbe50 msgr2=0x7f39600fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.262+0000 7f3965fff700 1 -- 172.21.15.99:0/2865555785 shutdown_connections 2023-08-25T08:57:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.262+0000 7f3965fff700 1 -- 172.21.15.99:0/2865555785 wait complete. 2023-08-25T08:57:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f3965fff700 1 Processor -- start 2023-08-25T08:57:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f3965fff700 1 -- start start 2023-08-25T08:57:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f3965fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 0x7f3960198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f3965fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3960101b10 0x7f3960198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f3965fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39601024b0 0x7f396019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f3965fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601a0310 con 0x7f3960101b10 2023-08-25T08:57:15.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f3965fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601a0450 con 0x7f39601024b0 2023-08-25T08:57:15.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f3965fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601a05e0 con 0x7f3960100920 2023-08-25T08:57:15.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f395f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 0x7f3960198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:15.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f395f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 0x7f3960198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45762/0 (socket says 172.21.15.99:45762) 2023-08-25T08:57:15.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f395f7fe700 1 -- 172.21.15.99:0/3991498942 learned_addr learned my addr 172.21.15.99:0/3991498942 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:15.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f395effd700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3960101b10 0x7f3960198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:15.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.263+0000 7f395ffff700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39601024b0 0x7f396019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:15.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395f7fe700 1 -- 172.21.15.99:0/3991498942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39601024b0 msgr2=0x7f396019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395f7fe700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39601024b0 0x7f396019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395f7fe700 1 -- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3960101b10 msgr2=0x7f3960198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395f7fe700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3960101b10 0x7f3960198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395f7fe700 1 -- 172.21.15.99:0/3991498942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500095f0 con 0x7f3960100920 2023-08-25T08:57:15.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395f7fe700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 0x7f3960198790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3950000c00 tx=0x7f3950005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:15.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395cff9700 1 -- 172.21.15.99:0/3991498942 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395001d070 con 0x7f3960100920 2023-08-25T08:57:15.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395cff9700 1 -- 172.21.15.99:0/3991498942 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3950022470 con 0x7f3960100920 2023-08-25T08:57:15.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39601a0860 con 0x7f3960100920 2023-08-25T08:57:15.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.264+0000 7f395cff9700 1 -- 172.21.15.99:0/3991498942 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950017650 con 0x7f3960100920 2023-08-25T08:57:15.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.265+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39601a0ce0 con 0x7f3960100920 2023-08-25T08:57:15.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.266+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396010c130 con 0x7f3960100920 2023-08-25T08:57:15.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.267+0000 7f395cff9700 1 -- 172.21.15.99:0/3991498942 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f39500225e0 con 0x7f3960100920 2023-08-25T08:57:15.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.269+0000 7f395cff9700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f39480a0020 0x7f39480a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:15.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.269+0000 7f395cff9700 1 -- 172.21.15.99:0/3991498942 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f39500cc0c0 con 0x7f3960100920 2023-08-25T08:57:15.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.269+0000 7f395effd700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f39480a0020 0x7f39480a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:15.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.270+0000 7f395effd700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f39480a0020 0x7f39480a24e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3954009c60 tx=0x7f3954009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:15.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.271+0000 7f395cff9700 1 -- 172.21.15.99:0/3991498942 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f39500962e0 con 0x7f3960100920 2023-08-25T08:57:15.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.494+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f396010c370 con 0x7f3960100920 2023-08-25T08:57:15.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.495+0000 7f395cff9700 1 -- 172.21.15.99:0/3991498942 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18422 (secure 0 0 0) 0x7f3950099900 con 0x7f3960100920 2023-08-25T08:57:15.497 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:15.497 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":79,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:56:32.216885+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:57:15.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f39480a0020 msgr2=0x7f39480a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f39480a0020 0x7f39480a24e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3954009c60 tx=0x7f3954009260 comp rx=0 tx=0).stop 2023-08-25T08:57:15.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 msgr2=0x7f3960198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 0x7f3960198790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3950000c00 tx=0x7f3950005720 comp rx=0 tx=0).stop 2023-08-25T08:57:15.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 shutdown_connections 2023-08-25T08:57:15.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f39480a0020 0x7f39480a24e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39601024b0 0x7f396019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3960100920 0x7f3960198790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 --2- 172.21.15.99:0/3991498942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3960101b10 0x7f3960198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 >> 172.21.15.99:0/3991498942 conn(0x7f39600fbe50 msgr2=0x7f39601073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:15.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 shutdown_connections 2023-08-25T08:57:15.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.498+0000 7f3965fff700 1 -- 172.21.15.99:0/3991498942 wait complete. 2023-08-25T08:57:15.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T08:57:15.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T08:57:15.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3673293691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc410cee0 msgr2=0x7f9cc410f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3673293691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc410cee0 0x7f9cc410f2d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00b0a0 tx=0x7f9cbc00b3b0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3673293691 shutdown_connections 2023-08-25T08:57:15.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3673293691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc410cee0 0x7f9cc410f2d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3673293691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc4074960 0x7f9cc4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3673293691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc4073380 0x7f9cc4073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3673293691 >> 172.21.15.99:0/3673293691 conn(0x7f9cc40fbe60 msgr2=0x7f9cc40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3673293691 shutdown_connections 2023-08-25T08:57:15.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.727+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3673293691 wait complete. 2023-08-25T08:57:15.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.728+0000 7f9cca7ff700 1 Processor -- start 2023-08-25T08:57:15.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.728+0000 7f9cca7ff700 1 -- start start 2023-08-25T08:57:15.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cca7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc4073380 0x7f9cc419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:15.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cca7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc4074960 0x7f9cc419d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:15.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cca7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc410cee0 0x7f9cc41a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cca7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc41a48c0 con 0x7f9cc4073380 2023-08-25T08:57:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cca7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc41a4a00 con 0x7f9cc410cee0 2023-08-25T08:57:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cca7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc41a4b90 con 0x7f9cc4074960 2023-08-25T08:57:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc4073380 0x7f9cc419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc4073380 0x7f9cc419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54820/0 (socket says 172.21.15.99:54820) 2023-08-25T08:57:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc3fff700 1 -- 172.21.15.99:0/3373606601 learned_addr learned my addr 172.21.15.99:0/3373606601 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc8d9c700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc410cee0 0x7f9cc41a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc37fe700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc4074960 0x7f9cc419d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc3fff700 1 -- 172.21.15.99:0/3373606601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc4074960 msgr2=0x7f9cc419d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc3fff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc4074960 0x7f9cc419d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc3fff700 1 -- 172.21.15.99:0/3373606601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc410cee0 msgr2=0x7f9cc41a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc3fff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc410cee0 0x7f9cc41a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc3fff700 1 -- 172.21.15.99:0/3373606601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc009d00 con 0x7f9cc4073380 2023-08-25T08:57:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc37fe700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc4074960 0x7f9cc419d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.729+0000 7f9cc8d9c700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc410cee0 0x7f9cc41a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.730+0000 7f9cc3fff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc4073380 0x7f9cc419cd40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400d950 tx=0x7f9cb400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.730+0000 7f9cc17fa700 1 -- 172.21.15.99:0/3373606601 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4009940 con 0x7f9cc4073380 2023-08-25T08:57:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.730+0000 7f9cc17fa700 1 -- 172.21.15.99:0/3373606601 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb4010460 con 0x7f9cc4073380 2023-08-25T08:57:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.730+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc41a4e70 con 0x7f9cc4073380 2023-08-25T08:57:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.730+0000 7f9cc17fa700 1 -- 172.21.15.99:0/3373606601 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb400f620 con 0x7f9cc4073380 2023-08-25T08:57:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.730+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc41a52a0 con 0x7f9cc4073380 2023-08-25T08:57:15.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.731+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc4102a30 con 0x7f9cc4073380 2023-08-25T08:57:15.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.733+0000 7f9cc17fa700 1 -- 172.21.15.99:0/3373606601 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f9cb40108d0 con 0x7f9cc4073380 2023-08-25T08:57:15.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.734+0000 7f9cc17fa700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9cac09ffd0 0x7f9cac0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.734+0000 7f9cc17fa700 1 -- 172.21.15.99:0/3373606601 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9cb40cac10 con 0x7f9cc4073380 2023-08-25T08:57:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.734+0000 7f9cc37fe700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9cac09ffd0 0x7f9cac0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.735+0000 7f9cc37fe700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9cac09ffd0 0x7f9cac0a2490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005950 tx=0x7f9cb80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.738+0000 7f9cc17fa700 1 -- 172.21.15.99:0/3373606601 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9cb4094da0 con 0x7f9cc4073380 2023-08-25T08:57:15.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:15 smithi181 ceph-mon[131296]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:15 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:15 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/69990206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:15.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:15 smithi099 ceph-mon[97641]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:15.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:15 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:15.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:15 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/69990206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:15.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:15 smithi157 ceph-mon[130879]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:15.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:15 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:15.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:15 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/69990206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:15.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.953+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9cc40008d0 con 0x7f9cc4073380 2023-08-25T08:57:15.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.953+0000 7f9cc17fa700 1 -- 172.21.15.99:0/3373606601 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9cb40983c0 con 0x7f9cc4073380 2023-08-25T08:57:15.955 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T08:57:15.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9cac09ffd0 msgr2=0x7f9cac0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9cac09ffd0 0x7f9cac0a2490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005950 tx=0x7f9cb80058e0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc4073380 msgr2=0x7f9cc419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:15.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc4073380 0x7f9cc419cd40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400d950 tx=0x7f9cb400dc60 comp rx=0 tx=0).stop 2023-08-25T08:57:15.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 shutdown_connections 2023-08-25T08:57:15.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9cac09ffd0 0x7f9cac0a2490 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc410cee0 0x7f9cc41a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc4073380 0x7f9cc419cd40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 --2- 172.21.15.99:0/3373606601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cc4074960 0x7f9cc419d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:15.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 >> 172.21.15.99:0/3373606601 conn(0x7f9cc40fbe60 msgr2=0x7f9cc40fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:15.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 shutdown_connections 2023-08-25T08:57:15.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:15.956+0000 7f9cca7ff700 1 -- 172.21.15.99:0/3373606601 wait complete. 2023-08-25T08:57:15.976 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T08:57:16.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.185+0000 7f85f31ff700 1 -- 172.21.15.99:0/3756459351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 msgr2=0x7f85ec108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.185+0000 7f85f31ff700 1 --2- 172.21.15.99:0/3756459351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 0x7f85ec108d70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85e400b0a0 tx=0x7f85e400b3b0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.186+0000 7f85f31ff700 1 -- 172.21.15.99:0/3756459351 shutdown_connections 2023-08-25T08:57:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.186+0000 7f85f31ff700 1 --2- 172.21.15.99:0/3756459351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 0x7f85ec108d70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.186+0000 7f85f31ff700 1 --2- 172.21.15.99:0/3756459351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec100930 0x7f85ec100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.186+0000 7f85f31ff700 1 --2- 172.21.15.99:0/3756459351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec101b20 0x7f85ec101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.186+0000 7f85f31ff700 1 -- 172.21.15.99:0/3756459351 >> 172.21.15.99:0/3756459351 conn(0x7f85ec0fbe60 msgr2=0x7f85ec0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.186+0000 7f85f31ff700 1 -- 172.21.15.99:0/3756459351 shutdown_connections 2023-08-25T08:57:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.186+0000 7f85f31ff700 1 -- 172.21.15.99:0/3756459351 wait complete. 2023-08-25T08:57:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f31ff700 1 Processor -- start 2023-08-25T08:57:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f31ff700 1 -- start start 2023-08-25T08:57:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f31ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec100930 0x7f85ec1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f31ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec101b20 0x7f85ec198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f31ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 0x7f85ec19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f31ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a0460 con 0x7f85ec101b20 2023-08-25T08:57:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f31ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a05a0 con 0x7f85ec100930 2023-08-25T08:57:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f31ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a0730 con 0x7f85ec1024c0 2023-08-25T08:57:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f0f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec100930 0x7f85ec1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec101b20 0x7f85ec198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f0f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec100930 0x7f85ec1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:46926/0 (socket says 172.21.15.99:46926) 2023-08-25T08:57:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f0f9b700 1 -- 172.21.15.99:0/434820617 learned_addr learned my addr 172.21.15.99:0/434820617 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.187+0000 7f85f179c700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 0x7f85ec19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f179c700 1 -- 172.21.15.99:0/434820617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec100930 msgr2=0x7f85ec1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:16.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f179c700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec100930 0x7f85ec1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f179c700 1 -- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec101b20 msgr2=0x7f85ec198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:16.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f179c700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec101b20 0x7f85ec198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f179c700 1 -- 172.21.15.99:0/434820617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4009d00 con 0x7f85ec1024c0 2023-08-25T08:57:16.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f0f9b700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec100930 0x7f85ec1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:16.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85ebfff700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec101b20 0x7f85ec198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:16.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f179c700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 0x7f85ec19fe90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009fd0 tx=0x7f85e400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:16.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85e9ffb700 1 -- 172.21.15.99:0/434820617 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400e040 con 0x7f85ec1024c0 2023-08-25T08:57:16.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85e9ffb700 1 -- 172.21.15.99:0/434820617 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e4004e60 con 0x7f85ec1024c0 2023-08-25T08:57:16.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec1a09b0 con 0x7f85ec1024c0 2023-08-25T08:57:16.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85e9ffb700 1 -- 172.21.15.99:0/434820617 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40052a0 con 0x7f85ec1024c0 2023-08-25T08:57:16.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.188+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec1a0e80 con 0x7f85ec1024c0 2023-08-25T08:57:16.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.189+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec10c1a0 con 0x7f85ec1024c0 2023-08-25T08:57:16.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.191+0000 7f85e9ffb700 1 -- 172.21.15.99:0/434820617 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f85e401d470 con 0x7f85ec1024c0 2023-08-25T08:57:16.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.192+0000 7f85e9ffb700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f85dc0a0020 0x7f85dc0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:16.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.193+0000 7f85e9ffb700 1 -- 172.21.15.99:0/434820617 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f85e40cc500 con 0x7f85ec1024c0 2023-08-25T08:57:16.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.193+0000 7f85f0f9b700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f85dc0a0020 0x7f85dc0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:16.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.193+0000 7f85f0f9b700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f85dc0a0020 0x7f85dc0a24e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f85e0005950 tx=0x7f85e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:16.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.194+0000 7f85e9ffb700 1 -- 172.21.15.99:0/434820617 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f85e4096720 con 0x7f85ec1024c0 2023-08-25T08:57:16.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.408+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f85ec1a1070 con 0x7f85ec1024c0 2023-08-25T08:57:16.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.409+0000 7f85e9ffb700 1 -- 172.21.15.99:0/434820617 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f85e4099d40 con 0x7f85ec1024c0 2023-08-25T08:57:16.410 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:16.410 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-08-25T08:57:16.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T08:57:16.411 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-08-25T08:57:16.411 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-08-25T08:57:16.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T08:57:16.411 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-08-25T08:57:16.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f85dc0a0020 msgr2=0x7f85dc0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:16.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f85dc0a0020 0x7f85dc0a24e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f85e0005950 tx=0x7f85e0009260 comp rx=0 tx=0).stop 2023-08-25T08:57:16.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 msgr2=0x7f85ec19fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:16.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 0x7f85ec19fe90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009fd0 tx=0x7f85e400bce0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 shutdown_connections 2023-08-25T08:57:16.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85ec1024c0 0x7f85ec19fe90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f85dc0a0020 0x7f85dc0a24e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec100930 0x7f85ec1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 --2- 172.21.15.99:0/434820617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec101b20 0x7f85ec198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 >> 172.21.15.99:0/434820617 conn(0x7f85ec0fbe60 msgr2=0x7f85ec107310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:16.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 shutdown_connections 2023-08-25T08:57:16.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.413+0000 7f85f31ff700 1 -- 172.21.15.99:0/434820617 wait complete. 2023-08-25T08:57:16.437 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T08:57:16.437 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T08:57:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 -- 172.21.15.99:0/2466335802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 msgr2=0x7fd004073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/2466335802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 0x7fd004073800 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009990 tx=0x7fcff4009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 -- 172.21.15.99:0/2466335802 shutdown_connections 2023-08-25T08:57:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/2466335802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd00410cee0 0x7fd00410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/2466335802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd004074960 0x7fd004072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/2466335802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 0x7fd004073800 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 -- 172.21.15.99:0/2466335802 >> 172.21.15.99:0/2466335802 conn(0x7fd0040fbe60 msgr2=0x7fd0040fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:16.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 -- 172.21.15.99:0/2466335802 shutdown_connections 2023-08-25T08:57:16.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.652+0000 7fd00c7ff700 1 -- 172.21.15.99:0/2466335802 wait complete. 2023-08-25T08:57:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.653+0000 7fd00c7ff700 1 Processor -- start 2023-08-25T08:57:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.653+0000 7fd00c7ff700 1 -- start start 2023-08-25T08:57:16.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.653+0000 7fd00c7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 0x7fd00419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:16.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.653+0000 7fd00c7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd004074960 0x7fd00419d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:16.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.653+0000 7fd00c7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd00410cee0 0x7fd0041a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:16.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.653+0000 7fd00c7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041a4780 con 0x7fd004073380 2023-08-25T08:57:16.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.653+0000 7fd00c7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041a48f0 con 0x7fd00410cee0 2023-08-25T08:57:16.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.653+0000 7fd00c7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041a4ab0 con 0x7fd004074960 2023-08-25T08:57:16.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 0x7fd00419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:16.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd009d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd004074960 0x7fd00419d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:16.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd009d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd004074960 0x7fd00419d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45830/0 (socket says 172.21.15.99:45830) 2023-08-25T08:57:16.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd009d9a700 1 -- 172.21.15.99:0/933284380 learned_addr learned my addr 172.21.15.99:0/933284380 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:16.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00ad9c700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd00410cee0 0x7fd0041a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00a59b700 1 -- 172.21.15.99:0/933284380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd004074960 msgr2=0x7fd00419d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00a59b700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd004074960 0x7fd00419d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00a59b700 1 -- 172.21.15.99:0/933284380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd00410cee0 msgr2=0x7fd0041a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:16.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00a59b700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd00410cee0 0x7fd0041a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:16.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00a59b700 1 -- 172.21.15.99:0/933284380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd000009520 con 0x7fd004073380 2023-08-25T08:57:16.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00ad9c700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd00410cee0 0x7fd0041a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:16.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fd00a59b700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 0x7fd00419cc40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e9f0 tx=0x7fd00000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:16.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.654+0000 7fcffb7fe700 1 -- 172.21.15.99:0/933284380 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000cb70 con 0x7fd004073380 2023-08-25T08:57:16.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.655+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff40095f0 con 0x7fd004073380 2023-08-25T08:57:16.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.655+0000 7fcffb7fe700 1 -- 172.21.15.99:0/933284380 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0000044e0 con 0x7fd004073380 2023-08-25T08:57:16.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.655+0000 7fcffb7fe700 1 -- 172.21.15.99:0/933284380 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000003d00 con 0x7fd004073380 2023-08-25T08:57:16.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.655+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0041a5160 con 0x7fd004073380 2023-08-25T08:57:16.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.656+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004102a30 con 0x7fd004073380 2023-08-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.658+0000 7fcffb7fe700 1 -- 172.21.15.99:0/933284380 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fd000003ec0 con 0x7fd004073380 2023-08-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.659+0000 7fcffb7fe700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fcff009ff50 0x7fcff00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.659+0000 7fcffb7fe700 1 -- 172.21.15.99:0/933284380 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd000014070 con 0x7fd004073380 2023-08-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.659+0000 7fd009d9a700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fcff009ff50 0x7fcff00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.660+0000 7fd009d9a700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fcff009ff50 0x7fcff00a2410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcff400b2e0 tx=0x7fcff4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.663+0000 7fcffb7fe700 1 -- 172.21.15.99:0/933284380 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd000095390 con 0x7fd004073380 2023-08-25T08:57:16.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:16 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3991498942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:16 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:16 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:16 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3373606601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:57:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:16 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:16 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:16 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/434820617' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-25T08:57:16.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:16 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3991498942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:16.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:16 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:16 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:16 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3373606601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:57:16.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:16 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:16 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:16 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/434820617' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-25T08:57:16.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:16 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3991498942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:16.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:16 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:16 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:16 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3373606601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:57:16.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:16 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:16 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:16.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:16 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/434820617' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-25T08:57:16.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:16.880+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd004112700 con 0x7fd004073380 2023-08-25T08:57:17.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.700+0000 7fcffb7fe700 1 -- 172.21.15.99:0/933284380 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd004112700 con 0x7fd004073380 2023-08-25T08:57:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fcff009ff50 msgr2=0x7fcff00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fcff009ff50 0x7fcff00a2410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcff400b2e0 tx=0x7fcff4005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 msgr2=0x7fd00419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 0x7fd00419cc40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e9f0 tx=0x7fd00000ed00 comp rx=0 tx=0).stop 2023-08-25T08:57:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 shutdown_connections 2023-08-25T08:57:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fcff009ff50 0x7fcff00a2410 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd00410cee0 0x7fd0041a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004073380 0x7fd00419cc40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 --2- 172.21.15.99:0/933284380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd004074960 0x7fd00419d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 >> 172.21.15.99:0/933284380 conn(0x7fd0040fbe60 msgr2=0x7fd0040fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:17.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 shutdown_connections 2023-08-25T08:57:17.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.705+0000 7fd00c7ff700 1 -- 172.21.15.99:0/933284380 wait complete. 2023-08-25T08:57:17.708 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-08-25T08:57:17.725 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T08:57:17.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 -- 172.21.15.99:0/2831941261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec1012a0 msgr2=0x7fccec103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 --2- 172.21.15.99:0/2831941261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec1012a0 0x7fccec103690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009990 tx=0x7fccdc009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 -- 172.21.15.99:0/2831941261 shutdown_connections 2023-08-25T08:57:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 --2- 172.21.15.99:0/2831941261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec106500 0x7fccec1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 --2- 172.21.15.99:0/2831941261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec1012a0 0x7fccec103690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 --2- 172.21.15.99:0/2831941261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec103bd0 0x7fccec105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 -- 172.21.15.99:0/2831941261 >> 172.21.15.99:0/2831941261 conn(0x7fccec0fabd0 msgr2=0x7fccec0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 -- 172.21.15.99:0/2831941261 shutdown_connections 2023-08-25T08:57:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.939+0000 7fccebfff700 1 -- 172.21.15.99:0/2831941261 wait complete. 2023-08-25T08:57:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fccebfff700 1 Processor -- start 2023-08-25T08:57:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fccebfff700 1 -- start start 2023-08-25T08:57:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fccebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec1012a0 0x7fccec198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fccebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec103bd0 0x7fccec198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:17.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fccebfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec106500 0x7fccec19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:17.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fccebfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a00e0 con 0x7fccec1012a0 2023-08-25T08:57:17.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fccebfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a0220 con 0x7fccec106500 2023-08-25T08:57:17.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fccebfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a03b0 con 0x7fccec103bd0 2023-08-25T08:57:17.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fcce3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec103bd0 0x7fccec198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:17.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.940+0000 7fcceaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec1012a0 0x7fccec198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec103bd0 0x7fccec198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:60168/0 (socket says 172.21.15.99:60168) 2023-08-25T08:57:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce3fff700 1 -- 172.21.15.99:0/1334089549 learned_addr learned my addr 172.21.15.99:0/1334089549 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcceb7fe700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec106500 0x7fccec19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce3fff700 1 -- 172.21.15.99:0/1334089549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec106500 msgr2=0x7fccec19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce3fff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec106500 0x7fccec19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce3fff700 1 -- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec1012a0 msgr2=0x7fccec198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce3fff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec1012a0 0x7fccec198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce3fff700 1 -- 172.21.15.99:0/1334089549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc0095f0 con 0x7fccec103bd0 2023-08-25T08:57:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcceb7fe700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec106500 0x7fccec19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce3fff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec103bd0 0x7fccec198aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e9a0 tx=0x7fccd400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce8ff9700 1 -- 172.21.15.99:0/1334089549 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4009790 con 0x7fccec103bd0 2023-08-25T08:57:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce8ff9700 1 -- 172.21.15.99:0/1334089549 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd400cbd0 con 0x7fccec103bd0 2023-08-25T08:57:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec1a0690 con 0x7fccec103bd0 2023-08-25T08:57:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.941+0000 7fcce8ff9700 1 -- 172.21.15.99:0/1334089549 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4018680 con 0x7fccec103bd0 2023-08-25T08:57:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.942+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec1a0ba0 con 0x7fccec103bd0 2023-08-25T08:57:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.942+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec108080 con 0x7fccec103bd0 2023-08-25T08:57:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.946+0000 7fcce8ff9700 1 -- 172.21.15.99:0/1334089549 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fccd400cd40 con 0x7fccec103bd0 2023-08-25T08:57:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.947+0000 7fcce8ff9700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fccd809fff0 0x7fccd80a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.947+0000 7fcce8ff9700 1 -- 172.21.15.99:0/1334089549 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fccd4014070 con 0x7fccec103bd0 2023-08-25T08:57:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.947+0000 7fcceaffd700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fccd809fff0 0x7fccd80a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.948+0000 7fcceaffd700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fccd809fff0 0x7fccd80a24b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00b2e0 tx=0x7fccdc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:17.951 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:17.951+0000 7fcce8ff9700 1 -- 172.21.15.99:0/1334089549 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fccd4051e00 con 0x7fccec103bd0 2023-08-25T08:57:17.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:17 smithi181 ceph-mon[131296]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:17 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:17 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:17 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:17 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:57:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:17 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:17.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:17 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:57:17.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:17 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/933284380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T08:57:18.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:17 smithi099 ceph-mon[97641]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:18.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:18.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:18.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:18.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:57:18.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:18.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:17 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:57:18.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:17 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/933284380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T08:57:18.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:17 smithi157 ceph-mon[130879]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:18.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:17 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:18.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:17 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:18.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:17 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:18.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:17 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:57:18.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:17 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:18.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:17 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:57:18.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:17 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/933284380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T08:57:18.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.181+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fccec10bfc0 con 0x7fccec103bd0 2023-08-25T08:57:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.716+0000 7fcce8ff9700 1 -- 172.21.15.99:0/1334089549 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7fccd4095270 con 0x7fccec103bd0 2023-08-25T08:57:18.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fccd809fff0 msgr2=0x7fccd80a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:18.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fccd809fff0 0x7fccd80a24b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00b2e0 tx=0x7fccdc005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec103bd0 msgr2=0x7fccec198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:18.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec103bd0 0x7fccec198aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e9a0 tx=0x7fccd400ecb0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 shutdown_connections 2023-08-25T08:57:18.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fccd809fff0 0x7fccd80a24b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec106500 0x7fccec19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec1012a0 0x7fccec198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 --2- 172.21.15.99:0/1334089549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec103bd0 0x7fccec198aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 >> 172.21.15.99:0/1334089549 conn(0x7fccec0fabd0 msgr2=0x7fccec0ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:18.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 shutdown_connections 2023-08-25T08:57:18.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.724+0000 7fccebfff700 1 -- 172.21.15.99:0/1334089549 wait complete. 2023-08-25T08:57:18.727 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-08-25T08:57:18.744 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T08:57:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 -- 172.21.15.99:0/3161679265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc074960 msgr2=0x7f4edc072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/3161679265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc074960 0x7f4edc072db0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009990 tx=0x7f4ecc009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 -- 172.21.15.99:0/3161679265 shutdown_connections 2023-08-25T08:57:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/3161679265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc10cee0 0x7f4edc10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/3161679265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc074960 0x7f4edc072db0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/3161679265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc073380 0x7f4edc073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 -- 172.21.15.99:0/3161679265 >> 172.21.15.99:0/3161679265 conn(0x7f4edc0fbe60 msgr2=0x7f4edc0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 -- 172.21.15.99:0/3161679265 shutdown_connections 2023-08-25T08:57:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.968+0000 7f4ee0bff700 1 -- 172.21.15.99:0/3161679265 wait complete. 2023-08-25T08:57:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.969+0000 7f4ee0bff700 1 Processor -- start 2023-08-25T08:57:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.969+0000 7f4ee0bff700 1 -- start start 2023-08-25T08:57:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4ee0bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc073380 0x7f4edc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4ee0bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc074960 0x7f4edc19d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4ee0bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc10cee0 0x7f4edc1a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4ee0bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1a47f0 con 0x7f4edc10cee0 2023-08-25T08:57:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4ee0bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1a4960 con 0x7f4edc073380 2023-08-25T08:57:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4ee0bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1a4b20 con 0x7f4edc074960 2023-08-25T08:57:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4eda59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc073380 0x7f4edc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4ed9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc074960 0x7f4edc19d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4eda59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc073380 0x7f4edc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48822/0 (socket says 172.21.15.99:48822) 2023-08-25T08:57:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4eda59c700 1 -- 172.21.15.99:0/373767195 learned_addr learned my addr 172.21.15.99:0/373767195 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4edad9d700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc10cee0 0x7f4edc1a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4eda59c700 1 -- 172.21.15.99:0/373767195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc074960 msgr2=0x7f4edc19d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4eda59c700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc074960 0x7f4edc19d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4eda59c700 1 -- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc10cee0 msgr2=0x7f4edc1a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4eda59c700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc10cee0 0x7f4edc1a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.970+0000 7f4eda59c700 1 -- 172.21.15.99:0/373767195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0095f0 con 0x7f4edc073380 2023-08-25T08:57:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.971+0000 7f4eda59c700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc073380 0x7f4edc19cc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b680 tx=0x7f4ecc0057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.971+0000 7f4ecb7fe700 1 -- 172.21.15.99:0/373767195 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc01d070 con 0x7f4edc073380 2023-08-25T08:57:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.971+0000 7f4ecb7fe700 1 -- 172.21.15.99:0/373767195 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ecc022470 con 0x7f4edc073380 2023-08-25T08:57:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.971+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc1a4da0 con 0x7f4edc073380 2023-08-25T08:57:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.971+0000 7f4ecb7fe700 1 -- 172.21.15.99:0/373767195 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc017650 con 0x7f4edc073380 2023-08-25T08:57:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.971+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1a51c0 con 0x7f4edc073380 2023-08-25T08:57:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.972+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc102a30 con 0x7f4edc073380 2023-08-25T08:57:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.974+0000 7f4ecb7fe700 1 -- 172.21.15.99:0/373767195 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f4ecc0225e0 con 0x7f4edc073380 2023-08-25T08:57:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.975+0000 7f4ecb7fe700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4ec409ff50 0x7f4ec40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.975+0000 7f4ecb7fe700 1 -- 172.21.15.99:0/373767195 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f4ecc0cc600 con 0x7f4edc073380 2023-08-25T08:57:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.975+0000 7f4ed9d9b700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4ec409ff50 0x7f4ec40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.976+0000 7f4ed9d9b700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4ec409ff50 0x7f4ec40a2410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4edc19e180 tx=0x7f4ed0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:18.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:18.979+0000 7f4ecb7fe700 1 -- 172.21.15.99:0/373767195 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4ecc096650 con 0x7f4edc073380 2023-08-25T08:57:18.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:18 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/933284380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-25T08:57:18.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:18 smithi181 ceph-mon[131296]: osdmap e80: 12 total, 12 up, 12 in 2023-08-25T08:57:19.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:18 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/933284380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-25T08:57:19.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:18 smithi099 ceph-mon[97641]: osdmap e80: 12 total, 12 up, 12 in 2023-08-25T08:57:19.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:18 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/933284380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-25T08:57:19.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:18 smithi157 ceph-mon[130879]: osdmap e80: 12 total, 12 up, 12 in 2023-08-25T08:57:19.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:19.241+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4edc112950 con 0x7f4edc073380 2023-08-25T08:57:19.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1334089549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T08:57:19.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T08:57:19.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: pgmap v262: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:19.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-25T08:57:19.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: osdmap e81: 12 total, 12 up, 12 in 2023-08-25T08:57:19.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T08:57:19.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/373767195' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T08:57:19.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T08:57:19.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-25T08:57:19.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: osdmap e82: 12 total, 12 up, 12 in 2023-08-25T08:57:19.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:19.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:19.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:19.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:19 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.026+0000 7f4ecb7fe700 1 -- 172.21.15.99:0/373767195 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4ecc0270c0 con 0x7f4edc073380 2023-08-25T08:57:20.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4ec409ff50 msgr2=0x7f4ec40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4ec409ff50 0x7f4ec40a2410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4edc19e180 tx=0x7f4ed0009260 comp rx=0 tx=0).stop 2023-08-25T08:57:20.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc073380 msgr2=0x7f4edc19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc073380 0x7f4edc19cc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b680 tx=0x7f4ecc0057b0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 shutdown_connections 2023-08-25T08:57:20.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc10cee0 0x7f4edc1a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4ec409ff50 0x7f4ec40a2410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc073380 0x7f4edc19cc80 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b680 tx=0x7f4ecc0057b0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 --2- 172.21.15.99:0/373767195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc074960 0x7f4edc19d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 >> 172.21.15.99:0/373767195 conn(0x7f4edc0fbe60 msgr2=0x7f4edc0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:20.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 shutdown_connections 2023-08-25T08:57:20.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.031+0000 7f4ee0bff700 1 -- 172.21.15.99:0/373767195 wait complete. 2023-08-25T08:57:20.037 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 2 and data pool 3 2023-08-25T08:57:20.058 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1334089549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T08:57:20.058 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T08:57:20.058 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: pgmap v262: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:20.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-25T08:57:20.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: osdmap e81: 12 total, 12 up, 12 in 2023-08-25T08:57:20.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T08:57:20.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/373767195' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T08:57:20.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T08:57:20.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-25T08:57:20.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: osdmap e82: 12 total, 12 up, 12 in 2023-08-25T08:57:20.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.060 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.060 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.060 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:19 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.061 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:57:20.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1334089549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T08:57:20.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T08:57:20.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: pgmap v262: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:57:20.129 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-25T08:57:20.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: osdmap e81: 12 total, 12 up, 12 in 2023-08-25T08:57:20.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T08:57:20.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/373767195' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T08:57:20.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T08:57:20.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-25T08:57:20.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: osdmap e82: 12 total, 12 up, 12 in 2023-08-25T08:57:20.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.130 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.131 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:19 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 -- 172.21.15.99:0/534690166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 msgr2=0x7fc878101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 --2- 172.21.15.99:0/534690166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 0x7fc878101f80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099c0 tx=0x7fc868009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 -- 172.21.15.99:0/534690166 shutdown_connections 2023-08-25T08:57:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 --2- 172.21.15.99:0/534690166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8781024c0 0x7fc878108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 --2- 172.21.15.99:0/534690166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc878100930 0x7fc878100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 --2- 172.21.15.99:0/534690166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 0x7fc878101f80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 -- 172.21.15.99:0/534690166 >> 172.21.15.99:0/534690166 conn(0x7fc8780fbe60 msgr2=0x7fc8780fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 -- 172.21.15.99:0/534690166 shutdown_connections 2023-08-25T08:57:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.272+0000 7fc8803ff700 1 -- 172.21.15.99:0/534690166 wait complete. 2023-08-25T08:57:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.273+0000 7fc8803ff700 1 Processor -- start 2023-08-25T08:57:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.273+0000 7fc8803ff700 1 -- start start 2023-08-25T08:57:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc8803ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc878100930 0x7fc878198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc8803ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 0x7fc878198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc8803ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8781024c0 0x7fc87819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc8803ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8781a03a0 con 0x7fc878101b20 2023-08-25T08:57:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc8803ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8781a04e0 con 0x7fc878100930 2023-08-25T08:57:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc8803ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8781a0670 con 0x7fc8781024c0 2023-08-25T08:57:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87d99a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 0x7fc878198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87e19b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc878100930 0x7fc878198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87d99a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 0x7fc878198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33174/0 (socket says 172.21.15.99:33174) 2023-08-25T08:57:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87d99a700 1 -- 172.21.15.99:0/3005060805 learned_addr learned my addr 172.21.15.99:0/3005060805 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87e99c700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8781024c0 0x7fc87819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87e19b700 1 -- 172.21.15.99:0/3005060805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8781024c0 msgr2=0x7fc87819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87e19b700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8781024c0 0x7fc87819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87e19b700 1 -- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 msgr2=0x7fc878198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87e19b700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 0x7fc878198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87e19b700 1 -- 172.21.15.99:0/3005060805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc870009c40 con 0x7fc878100930 2023-08-25T08:57:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.274+0000 7fc87d99a700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 0x7fc878198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.275+0000 7fc87e19b700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc878100930 0x7fc878198820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc87400e9c0 tx=0x7fc87400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.275+0000 7fc86f7fe700 1 -- 172.21.15.99:0/3005060805 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874004220 con 0x7fc878100930 2023-08-25T08:57:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.275+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8680095f0 con 0x7fc878100930 2023-08-25T08:57:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.275+0000 7fc86f7fe700 1 -- 172.21.15.99:0/3005060805 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc874004380 con 0x7fc878100930 2023-08-25T08:57:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.275+0000 7fc86f7fe700 1 -- 172.21.15.99:0/3005060805 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874003bb0 con 0x7fc878100930 2023-08-25T08:57:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.275+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8781a0d20 con 0x7fc878100930 2023-08-25T08:57:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.276+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc878192f80 con 0x7fc878100930 2023-08-25T08:57:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.278+0000 7fc86f7fe700 1 -- 172.21.15.99:0/3005060805 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fc874003d10 con 0x7fc878100930 2023-08-25T08:57:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.279+0000 7fc86f7fe700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc8640a0020 0x7fc8640a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.279+0000 7fc86f7fe700 1 -- 172.21.15.99:0/3005060805 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc874014070 con 0x7fc878100930 2023-08-25T08:57:20.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.280+0000 7fc87d99a700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc8640a0020 0x7fc8640a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:20.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.280+0000 7fc87d99a700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc8640a0020 0x7fc8640a24e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc868004800 tx=0x7fc86801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:20.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.283+0000 7fc86f7fe700 1 -- 172.21.15.99:0/3005060805 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc8740951c0 con 0x7fc878100930 2023-08-25T08:57:20.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.496+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc87810c190 con 0x7fc878100930 2023-08-25T08:57:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.497+0000 7fc86f7fe700 1 -- 172.21.15.99:0/3005060805 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22132 (secure 0 0 0) 0x7fc8740987e0 con 0x7fc878100930 2023-08-25T08:57:20.499 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:20.499 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":82,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:19.244312+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:57:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc8640a0020 msgr2=0x7fc8640a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc8640a0020 0x7fc8640a24e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc868004800 tx=0x7fc86801a040 comp rx=0 tx=0).stop 2023-08-25T08:57:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc878100930 msgr2=0x7fc878198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc878100930 0x7fc878198820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc87400e9c0 tx=0x7fc87400ecd0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 shutdown_connections 2023-08-25T08:57:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8781024c0 0x7fc87819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc8640a0020 0x7fc8640a24e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc878100930 0x7fc878198820 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 --2- 172.21.15.99:0/3005060805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc878101b20 0x7fc878198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 >> 172.21.15.99:0/3005060805 conn(0x7fc8780fbe60 msgr2=0x7fc878107410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:20.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 shutdown_connections 2023-08-25T08:57:20.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.500+0000 7fc8803ff700 1 -- 172.21.15.99:0/3005060805 wait complete. 2023-08-25T08:57:20.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:57:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.720+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3892236062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8101b20 msgr2=0x7fa0b8101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.720+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3892236062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8101b20 0x7fa0b8101f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80099c0 tx=0x7fa0a8009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.721+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3892236062 shutdown_connections 2023-08-25T08:57:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.721+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3892236062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b81024c0 0x7fa0b8108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.721+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3892236062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b8100930 0x7fa0b8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.721+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3892236062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8101b20 0x7fa0b8101f80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.721+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3892236062 >> 172.21.15.99:0/3892236062 conn(0x7fa0b80fbe60 msgr2=0x7fa0b80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.721+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3892236062 shutdown_connections 2023-08-25T08:57:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.721+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3892236062 wait complete. 2023-08-25T08:57:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0c05ff700 1 Processor -- start 2023-08-25T08:57:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0c05ff700 1 -- start start 2023-08-25T08:57:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0c05ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8100930 0x7fa0b81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0c05ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b8101b20 0x7fa0b8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0c05ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b81024c0 0x7fa0b819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0c05ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81a0360 con 0x7fa0b8100930 2023-08-25T08:57:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0c05ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81a04a0 con 0x7fa0b8101b20 2023-08-25T08:57:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0c05ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81a0630 con 0x7fa0b81024c0 2023-08-25T08:57:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0be39b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8100930 0x7fa0b81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0be39b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8100930 0x7fa0b81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33200/0 (socket says 172.21.15.99:33200) 2023-08-25T08:57:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0be39b700 1 -- 172.21.15.99:0/3429428679 learned_addr learned my addr 172.21.15.99:0/3429428679 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.722+0000 7fa0bdb9a700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b8101b20 0x7fa0b8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0beb9c700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b81024c0 0x7fa0b819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0be39b700 1 -- 172.21.15.99:0/3429428679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b81024c0 msgr2=0x7fa0b819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0be39b700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b81024c0 0x7fa0b819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0be39b700 1 -- 172.21.15.99:0/3429428679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b8101b20 msgr2=0x7fa0b8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0be39b700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b8101b20 0x7fa0b8198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0be39b700 1 -- 172.21.15.99:0/3429428679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a80095f0 con 0x7fa0b8100930 2023-08-25T08:57:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0be39b700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8100930 0x7fa0b81987e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400e970 tx=0x7fa0b400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0af7fe700 1 -- 172.21.15.99:0/3429428679 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4009790 con 0x7fa0b8100930 2023-08-25T08:57:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0af7fe700 1 -- 172.21.15.99:0/3429428679 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b400c7e0 con 0x7fa0b8100930 2023-08-25T08:57:20.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b81a0910 con 0x7fa0b8100930 2023-08-25T08:57:20.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0af7fe700 1 -- 172.21.15.99:0/3429428679 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4003c70 con 0x7fa0b8100930 2023-08-25T08:57:20.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.723+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b81a0e70 con 0x7fa0b8100930 2023-08-25T08:57:20.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.724+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b810c190 con 0x7fa0b8100930 2023-08-25T08:57:20.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.726+0000 7fa0af7fe700 1 -- 172.21.15.99:0/3429428679 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fa0b400cc50 con 0x7fa0b8100930 2023-08-25T08:57:20.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.728+0000 7fa0af7fe700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa0a40a0020 0x7fa0a40a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:20.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.728+0000 7fa0af7fe700 1 -- 172.21.15.99:0/3429428679 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0b4014070 con 0x7fa0b8100930 2023-08-25T08:57:20.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.728+0000 7fa0bdb9a700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa0a40a0020 0x7fa0a40a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:20.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.729+0000 7fa0bdb9a700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa0a40a0020 0x7fa0a40a24e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b580 tx=0x7fa0a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:20.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.730+0000 7fa0af7fe700 1 -- 172.21.15.99:0/3429428679 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa0b4095280 con 0x7fa0b8100930 2023-08-25T08:57:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.933+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0b81a10c0 con 0x7fa0b8100930 2023-08-25T08:57:20.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.935+0000 7fa0af7fe700 1 -- 172.21.15.99:0/3429428679 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22119 (secure 0 0 0) 0x7fa0b40988a0 con 0x7fa0b8100930 2023-08-25T08:57:20.936 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:20.936 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:20.500911+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:57:20.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa0a40a0020 msgr2=0x7fa0a40a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa0a40a0020 0x7fa0a40a24e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b580 tx=0x7fa0a8005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8100930 msgr2=0x7fa0b81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:20.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8100930 0x7fa0b81987e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400e970 tx=0x7fa0b400ec80 comp rx=0 tx=0).stop 2023-08-25T08:57:20.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 shutdown_connections 2023-08-25T08:57:20.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b81024c0 0x7fa0b819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0b8100930 0x7fa0b81987e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa0a40a0020 0x7fa0a40a24e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 --2- 172.21.15.99:0/3429428679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b8101b20 0x7fa0b8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:20.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 >> 172.21.15.99:0/3429428679 conn(0x7fa0b80fbe60 msgr2=0x7fa0b80fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:20.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 shutdown_connections 2023-08-25T08:57:20.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:20.937+0000 7fa0c05ff700 1 -- 172.21.15.99:0/3429428679 wait complete. 2023-08-25T08:57:20.958 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T08:57:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 -- 172.21.15.99:0/1970508287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754100930 msgr2=0x7fb754100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 --2- 172.21.15.99:0/1970508287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754100930 0x7fb754100d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7440099c0 tx=0x7fb744009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 -- 172.21.15.99:0/1970508287 shutdown_connections 2023-08-25T08:57:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 --2- 172.21.15.99:0/1970508287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7541024c0 0x7fb754108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 --2- 172.21.15.99:0/1970508287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754100930 0x7fb754100d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 --2- 172.21.15.99:0/1970508287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754101b20 0x7fb754101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 -- 172.21.15.99:0/1970508287 >> 172.21.15.99:0/1970508287 conn(0x7fb7540fbea0 msgr2=0x7fb7540fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 -- 172.21.15.99:0/1970508287 shutdown_connections 2023-08-25T08:57:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.159+0000 7fb759bff700 1 -- 172.21.15.99:0/1970508287 wait complete. 2023-08-25T08:57:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.160+0000 7fb759bff700 1 Processor -- start 2023-08-25T08:57:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.160+0000 7fb759bff700 1 -- start start 2023-08-25T08:57:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb759bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754100930 0x7fb7540746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb759bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754101b20 0x7fb754072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb759bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7541024c0 0x7fb754073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb759bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540739f0 con 0x7fb754101b20 2023-08-25T08:57:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb759bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb754073b60 con 0x7fb7541024c0 2023-08-25T08:57:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb759bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7541a0720 con 0x7fb754100930 2023-08-25T08:57:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754101b20 0x7fb754072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754101b20 0x7fb754072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33212/0 (socket says 172.21.15.99:33212) 2023-08-25T08:57:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 -- 172.21.15.99:0/1941399441 learned_addr learned my addr 172.21.15.99:0/1941399441 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb7537fe700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754100930 0x7fb7540746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb753fff700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7541024c0 0x7fb754073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 -- 172.21.15.99:0/1941399441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754100930 msgr2=0x7fb7540746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754100930 0x7fb7540746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 -- 172.21.15.99:0/1941399441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7541024c0 msgr2=0x7fb754073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7541024c0 0x7fb754073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 -- 172.21.15.99:0/1941399441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440095f0 con 0x7fb754101b20 2023-08-25T08:57:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb7537fe700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754100930 0x7fb7540746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb753fff700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7541024c0 0x7fb754073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:21.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.161+0000 7fb752ffd700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754101b20 0x7fb754072d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e9a0 tx=0x7fb74800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:21.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.162+0000 7fb750ff9700 1 -- 172.21.15.99:0/1941399441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748009790 con 0x7fb754101b20 2023-08-25T08:57:21.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.162+0000 7fb750ff9700 1 -- 172.21.15.99:0/1941399441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb74800cbd0 con 0x7fb754101b20 2023-08-25T08:57:21.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.162+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7541a0a00 con 0x7fb754101b20 2023-08-25T08:57:21.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.162+0000 7fb750ff9700 1 -- 172.21.15.99:0/1941399441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748018680 con 0x7fb754101b20 2023-08-25T08:57:21.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.162+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7541a0f60 con 0x7fb754101b20 2023-08-25T08:57:21.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.163+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb754107f10 con 0x7fb754101b20 2023-08-25T08:57:21.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.166+0000 7fb750ff9700 1 -- 172.21.15.99:0/1941399441 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fb74800cd40 con 0x7fb754101b20 2023-08-25T08:57:21.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.168+0000 7fb750ff9700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb73c09ffd0 0x7fb73c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:21.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.168+0000 7fb750ff9700 1 -- 172.21.15.99:0/1941399441 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb748014070 con 0x7fb754101b20 2023-08-25T08:57:21.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.168+0000 7fb7537fe700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb73c09ffd0 0x7fb73c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:21.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.169+0000 7fb7537fe700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb73c09ffd0 0x7fb73c0a2490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb744009990 tx=0x7fb74401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:21.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.170+0000 7fb750ff9700 1 -- 172.21.15.99:0/1941399441 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb748095770 con 0x7fb754101b20 2023-08-25T08:57:21.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-25T08:57:21.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-25T08:57:21.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-25T08:57:21.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: pgmap v265: 97 pgs: 26 active+clean, 29 creating+peering, 42 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-08-25T08:57:21.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:21.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:57:21.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3005060805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:21.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: osdmap e83: 12 total, 12 up, 12 in 2023-08-25T08:57:21.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:21 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3429428679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:21.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-25T08:57:21.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-25T08:57:21.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-25T08:57:21.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: pgmap v265: 97 pgs: 26 active+clean, 29 creating+peering, 42 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-08-25T08:57:21.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:21.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:57:21.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3005060805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:21.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: osdmap e83: 12 total, 12 up, 12 in 2023-08-25T08:57:21.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:21 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3429428679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:21.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:21.416+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb75402cde0 con 0x7fb754101b20 2023-08-25T08:57:21.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-25T08:57:21.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-25T08:57:21.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-25T08:57:21.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: pgmap v265: 97 pgs: 26 active+clean, 29 creating+peering, 42 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-08-25T08:57:21.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:57:21.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:57:21.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:57:21.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3005060805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:21.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: osdmap e83: 12 total, 12 up, 12 in 2023-08-25T08:57:21.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:21 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3429428679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:22.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.042+0000 7fb750ff9700 1 -- 172.21.15.99:0/1941399441 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb7480260b0 con 0x7fb754101b20 2023-08-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb73c09ffd0 msgr2=0x7fb73c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb73c09ffd0 0x7fb73c0a2490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb744009990 tx=0x7fb74401a040 comp rx=0 tx=0).stop 2023-08-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754101b20 msgr2=0x7fb754072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754101b20 0x7fb754072d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e9a0 tx=0x7fb74800ed60 comp rx=0 tx=0).stop 2023-08-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 shutdown_connections 2023-08-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb73c09ffd0 0x7fb73c0a2490 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7541024c0 0x7fb754073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754100930 0x7fb7540746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 --2- 172.21.15.99:0/1941399441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb754101b20 0x7fb754072d10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 >> 172.21.15.99:0/1941399441 conn(0x7fb7540fbea0 msgr2=0x7fb754105050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 shutdown_connections 2023-08-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.045+0000 7fb759bff700 1 -- 172.21.15.99:0/1941399441 wait complete. 2023-08-25T08:57:22.065 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-08-25T08:57:22.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-08-25T08:57:22.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 -- 172.21.15.99:0/1897987996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0568101b00 msgr2=0x7f0568101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:22.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 --2- 172.21.15.99:0/1897987996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0568101b00 0x7f0568101f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f055c0099c0 tx=0x7f055c009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 -- 172.21.15.99:0/1897987996 shutdown_connections 2023-08-25T08:57:22.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 --2- 172.21.15.99:0/1897987996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05681024a0 0x7f0568108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 --2- 172.21.15.99:0/1897987996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0568100910 0x7f0568100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 --2- 172.21.15.99:0/1897987996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0568101b00 0x7f0568101f60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 -- 172.21.15.99:0/1897987996 >> 172.21.15.99:0/1897987996 conn(0x7f05680fbe60 msgr2=0x7f05680fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:22.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 -- 172.21.15.99:0/1897987996 shutdown_connections 2023-08-25T08:57:22.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.267+0000 7f056e3ff700 1 -- 172.21.15.99:0/1897987996 wait complete. 2023-08-25T08:57:22.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.268+0000 7f056e3ff700 1 Processor -- start 2023-08-25T08:57:22.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.268+0000 7f056e3ff700 1 -- start start 2023-08-25T08:57:22.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056e3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0568100910 0x7f0568198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:22.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056e3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0568101b00 0x7f0568198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:22.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056e3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05681024a0 0x7f056819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:22.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056e3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681a03e0 con 0x7f05681024a0 2023-08-25T08:57:22.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056e3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681a0520 con 0x7f0568100910 2023-08-25T08:57:22.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056e3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681a06b0 con 0x7f0568101b00 2023-08-25T08:57:22.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056c99c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05681024a0 0x7f056819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:22.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f0567fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0568100910 0x7f0568198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:22.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056c99c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05681024a0 0x7f056819fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33234/0 (socket says 172.21.15.99:33234) 2023-08-25T08:57:22.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056c99c700 1 -- 172.21.15.99:0/4140835722 learned_addr learned my addr 172.21.15.99:0/4140835722 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:22.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f05677fe700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0568101b00 0x7f0568198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:22.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056c99c700 1 -- 172.21.15.99:0/4140835722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0568101b00 msgr2=0x7f0568198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:22.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056c99c700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0568101b00 0x7f0568198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056c99c700 1 -- 172.21.15.99:0/4140835722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0568100910 msgr2=0x7f0568198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:22.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056c99c700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0568100910 0x7f0568198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:22.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.269+0000 7f056c99c700 1 -- 172.21.15.99:0/4140835722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0095f0 con 0x7f05681024a0 2023-08-25T08:57:22.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.270+0000 7f056c99c700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05681024a0 0x7f056819fe10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0560007ee0 tx=0x7f056000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:22.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.270+0000 7f05657fa700 1 -- 172.21.15.99:0/4140835722 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560017070 con 0x7f05681024a0 2023-08-25T08:57:22.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.270+0000 7f05657fa700 1 -- 172.21.15.99:0/4140835722 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056000f040 con 0x7f05681024a0 2023-08-25T08:57:22.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.270+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05681a0990 con 0x7f05681024a0 2023-08-25T08:57:22.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.270+0000 7f05657fa700 1 -- 172.21.15.99:0/4140835722 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560005580 con 0x7f05681024a0 2023-08-25T08:57:22.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.270+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05681a0e50 con 0x7f05681024a0 2023-08-25T08:57:22.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.271+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056810c170 con 0x7f05681024a0 2023-08-25T08:57:22.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.273+0000 7f05657fa700 1 -- 172.21.15.99:0/4140835722 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f056000db20 con 0x7f05681024a0 2023-08-25T08:57:22.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.274+0000 7f05657fa700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f055009ff50 0x7f05500a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:22.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.274+0000 7f05657fa700 1 -- 172.21.15.99:0/4140835722 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f05600ca890 con 0x7f05681024a0 2023-08-25T08:57:22.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.274+0000 7f0567fff700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f055009ff50 0x7f05500a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:22.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.275+0000 7f0567fff700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f055009ff50 0x7f05500a2410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0568101960 tx=0x7f0558007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:22.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.278+0000 7f05657fa700 1 -- 172.21.15.99:0/4140835722 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f05600946d0 con 0x7f05681024a0 2023-08-25T08:57:22.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:22 smithi099 ceph-mon[97641]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-08-25T08:57:22.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:22 smithi099 ceph-mon[97641]: Cluster is now healthy 2023-08-25T08:57:22.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:22 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] up:active 2023-08-25T08:57:22.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:22 smithi099 ceph-mon[97641]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:22.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:22 smithi099 ceph-mon[97641]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:22.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:22 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1941399441' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-25T08:57:22.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:22 smithi157 ceph-mon[130879]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-08-25T08:57:22.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:22 smithi157 ceph-mon[130879]: Cluster is now healthy 2023-08-25T08:57:22.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:22 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] up:active 2023-08-25T08:57:22.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:22 smithi157 ceph-mon[130879]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:22.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:22 smithi157 ceph-mon[130879]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:22.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:22 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1941399441' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-25T08:57:22.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:22 smithi181 ceph-mon[131296]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-08-25T08:57:22.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:22 smithi181 ceph-mon[131296]: Cluster is now healthy 2023-08-25T08:57:22.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:22 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] up:active 2023-08-25T08:57:22.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:22 smithi181 ceph-mon[131296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:22.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:22 smithi181 ceph-mon[131296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:22.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:22 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1941399441' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-25T08:57:22.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:22.527+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f05681a1160 con 0x7f05681024a0 2023-08-25T08:57:23.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.048+0000 7f05657fa700 1 -- 172.21.15.99:0/4140835722 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0560097cf0 con 0x7f05681024a0 2023-08-25T08:57:23.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.051+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f055009ff50 msgr2=0x7f05500a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:23.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.051+0000 7f056e3ff700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f055009ff50 0x7f05500a2410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0568101960 tx=0x7f0558007400 comp rx=0 tx=0).stop 2023-08-25T08:57:23.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.051+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05681024a0 msgr2=0x7f056819fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:23.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.051+0000 7f056e3ff700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05681024a0 0x7f056819fe10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0560007ee0 tx=0x7f056000d3b0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.051+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 shutdown_connections 2023-08-25T08:57:23.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.052+0000 7f056e3ff700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05681024a0 0x7f056819fe10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.052+0000 7f056e3ff700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f055009ff50 0x7f05500a2410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.052+0000 7f056e3ff700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0568100910 0x7f0568198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.052+0000 7f056e3ff700 1 --2- 172.21.15.99:0/4140835722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0568101b00 0x7f0568198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.052+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 >> 172.21.15.99:0/4140835722 conn(0x7f05680fbe60 msgr2=0x7f05680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:23.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.052+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 shutdown_connections 2023-08-25T08:57:23.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.052+0000 7f056e3ff700 1 -- 172.21.15.99:0/4140835722 wait complete. 2023-08-25T08:57:23.075 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-08-25T08:57:23.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 -- 172.21.15.99:0/2631483736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 msgr2=0x7f9d7c0ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:23.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 --2- 172.21.15.99:0/2631483736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 0x7f9d7c0ffc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d780099c0 tx=0x7f9d78009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 -- 172.21.15.99:0/2631483736 shutdown_connections 2023-08-25T08:57:23.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 --2- 172.21.15.99:0/2631483736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c1013a0 0x7f9d7c107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 --2- 172.21.15.99:0/2631483736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 0x7f9d7c0ffc30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 --2- 172.21.15.99:0/2631483736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7c100a00 0x7f9d7c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 -- 172.21.15.99:0/2631483736 >> 172.21.15.99:0/2631483736 conn(0x7f9d7c0fad80 msgr2=0x7f9d7c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:23.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 -- 172.21.15.99:0/2631483736 shutdown_connections 2023-08-25T08:57:23.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.276+0000 7f9d837ff700 1 -- 172.21.15.99:0/2631483736 wait complete. 2023-08-25T08:57:23.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.277+0000 7f9d837ff700 1 Processor -- start 2023-08-25T08:57:23.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.277+0000 7f9d837ff700 1 -- start start 2023-08-25T08:57:23.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d837ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 0x7f9d7c1037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:23.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d837ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c100a00 0x7f9d7c101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:23.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d837ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7c1013a0 0x7f9d7c102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:23.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d837ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c102ac0 con 0x7f9d7c0ff810 2023-08-25T08:57:23.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d837ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c102c30 con 0x7f9d7c1013a0 2023-08-25T08:57:23.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d837ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c1a0680 con 0x7f9d7c100a00 2023-08-25T08:57:23.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d8159b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 0x7f9d7c1037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:23.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d8159b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 0x7f9d7c1037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33244/0 (socket says 172.21.15.99:33244) 2023-08-25T08:57:23.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d8159b700 1 -- 172.21.15.99:0/1460415830 learned_addr learned my addr 172.21.15.99:0/1460415830 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:23.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d80d9a700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c100a00 0x7f9d7c101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:23.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d81d9c700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7c1013a0 0x7f9d7c102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:23.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d8159b700 1 -- 172.21.15.99:0/1460415830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c100a00 msgr2=0x7f9d7c101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:23.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d8159b700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c100a00 0x7f9d7c101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d8159b700 1 -- 172.21.15.99:0/1460415830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7c1013a0 msgr2=0x7f9d7c102320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:23.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d8159b700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7c1013a0 0x7f9d7c102320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:23.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.278+0000 7f9d8159b700 1 -- 172.21.15.99:0/1460415830 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d780095f0 con 0x7f9d7c0ff810 2023-08-25T08:57:23.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.279+0000 7f9d8159b700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 0x7f9d7c1037a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009fd0 tx=0x7f9d78009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:23.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.279+0000 7f9d727fc700 1 -- 172.21.15.99:0/1460415830 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7801d070 con 0x7f9d7c0ff810 2023-08-25T08:57:23.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.279+0000 7f9d727fc700 1 -- 172.21.15.99:0/1460415830 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d78005030 con 0x7f9d7c0ff810 2023-08-25T08:57:23.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.279+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c1a0900 con 0x7f9d7c0ff810 2023-08-25T08:57:23.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.279+0000 7f9d727fc700 1 -- 172.21.15.99:0/1460415830 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78004a90 con 0x7f9d7c0ff810 2023-08-25T08:57:23.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.279+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c1a0d20 con 0x7f9d7c0ff810 2023-08-25T08:57:23.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.280+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c10b020 con 0x7f9d7c0ff810 2023-08-25T08:57:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.282+0000 7f9d727fc700 1 -- 172.21.15.99:0/1460415830 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f9d780051a0 con 0x7f9d7c0ff810 2023-08-25T08:57:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.283+0000 7f9d727fc700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9d6809ff50 0x7f9d680a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.284+0000 7f9d727fc700 1 -- 172.21.15.99:0/1460415830 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9d780cbea0 con 0x7f9d7c0ff810 2023-08-25T08:57:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.284+0000 7f9d80d9a700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9d6809ff50 0x7f9d680a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.284+0000 7f9d80d9a700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9d6809ff50 0x7f9d680a2410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009c60 tx=0x7f9d6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:23.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.287+0000 7f9d727fc700 1 -- 172.21.15.99:0/1460415830 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9d78095d70 con 0x7f9d7c0ff810 2023-08-25T08:57:23.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:23 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1941399441' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-25T08:57:23.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:23 smithi099 ceph-mon[97641]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:23.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:23 smithi099 ceph-mon[97641]: pgmap v267: 97 pgs: 53 active+clean, 29 creating+peering, 15 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 864 B/s wr, 2 op/s 2023-08-25T08:57:23.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:23 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4140835722' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-08-25T08:57:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:23 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1941399441' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-25T08:57:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:23 smithi157 ceph-mon[130879]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:23 smithi157 ceph-mon[130879]: pgmap v267: 97 pgs: 53 active+clean, 29 creating+peering, 15 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 864 B/s wr, 2 op/s 2023-08-25T08:57:23.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:23 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4140835722' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-08-25T08:57:23.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:23 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1941399441' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-25T08:57:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:23 smithi181 ceph-mon[131296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:23 smithi181 ceph-mon[131296]: pgmap v267: 97 pgs: 53 active+clean, 29 creating+peering, 15 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 864 B/s wr, 2 op/s 2023-08-25T08:57:23.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:23 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4140835722' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-08-25T08:57:23.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:23.534+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f9d7c10b200 con 0x7f9d7c0ff810 2023-08-25T08:57:24.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.024+0000 7f9d727fc700 1 -- 172.21.15.99:0/1460415830 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9d78099390 con 0x7f9d7c0ff810 2023-08-25T08:57:24.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9d6809ff50 msgr2=0x7f9d680a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9d6809ff50 0x7f9d680a2410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009c60 tx=0x7f9d6c009260 comp rx=0 tx=0).stop 2023-08-25T08:57:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 msgr2=0x7f9d7c1037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 0x7f9d7c1037a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009fd0 tx=0x7f9d78009580 comp rx=0 tx=0).stop 2023-08-25T08:57:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 shutdown_connections 2023-08-25T08:57:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9d6809ff50 0x7f9d680a2410 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7c1013a0 0x7f9d7c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d7c0ff810 0x7f9d7c1037a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 --2- 172.21.15.99:0/1460415830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c100a00 0x7f9d7c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 >> 172.21.15.99:0/1460415830 conn(0x7f9d7c0fad80 msgr2=0x7f9d7c067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 shutdown_connections 2023-08-25T08:57:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.027+0000 7f9d837ff700 1 -- 172.21.15.99:0/1460415830 wait complete. 2023-08-25T08:57:24.048 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 08:57:23 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-b[155907]: 2023-08-25T08:57:23.744+0000 7ff089b6b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T08:57:24.052 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-08-25T08:57:24.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 -- 172.21.15.99:0/3435533417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41024c0 msgr2=0x7f57a4108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:24.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 --2- 172.21.15.99:0/3435533417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41024c0 0x7f57a4108d70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f579c00b0d0 tx=0x7f579c00b3e0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 -- 172.21.15.99:0/3435533417 shutdown_connections 2023-08-25T08:57:24.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 --2- 172.21.15.99:0/3435533417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41024c0 0x7f57a4108d70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 --2- 172.21.15.99:0/3435533417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a4100930 0x7f57a4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 --2- 172.21.15.99:0/3435533417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57a4101b20 0x7f57a4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 -- 172.21.15.99:0/3435533417 >> 172.21.15.99:0/3435533417 conn(0x7f57a40fbea0 msgr2=0x7f57a40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:24.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 -- 172.21.15.99:0/3435533417 shutdown_connections 2023-08-25T08:57:24.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.255+0000 7f57abfff700 1 -- 172.21.15.99:0/3435533417 wait complete. 2023-08-25T08:57:24.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.256+0000 7f57abfff700 1 Processor -- start 2023-08-25T08:57:24.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.256+0000 7f57abfff700 1 -- start start 2023-08-25T08:57:24.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.256+0000 7f57abfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a4100930 0x7f57a41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:24.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.256+0000 7f57abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57a4101b20 0x7f57a4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:24.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.256+0000 7f57abfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a41024c0 0x7f57a419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:24.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.256+0000 7f57abfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a0450 con 0x7f57a4100930 2023-08-25T08:57:24.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.256+0000 7f57abfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a0590 con 0x7f57a41024c0 2023-08-25T08:57:24.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.256+0000 7f57abfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a0720 con 0x7f57a4101b20 2023-08-25T08:57:24.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a4100930 0x7f57a41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:24.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a4100930 0x7f57a41988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33264/0 (socket says 172.21.15.99:33264) 2023-08-25T08:57:24.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 -- 172.21.15.99:0/1614564855 learned_addr learned my addr 172.21.15.99:0/1614564855 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:24.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a959a700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57a4101b20 0x7f57a4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:24.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57aa59c700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a41024c0 0x7f57a419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:24.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 -- 172.21.15.99:0/1614564855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57a4101b20 msgr2=0x7f57a4198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:24.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57a4101b20 0x7f57a4198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 -- 172.21.15.99:0/1614564855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a41024c0 msgr2=0x7f57a419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:24.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a41024c0 0x7f57a419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:24.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 -- 172.21.15.99:0/1614564855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c009d00 con 0x7f57a4100930 2023-08-25T08:57:24.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a959a700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57a4101b20 0x7f57a4198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:24.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57a9d9b700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a4100930 0x7f57a41988d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57a000e970 tx=0x7f57a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:24.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f579affd700 1 -- 172.21.15.99:0/1614564855 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0009790 con 0x7f57a4100930 2023-08-25T08:57:24.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a41a0a00 con 0x7f57a4100930 2023-08-25T08:57:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f579affd700 1 -- 172.21.15.99:0/1614564855 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a00044e0 con 0x7f57a4100930 2023-08-25T08:57:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.257+0000 7f579affd700 1 -- 172.21.15.99:0/1614564855 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0003e30 con 0x7f57a4100930 2023-08-25T08:57:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.258+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a41a0f10 con 0x7f57a4100930 2023-08-25T08:57:24.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.258+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a410c150 con 0x7f57a4100930 2023-08-25T08:57:24.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.261+0000 7f579affd700 1 -- 172.21.15.99:0/1614564855 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f57a0004000 con 0x7f57a4100930 2023-08-25T08:57:24.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.262+0000 7f579affd700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f579009ff50 0x7f57900a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.262+0000 7f579affd700 1 -- 172.21.15.99:0/1614564855 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57a0014070 con 0x7f57a4100930 2023-08-25T08:57:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.262+0000 7f57a959a700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f579009ff50 0x7f57900a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.263+0000 7f57a959a700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f579009ff50 0x7f57900a2410 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5794009cb0 tx=0x7f5794009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:24.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.264+0000 7f579affd700 1 -- 172.21.15.99:0/1614564855 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f57a0095380 con 0x7f57a4100930 2023-08-25T08:57:24.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-25T08:57:24.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4140835722' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-08-25T08:57:24.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-25T08:57:24.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:24.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-08-25T08:57:24.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-08-25T08:57:24.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1460415830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T08:57:24.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] up:active 2023-08-25T08:57:24.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1460415830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-25T08:57:24.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-25T08:57:24.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-25T08:57:24.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: Cluster is now healthy 2023-08-25T08:57:24.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-08-25T08:57:24.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-08-25T08:57:24.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:24 smithi099 ceph-mon[97641]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-08-25T08:57:24.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-25T08:57:24.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4140835722' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-08-25T08:57:24.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-25T08:57:24.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:24.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-08-25T08:57:24.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-08-25T08:57:24.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1460415830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T08:57:24.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] up:active 2023-08-25T08:57:24.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1460415830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-25T08:57:24.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-25T08:57:24.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-25T08:57:24.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: Cluster is now healthy 2023-08-25T08:57:24.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-08-25T08:57:24.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-08-25T08:57:24.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:24 smithi157 ceph-mon[130879]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-08-25T08:57:24.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-25T08:57:24.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4140835722' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-08-25T08:57:24.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-25T08:57:24.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-25T08:57:24.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-08-25T08:57:24.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-08-25T08:57:24.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1460415830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T08:57:24.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] up:active 2023-08-25T08:57:24.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1460415830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-25T08:57:24.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-25T08:57:24.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-25T08:57:24.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: Cluster is now healthy 2023-08-25T08:57:24.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-08-25T08:57:24.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-08-25T08:57:24.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:24 smithi181 ceph-mon[131296]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-08-25T08:57:24.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:24.512+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f57a410c380 con 0x7f57a4100930 2023-08-25T08:57:25.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.030+0000 7f579affd700 1 -- 172.21.15.99:0/1614564855 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f57a00989a0 con 0x7f57a4100930 2023-08-25T08:57:25.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.033+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f579009ff50 msgr2=0x7f57900a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.033+0000 7f57abfff700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f579009ff50 0x7f57900a2410 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5794009cb0 tx=0x7f5794009260 comp rx=0 tx=0).stop 2023-08-25T08:57:25.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.033+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a4100930 msgr2=0x7f57a41988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.033+0000 7f57abfff700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a4100930 0x7f57a41988d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57a000e970 tx=0x7f57a000ed30 comp rx=0 tx=0).stop 2023-08-25T08:57:25.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.033+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 shutdown_connections 2023-08-25T08:57:25.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.033+0000 7f57abfff700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f579009ff50 0x7f57900a2410 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.033+0000 7f57abfff700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a41024c0 0x7f57a419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.033+0000 7f57abfff700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a4100930 0x7f57a41988d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.034+0000 7f57abfff700 1 --2- 172.21.15.99:0/1614564855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57a4101b20 0x7f57a4198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.034+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 >> 172.21.15.99:0/1614564855 conn(0x7f57a40fbea0 msgr2=0x7f57a40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:25.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.034+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 shutdown_connections 2023-08-25T08:57:25.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.034+0000 7f57abfff700 1 -- 172.21.15.99:0/1614564855 wait complete. 2023-08-25T08:57:25.053 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-08-25T08:57:25.054 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-08-25T08:57:25.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 -- 172.21.15.99:0/3568666837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94100930 msgr2=0x7f3b94100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 --2- 172.21.15.99:0/3568666837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94100930 0x7f3b94100d50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3b840099c0 tx=0x7f3b84009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 -- 172.21.15.99:0/3568666837 shutdown_connections 2023-08-25T08:57:25.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 --2- 172.21.15.99:0/3568666837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b941024c0 0x7f3b94108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 --2- 172.21.15.99:0/3568666837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94100930 0x7f3b94100d50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 --2- 172.21.15.99:0/3568666837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94101b20 0x7f3b94101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 -- 172.21.15.99:0/3568666837 >> 172.21.15.99:0/3568666837 conn(0x7f3b940fbea0 msgr2=0x7f3b940fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:25.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 -- 172.21.15.99:0/3568666837 shutdown_connections 2023-08-25T08:57:25.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.256+0000 7f3b995ff700 1 -- 172.21.15.99:0/3568666837 wait complete. 2023-08-25T08:57:25.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.257+0000 7f3b995ff700 1 Processor -- start 2023-08-25T08:57:25.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b995ff700 1 -- start start 2023-08-25T08:57:25.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b995ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94100930 0x7f3b940746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:25.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b995ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94101b20 0x7f3b94072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:25.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b995ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b941024c0 0x7f3b94073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:25.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b995ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b940739f0 con 0x7f3b94101b20 2023-08-25T08:57:25.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b995ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b94073b60 con 0x7f3b94100930 2023-08-25T08:57:25.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b995ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941a06b0 con 0x7f3b941024c0 2023-08-25T08:57:25.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b927fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94101b20 0x7f3b94072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:25.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b927fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94101b20 0x7f3b94072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33272/0 (socket says 172.21.15.99:33272) 2023-08-25T08:57:25.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b927fc700 1 -- 172.21.15.99:0/4104483697 learned_addr learned my addr 172.21.15.99:0/4104483697 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:25.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b937fe700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b941024c0 0x7f3b94073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:25.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b92ffd700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94100930 0x7f3b940746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:25.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b927fc700 1 -- 172.21.15.99:0/4104483697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b941024c0 msgr2=0x7f3b94073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b927fc700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b941024c0 0x7f3b94073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.258+0000 7f3b927fc700 1 -- 172.21.15.99:0/4104483697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94100930 msgr2=0x7f3b940746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.259+0000 7f3b927fc700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94100930 0x7f3b940746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.259+0000 7f3b927fc700 1 -- 172.21.15.99:0/4104483697 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840095f0 con 0x7f3b94101b20 2023-08-25T08:57:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.259+0000 7f3b927fc700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94101b20 0x7f3b94072d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800e9a0 tx=0x7f3b8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.259+0000 7f3b7bfff700 1 -- 172.21.15.99:0/4104483697 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88009790 con 0x7f3b94101b20 2023-08-25T08:57:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.259+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b941a0990 con 0x7f3b94101b20 2023-08-25T08:57:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.259+0000 7f3b7bfff700 1 -- 172.21.15.99:0/4104483697 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b88010430 con 0x7f3b94101b20 2023-08-25T08:57:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.259+0000 7f3b7bfff700 1 -- 172.21.15.99:0/4104483697 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88018680 con 0x7f3b94101b20 2023-08-25T08:57:25.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.259+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b941a0ef0 con 0x7f3b94101b20 2023-08-25T08:57:25.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.260+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b941078b0 con 0x7f3b94101b20 2023-08-25T08:57:25.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.262+0000 7f3b7bfff700 1 -- 172.21.15.99:0/4104483697 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f3b880105a0 con 0x7f3b94101b20 2023-08-25T08:57:25.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.263+0000 7f3b7bfff700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3b7c09ff50 0x7f3b7c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:25.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.263+0000 7f3b7bfff700 1 -- 172.21.15.99:0/4104483697 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3b88014070 con 0x7f3b94101b20 2023-08-25T08:57:25.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.264+0000 7f3b92ffd700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3b7c09ff50 0x7f3b7c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:25.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.264+0000 7f3b92ffd700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3b7c09ff50 0x7f3b7c0a2410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009990 tx=0x7f3b84005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:25.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.265+0000 7f3b7bfff700 1 -- 172.21.15.99:0/4104483697 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3b88095380 con 0x7f3b94101b20 2023-08-25T08:57:25.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-mon[97641]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.8 KiB/s wr, 11 op/s 2023-08-25T08:57:25.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1614564855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-25T08:57:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] up:active 2023-08-25T08:57:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1614564855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-25T08:57:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T08:57:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T08:57:25.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:25 smithi157 ceph-mon[130879]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.8 KiB/s wr, 11 op/s 2023-08-25T08:57:25.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:25 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1614564855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-25T08:57:25.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:25 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] up:active 2023-08-25T08:57:25.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:25 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1614564855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-25T08:57:25.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:25 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T08:57:25.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:25 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T08:57:25.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:25 smithi181 ceph-mon[131296]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.8 KiB/s wr, 11 op/s 2023-08-25T08:57:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:25 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1614564855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-25T08:57:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:25 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] up:active 2023-08-25T08:57:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:25 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1614564855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-25T08:57:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:25 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T08:57:25.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:25 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T08:57:25.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.498+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b9402cdf0 con 0x7f3b7c09ff50 2023-08-25T08:57:25.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.544+0000 7f3b7bfff700 1 -- 172.21.15.99:0/4104483697 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3b9402cdf0 con 0x7f3b7c09ff50 2023-08-25T08:57:25.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.546+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3b7c09ff50 msgr2=0x7f3b7c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.546+0000 7f3b995ff700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3b7c09ff50 0x7f3b7c0a2410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009990 tx=0x7f3b84005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.546+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94101b20 msgr2=0x7f3b94072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.546+0000 7f3b995ff700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94101b20 0x7f3b94072d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800e9a0 tx=0x7f3b8800ecb0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.547+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 shutdown_connections 2023-08-25T08:57:25.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.547+0000 7f3b995ff700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b941024c0 0x7f3b94073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.547+0000 7f3b995ff700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3b7c09ff50 0x7f3b7c0a2410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.547+0000 7f3b995ff700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94100930 0x7f3b940746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.547+0000 7f3b995ff700 1 --2- 172.21.15.99:0/4104483697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b94101b20 0x7f3b94072d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.547+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 >> 172.21.15.99:0/4104483697 conn(0x7f3b940fbea0 msgr2=0x7f3b940fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:25.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.547+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 shutdown_connections 2023-08-25T08:57:25.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.547+0000 7f3b995ff700 1 -- 172.21.15.99:0/4104483697 wait complete. 2023-08-25T08:57:25.565 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-08-25T08:57:25.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 -- 172.21.15.99:0/1532799901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c106650 msgr2=0x7f256c108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 --2- 172.21.15.99:0/1532799901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c106650 0x7f256c108a60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2564009f90 tx=0x7f256400b330 comp rx=0 tx=0).stop 2023-08-25T08:57:25.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 -- 172.21.15.99:0/1532799901 shutdown_connections 2023-08-25T08:57:25.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 --2- 172.21.15.99:0/1532799901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c106650 0x7f256c108a60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 --2- 172.21.15.99:0/1532799901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c1013f0 0x7f256c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 --2- 172.21.15.99:0/1532799901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c103d20 0x7f256c106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 -- 172.21.15.99:0/1532799901 >> 172.21.15.99:0/1532799901 conn(0x7f256c0fad80 msgr2=0x7f256c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:25.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 -- 172.21.15.99:0/1532799901 shutdown_connections 2023-08-25T08:57:25.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.768+0000 7f2571bff700 1 -- 172.21.15.99:0/1532799901 wait complete. 2023-08-25T08:57:25.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.769+0000 7f2571bff700 1 Processor -- start 2023-08-25T08:57:25.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.769+0000 7f2571bff700 1 -- start start 2023-08-25T08:57:25.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f2571bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c1013f0 0x7f256c100d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:25.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f2571bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c103d20 0x7f256c0ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:25.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f2571bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c106650 0x7f256c0ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:25.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f2571bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c100080 con 0x7f256c1013f0 2023-08-25T08:57:25.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f2571bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1001f0 con 0x7f256c103d20 2023-08-25T08:57:25.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f2571bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1a0770 con 0x7f256c106650 2023-08-25T08:57:25.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c1013f0 0x7f256c100d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:25.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c1013f0 0x7f256c100d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33306/0 (socket says 172.21.15.99:33306) 2023-08-25T08:57:25.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 -- 172.21.15.99:0/1597645139 learned_addr learned my addr 172.21.15.99:0/1597645139 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:25.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256bfff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c106650 0x7f256c0ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:25.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f2563fff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c103d20 0x7f256c0ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:25.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 -- 172.21.15.99:0/1597645139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c106650 msgr2=0x7f256c0ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c106650 0x7f256c0ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 -- 172.21.15.99:0/1597645139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c103d20 msgr2=0x7f256c0ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:25.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c103d20 0x7f256c0ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:25.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 -- 172.21.15.99:0/1597645139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564009c40 con 0x7f256c1013f0 2023-08-25T08:57:25.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.770+0000 7f256b7fe700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c1013f0 0x7f256c100d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f255c00e9f0 tx=0x7f255c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:25.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.771+0000 7f25697fa700 1 -- 172.21.15.99:0/1597645139 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c004a30 con 0x7f256c1013f0 2023-08-25T08:57:25.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.771+0000 7f25697fa700 1 -- 172.21.15.99:0/1597645139 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255c00cc80 con 0x7f256c1013f0 2023-08-25T08:57:25.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.771+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c1a0a50 con 0x7f256c1013f0 2023-08-25T08:57:25.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.771+0000 7f25697fa700 1 -- 172.21.15.99:0/1597645139 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c005530 con 0x7f256c1013f0 2023-08-25T08:57:25.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.771+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c1a0f60 con 0x7f256c1013f0 2023-08-25T08:57:25.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.772+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c0fc900 con 0x7f256c1013f0 2023-08-25T08:57:25.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.774+0000 7f25697fa700 1 -- 172.21.15.99:0/1597645139 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f255c004b90 con 0x7f256c1013f0 2023-08-25T08:57:25.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.776+0000 7f25697fa700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f255809ffd0 0x7f25580a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:25.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.776+0000 7f25697fa700 1 -- 172.21.15.99:0/1597645139 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f255c014070 con 0x7f256c1013f0 2023-08-25T08:57:25.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.776+0000 7f2563fff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f255809ffd0 0x7f25580a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:25.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.777+0000 7f2563fff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f255809ffd0 0x7f25580a2490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2554009c60 tx=0x7f2554009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:25.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:25.780+0000 7f25697fa700 1 -- 172.21.15.99:0/1597645139 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f255c0950c0 con 0x7f256c1013f0 2023-08-25T08:57:25.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:57:25.504+0000 7fd9c7572700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:57:25.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:57:25.504+0000 7fd9c7572700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:57:25.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:57:25.504+0000 7fd9c7572700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:57:25.813 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:57:25.504+0000 7fd9c7572700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:57:25.813 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:57:25 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:57:25.504+0000 7fd9c7572700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:57:26.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.014+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f256c02ce30 con 0x7f255809ffd0 2023-08-25T08:57:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.019+0000 7f25697fa700 1 -- 172.21.15.99:0/1597645139 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f256c02ce30 con 0x7f255809ffd0 2023-08-25T08:57:26.019 INFO:teuthology.orchestra.run.smithi099.stdout:/volumes/_nogroup/sv_0/d9b313d6-9e02-41ad-95b3-4fba9e202a74 2023-08-25T08:57:26.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.021+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f255809ffd0 msgr2=0x7f25580a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.021+0000 7f2571bff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f255809ffd0 0x7f25580a2490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2554009c60 tx=0x7f2554009260 comp rx=0 tx=0).stop 2023-08-25T08:57:26.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.021+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c1013f0 msgr2=0x7f256c100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.021+0000 7f2571bff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c1013f0 0x7f256c100d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f255c00e9f0 tx=0x7f255c00ed00 comp rx=0 tx=0).stop 2023-08-25T08:57:26.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.022+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 shutdown_connections 2023-08-25T08:57:26.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.022+0000 7f2571bff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c106650 0x7f256c0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.022+0000 7f2571bff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f256c1013f0 0x7f256c100d60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.022+0000 7f2571bff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f255809ffd0 0x7f25580a2490 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.022+0000 7f2571bff700 1 --2- 172.21.15.99:0/1597645139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c103d20 0x7f256c0ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.022+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 >> 172.21.15.99:0/1597645139 conn(0x7f256c0fad80 msgr2=0x7f256c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:26.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.022+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 shutdown_connections 2023-08-25T08:57:26.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.022+0000 7f2571bff700 1 -- 172.21.15.99:0/1597645139 wait complete. 2023-08-25T08:57:26.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-08-25T08:57:26.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 -- 172.21.15.99:0/2272103861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5900100930 msgr2=0x7f5900100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 --2- 172.21.15.99:0/2272103861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5900100930 0x7f5900100d50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0099c0 tx=0x7f58fc009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 -- 172.21.15.99:0/2272103861 shutdown_connections 2023-08-25T08:57:26.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 --2- 172.21.15.99:0/2272103861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59001024c0 0x7f5900108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 --2- 172.21.15.99:0/2272103861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5900100930 0x7f5900100d50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 --2- 172.21.15.99:0/2272103861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5900101b20 0x7f5900101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 -- 172.21.15.99:0/2272103861 >> 172.21.15.99:0/2272103861 conn(0x7f59000fbea0 msgr2=0x7f59000fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 -- 172.21.15.99:0/2272103861 shutdown_connections 2023-08-25T08:57:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.244+0000 7f59073ff700 1 -- 172.21.15.99:0/2272103861 wait complete. 2023-08-25T08:57:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.245+0000 7f59073ff700 1 Processor -- start 2023-08-25T08:57:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.245+0000 7f59073ff700 1 -- start start 2023-08-25T08:57:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.245+0000 7f59073ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5900100930 0x7f59001966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.245+0000 7f59073ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5900101b20 0x7f5900196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.245+0000 7f59073ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59001024c0 0x7f590019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.245+0000 7f59073ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590019e250 con 0x7f59001024c0 2023-08-25T08:57:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.245+0000 7f59073ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590019e390 con 0x7f5900100930 2023-08-25T08:57:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.245+0000 7f59073ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590019e520 con 0x7f5900101b20 2023-08-25T08:57:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59001024c0 0x7f590019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59001024c0 0x7f590019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33320/0 (socket says 172.21.15.99:33320) 2023-08-25T08:57:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 -- 172.21.15.99:0/3569647684 learned_addr learned my addr 172.21.15.99:0/3569647684 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590499a700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5900101b20 0x7f5900196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590519b700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5900100930 0x7f59001966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 -- 172.21.15.99:0/3569647684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5900101b20 msgr2=0x7f5900196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5900101b20 0x7f5900196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 -- 172.21.15.99:0/3569647684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5900100930 msgr2=0x7f59001966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5900100930 0x7f59001966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 -- 172.21.15.99:0/3569647684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc0095f0 con 0x7f59001024c0 2023-08-25T08:57:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.246+0000 7f590599c700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59001024c0 0x7f590019dc80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58f800ef30 tx=0x7f58f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.247+0000 7f58f67fc700 1 -- 172.21.15.99:0/3569647684 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8018070 con 0x7f59001024c0 2023-08-25T08:57:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.247+0000 7f58f67fc700 1 -- 172.21.15.99:0/3569647684 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f800f040 con 0x7f59001024c0 2023-08-25T08:57:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.247+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590019e7a0 con 0x7f59001024c0 2023-08-25T08:57:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.247+0000 7f58f67fc700 1 -- 172.21.15.99:0/3569647684 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800cd00 con 0x7f59001024c0 2023-08-25T08:57:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.247+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590019ecb0 con 0x7f59001024c0 2023-08-25T08:57:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.248+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5900107ec0 con 0x7f59001024c0 2023-08-25T08:57:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.250+0000 7f58f67fc700 1 -- 172.21.15.99:0/3569647684 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f58f8003a00 con 0x7f59001024c0 2023-08-25T08:57:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.251+0000 7f58f67fc700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f58ec09ffd0 0x7f58ec0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.251+0000 7f58f67fc700 1 -- 172.21.15.99:0/3569647684 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58f80cac10 con 0x7f59001024c0 2023-08-25T08:57:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.251+0000 7f590519b700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f58ec09ffd0 0x7f58ec0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.252+0000 7f590519b700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f58ec09ffd0 0x7f58ec0a2490 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009990 tx=0x7f58fc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:26.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.255+0000 7f58f67fc700 1 -- 172.21.15.99:0/3569647684 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f58f8094ae0 con 0x7f59001024c0 2023-08-25T08:57:26.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:26 smithi099 ceph-mon[97641]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:26 smithi157 ceph-mon[130879]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.489+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f590002ce50 con 0x7f58ec09ffd0 2023-08-25T08:57:26.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:26 smithi181 ceph-mon[131296]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.515+0000 7f58f67fc700 1 -- 172.21.15.99:0/3569647684 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f590002ce50 con 0x7f58ec09ffd0 2023-08-25T08:57:26.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f58ec09ffd0 msgr2=0x7f58ec0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f58ec09ffd0 0x7f58ec0a2490 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009990 tx=0x7f58fc005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59001024c0 msgr2=0x7f590019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59001024c0 0x7f590019dc80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58f800ef30 tx=0x7f58f800c3a0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 shutdown_connections 2023-08-25T08:57:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59001024c0 0x7f590019dc80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f58ec09ffd0 0x7f58ec0a2490 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5900100930 0x7f59001966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 --2- 172.21.15.99:0/3569647684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5900101b20 0x7f5900196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 >> 172.21.15.99:0/3569647684 conn(0x7f59000fbea0 msgr2=0x7f5900105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 shutdown_connections 2023-08-25T08:57:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.517+0000 7f59073ff700 1 -- 172.21.15.99:0/3569647684 wait complete. 2023-08-25T08:57:26.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-08-25T08:57:26.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 -- 172.21.15.99:0/1161564673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b4100930 msgr2=0x7fa1b4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/1161564673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b4100930 0x7fa1b4100d50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00099c0 tx=0x7fa1b0009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 -- 172.21.15.99:0/1161564673 shutdown_connections 2023-08-25T08:57:26.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/1161564673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b41024c0 0x7fa1b4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/1161564673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b4100930 0x7fa1b4100d50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/1161564673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b4101b20 0x7fa1b4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 -- 172.21.15.99:0/1161564673 >> 172.21.15.99:0/1161564673 conn(0x7fa1b40fbea0 msgr2=0x7fa1b40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:26.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 -- 172.21.15.99:0/1161564673 shutdown_connections 2023-08-25T08:57:26.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.736+0000 7fa1bbdff700 1 -- 172.21.15.99:0/1161564673 wait complete. 2023-08-25T08:57:26.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.737+0000 7fa1bbdff700 1 Processor -- start 2023-08-25T08:57:26.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.737+0000 7fa1bbdff700 1 -- start start 2023-08-25T08:57:26.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1bbdff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b4100930 0x7fa1b4196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:26.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1bbdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4101b20 0x7fa1b4196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:26.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1bbdff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b41024c0 0x7fa1b419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:26.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1bbdff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b419e1f0 con 0x7fa1b41024c0 2023-08-25T08:57:26.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1bbdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b419e330 con 0x7fa1b4101b20 2023-08-25T08:57:26.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1bbdff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b419e4c0 con 0x7fa1b4100930 2023-08-25T08:57:26.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b939a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4101b20 0x7fa1b4196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:26.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1ba39c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b41024c0 0x7fa1b419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:26.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b939a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4101b20 0x7fa1b4196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:49014/0 (socket says 172.21.15.99:49014) 2023-08-25T08:57:26.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b939a700 1 -- 172.21.15.99:0/3181116435 learned_addr learned my addr 172.21.15.99:0/3181116435 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:26.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b9b9b700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b4100930 0x7fa1b4196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:26.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b939a700 1 -- 172.21.15.99:0/3181116435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b4100930 msgr2=0x7fa1b4196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b939a700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b4100930 0x7fa1b4196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b939a700 1 -- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b41024c0 msgr2=0x7fa1b419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b939a700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b41024c0 0x7fa1b419dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b939a700 1 -- 172.21.15.99:0/3181116435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac009c40 con 0x7fa1b4101b20 2023-08-25T08:57:26.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1b9b9b700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b4100930 0x7fa1b4196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:26.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.738+0000 7fa1ba39c700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b41024c0 0x7fa1b419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:26.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.739+0000 7fa1b939a700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4101b20 0x7fa1b4196bb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400e9a0 tx=0x7fa1a400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:26.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.739+0000 7fa1aaffd700 1 -- 172.21.15.99:0/3181116435 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4009790 con 0x7fa1b4101b20 2023-08-25T08:57:26.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.739+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b00095f0 con 0x7fa1b4101b20 2023-08-25T08:57:26.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.739+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b419eb70 con 0x7fa1b4101b20 2023-08-25T08:57:26.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.739+0000 7fa1aaffd700 1 -- 172.21.15.99:0/3181116435 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a400ce80 con 0x7fa1b4101b20 2023-08-25T08:57:26.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.739+0000 7fa1aaffd700 1 -- 172.21.15.99:0/3181116435 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4018420 con 0x7fa1b4101b20 2023-08-25T08:57:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.740+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b41078b0 con 0x7fa1b4101b20 2023-08-25T08:57:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.742+0000 7fa1aaffd700 1 -- 172.21.15.99:0/3181116435 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fa1a40106a0 con 0x7fa1b4101b20 2023-08-25T08:57:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.744+0000 7fa1aaffd700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa1a009ffd0 0x7fa1a00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.744+0000 7fa1aaffd700 1 -- 172.21.15.99:0/3181116435 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa1a4014070 con 0x7fa1b4101b20 2023-08-25T08:57:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.744+0000 7fa1b9b9b700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa1a009ffd0 0x7fa1a00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.744+0000 7fa1b9b9b700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa1a009ffd0 0x7fa1a00a2490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009990 tx=0x7fa1b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.746+0000 7fa1aaffd700 1 -- 172.21.15.99:0/3181116435 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa1a4095350 con 0x7fa1b4101b20 2023-08-25T08:57:26.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.978+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1b402cce0 con 0x7fa1a009ffd0 2023-08-25T08:57:26.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.982+0000 7fa1aaffd700 1 -- 172.21.15.99:0/3181116435 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fa1b402cce0 con 0x7fa1a009ffd0 2023-08-25T08:57:26.981 INFO:teuthology.orchestra.run.smithi099.stdout:/volumes/_nogroup/sv_1/a9ce6aa6-bf74-4eec-a0a8-e527a306088d 2023-08-25T08:57:26.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa1a009ffd0 msgr2=0x7fa1a00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa1a009ffd0 0x7fa1a00a2490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009990 tx=0x7fa1b0005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4101b20 msgr2=0x7fa1b4196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4101b20 0x7fa1b4196bb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400e9a0 tx=0x7fa1a400ecb0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 shutdown_connections 2023-08-25T08:57:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b41024c0 0x7fa1b419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b4100930 0x7fa1b4196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa1a009ffd0 0x7fa1a00a2490 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 --2- 172.21.15.99:0/3181116435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4101b20 0x7fa1b4196bb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:26.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 >> 172.21.15.99:0/3181116435 conn(0x7fa1b40fbea0 msgr2=0x7fa1b40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:26.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 shutdown_connections 2023-08-25T08:57:26.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:26.984+0000 7fa1bbdff700 1 -- 172.21.15.99:0/3181116435 wait complete. 2023-08-25T08:57:27.002 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:57:27.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 -- 172.21.15.99:0/319707127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d74100930 msgr2=0x7f0d74100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 --2- 172.21.15.99:0/319707127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d74100930 0x7f0d74100d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d640099c0 tx=0x7f0d64009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 -- 172.21.15.99:0/319707127 shutdown_connections 2023-08-25T08:57:27.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 --2- 172.21.15.99:0/319707127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d741024c0 0x7f0d74108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 --2- 172.21.15.99:0/319707127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d74100930 0x7f0d74100d50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 --2- 172.21.15.99:0/319707127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74101b20 0x7f0d74101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 -- 172.21.15.99:0/319707127 >> 172.21.15.99:0/319707127 conn(0x7f0d740fbe60 msgr2=0x7f0d740fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:27.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 -- 172.21.15.99:0/319707127 shutdown_connections 2023-08-25T08:57:27.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.206+0000 7f0d79dff700 1 -- 172.21.15.99:0/319707127 wait complete. 2023-08-25T08:57:27.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.207+0000 7f0d79dff700 1 Processor -- start 2023-08-25T08:57:27.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.207+0000 7f0d79dff700 1 -- start start 2023-08-25T08:57:27.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d79dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74100930 0x7f0d740746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:27.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d79dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d74101b20 0x7f0d74072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:27.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d79dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d741024c0 0x7f0d74073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d79dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740739f0 con 0x7f0d74101b20 2023-08-25T08:57:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d79dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d74073b60 con 0x7f0d741024c0 2023-08-25T08:57:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d79dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d741a0650 con 0x7f0d74100930 2023-08-25T08:57:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d73fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d741024c0 0x7f0d74073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d73fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d741024c0 0x7f0d74073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:45518/0 (socket says 172.21.15.99:45518) 2023-08-25T08:57:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d74101b20 0x7f0d74072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d73fff700 1 -- 172.21.15.99:0/271461051 learned_addr learned my addr 172.21.15.99:0/271461051 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:27.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.208+0000 7f0d737fe700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74100930 0x7f0d740746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:27.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d73fff700 1 -- 172.21.15.99:0/271461051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74100930 msgr2=0x7f0d740746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d73fff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74100930 0x7f0d740746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d73fff700 1 -- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d74101b20 msgr2=0x7f0d74072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d73fff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d74101b20 0x7f0d74072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d73fff700 1 -- 172.21.15.99:0/271461051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640095f0 con 0x7f0d741024c0 2023-08-25T08:57:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d737fe700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74100930 0x7f0d740746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d73fff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d741024c0 0x7f0d74073250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00ef30 tx=0x7f0d6c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d70ff9700 1 -- 172.21.15.99:0/271461051 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c018070 con 0x7f0d741024c0 2023-08-25T08:57:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d70ff9700 1 -- 172.21.15.99:0/271461051 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6c00f040 con 0x7f0d741024c0 2023-08-25T08:57:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d741a08d0 con 0x7f0d741024c0 2023-08-25T08:57:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d70ff9700 1 -- 172.21.15.99:0/271461051 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c00cdb0 con 0x7f0d741024c0 2023-08-25T08:57:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.209+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d741a0de0 con 0x7f0d741024c0 2023-08-25T08:57:27.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.210+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7410c190 con 0x7f0d741024c0 2023-08-25T08:57:27.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.212+0000 7f0d70ff9700 1 -- 172.21.15.99:0/271461051 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f0d6c003a00 con 0x7f0d741024c0 2023-08-25T08:57:27.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.213+0000 7f0d70ff9700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0d5c0a00a0 0x7f0d5c0a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:27.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.213+0000 7f0d70ff9700 1 -- 172.21.15.99:0/271461051 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0d6c0caee0 con 0x7f0d741024c0 2023-08-25T08:57:27.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.213+0000 7f0d737fe700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0d5c0a00a0 0x7f0d5c0a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:27.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.214+0000 7f0d737fe700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0d5c0a00a0 0x7f0d5c0a2560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009990 tx=0x7f0d6401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:27.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.218+0000 7f0d70ff9700 1 -- 172.21.15.99:0/271461051 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0d6c094c50 con 0x7f0d741024c0 2023-08-25T08:57:27.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:27 smithi099 ceph-mon[97641]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:27 smithi099 ceph-mon[97641]: pgmap v269: 97 pgs: 97 active+clean; 596 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-08-25T08:57:27.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:27 smithi099 ceph-mon[97641]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:27.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:27 smithi157 ceph-mon[130879]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:27.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:27 smithi157 ceph-mon[130879]: pgmap v269: 97 pgs: 97 active+clean; 596 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-08-25T08:57:27.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:27 smithi157 ceph-mon[130879]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:27.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.463+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d7410c380 con 0x7f0d741024c0 2023-08-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.464+0000 7f0d70ff9700 1 -- 172.21.15.99:0/271461051 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11283 (secure 0 0 0) 0x7f0d6c098270 con 0x7f0d741024c0 2023-08-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:27.465 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T08:57:19.244738+0000","modified":"2023-08-25T08:57:25.030649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24547,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T08:57:27.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.466+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0d5c0a00a0 msgr2=0x7f0d5c0a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.466+0000 7f0d79dff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0d5c0a00a0 0x7f0d5c0a2560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009990 tx=0x7f0d6401a040 comp rx=0 tx=0).stop 2023-08-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.466+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d741024c0 msgr2=0x7f0d74073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.466+0000 7f0d79dff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d741024c0 0x7f0d74073250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00ef30 tx=0x7f0d6c00c3a0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.467+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 shutdown_connections 2023-08-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.467+0000 7f0d79dff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0d5c0a00a0 0x7f0d5c0a2560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.467+0000 7f0d79dff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d741024c0 0x7f0d74073250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.467+0000 7f0d79dff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74100930 0x7f0d740746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.467+0000 7f0d79dff700 1 --2- 172.21.15.99:0/271461051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d74101b20 0x7f0d74072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.467+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 >> 172.21.15.99:0/271461051 conn(0x7f0d740fbe60 msgr2=0x7f0d740fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.467+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 shutdown_connections 2023-08-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.467+0000 7f0d79dff700 1 -- 172.21.15.99:0/271461051 wait complete. 2023-08-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2023-08-25T08:57:27.486 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:57:27.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:27 smithi181 ceph-mon[131296]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:27 smithi181 ceph-mon[131296]: pgmap v269: 97 pgs: 97 active+clean; 596 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-08-25T08:57:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:27 smithi181 ceph-mon[131296]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:27.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 -- 172.21.15.99:0/1118994552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8920103cc0 msgr2=0x7f89201060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 --2- 172.21.15.99:0/1118994552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8920103cc0 0x7f89201060b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8908009990 tx=0x7f8908009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 -- 172.21.15.99:0/1118994552 shutdown_connections 2023-08-25T08:57:27.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 --2- 172.21.15.99:0/1118994552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89201065f0 0x7f8920108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 --2- 172.21.15.99:0/1118994552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8920101390 0x7f8920103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 --2- 172.21.15.99:0/1118994552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8920103cc0 0x7f89201060b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 -- 172.21.15.99:0/1118994552 >> 172.21.15.99:0/1118994552 conn(0x7f89200fad80 msgr2=0x7f89200fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:27.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 -- 172.21.15.99:0/1118994552 shutdown_connections 2023-08-25T08:57:27.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.685+0000 7f89253ff700 1 -- 172.21.15.99:0/1118994552 wait complete. 2023-08-25T08:57:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.686+0000 7f89253ff700 1 Processor -- start 2023-08-25T08:57:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.686+0000 7f89253ff700 1 -- start start 2023-08-25T08:57:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.686+0000 7f89253ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8920101390 0x7f8920100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.686+0000 7f89253ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8920103cc0 0x7f89200ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.686+0000 7f89253ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89201065f0 0x7f89200ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:27.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.686+0000 7f89253ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892019a3b0 con 0x7f89201065f0 2023-08-25T08:57:27.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.686+0000 7f89253ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89200ffdc0 con 0x7f8920103cc0 2023-08-25T08:57:27.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.686+0000 7f89253ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89200fff80 con 0x7f8920101390 2023-08-25T08:57:27.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89201065f0 0x7f89200ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:27.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8920101390 0x7f8920100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8920101390 0x7f8920100d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:60312/0 (socket says 172.21.15.99:60312) 2023-08-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 -- 172.21.15.99:0/3049367416 learned_addr learned my addr 172.21.15.99:0/3049367416 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f8917fff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8920103cc0 0x7f89200ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 -- 172.21.15.99:0/3049367416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8920103cc0 msgr2=0x7f89200ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8920103cc0 0x7f89200ff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 -- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89201065f0 msgr2=0x7f89200ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89201065f0 0x7f89200ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 -- 172.21.15.99:0/3049367416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080095f0 con 0x7f8920101390 2023-08-25T08:57:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891f7fe700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89201065f0 0x7f89200ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.687+0000 7f891effd700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8920101390 0x7f8920100d00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f891000e9a0 tx=0x7f891000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.688+0000 7f891cff9700 1 -- 172.21.15.99:0/3049367416 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910009790 con 0x7f8920101390 2023-08-25T08:57:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.688+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89201a4d60 con 0x7f8920101390 2023-08-25T08:57:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.688+0000 7f891cff9700 1 -- 172.21.15.99:0/3049367416 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8910010430 con 0x7f8920101390 2023-08-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.688+0000 7f891cff9700 1 -- 172.21.15.99:0/3049367416 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910018680 con 0x7f8920101390 2023-08-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.688+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89201a5190 con 0x7f8920101390 2023-08-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.689+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89200fc900 con 0x7f8920101390 2023-08-25T08:57:27.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.691+0000 7f891cff9700 1 -- 172.21.15.99:0/3049367416 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f89100105a0 con 0x7f8920101390 2023-08-25T08:57:27.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.692+0000 7f891cff9700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f890c0a0020 0x7f890c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:27.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.692+0000 7f891cff9700 1 -- 172.21.15.99:0/3049367416 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8910014070 con 0x7f8920101390 2023-08-25T08:57:27.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.692+0000 7f8917fff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f890c0a0020 0x7f890c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:27.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.693+0000 7f8917fff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f890c0a0020 0x7f890c0a24e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f890800b2e0 tx=0x7f8908005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:27.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.694+0000 7f891cff9700 1 -- 172.21.15.99:0/3049367416 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8910095390 con 0x7f8920101390 2023-08-25T08:57:27.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.898+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f892010c390 con 0x7f8920101390 2023-08-25T08:57:27.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.900+0000 7f891cff9700 1 -- 172.21.15.99:0/3049367416 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f89100989b0 con 0x7f8920101390 2023-08-25T08:57:27.902 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:27.902 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:20.500911+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:57:27.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.902+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f890c0a0020 msgr2=0x7f890c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.902+0000 7f89253ff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f890c0a0020 0x7f890c0a24e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f890800b2e0 tx=0x7f8908005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8920101390 msgr2=0x7f8920100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:27.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8920101390 0x7f8920100d00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f891000e9a0 tx=0x7f891000ecb0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 shutdown_connections 2023-08-25T08:57:27.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89201065f0 0x7f89200ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8920101390 0x7f8920100d00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f890c0a0020 0x7f890c0a24e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 --2- 172.21.15.99:0/3049367416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8920103cc0 0x7f89200ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:27.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 >> 172.21.15.99:0/3049367416 conn(0x7f89200fad80 msgr2=0x7f8920102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:27.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 shutdown_connections 2023-08-25T08:57:27.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:27.903+0000 7f89253ff700 1 -- 172.21.15.99:0/3049367416 wait complete. 2023-08-25T08:57:27.922 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T08:57:27.923 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T08:57:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 -- 172.21.15.99:0/2656023647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee60101b20 msgr2=0x7fee60101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 --2- 172.21.15.99:0/2656023647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee60101b20 0x7fee60101f80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee500099c0 tx=0x7fee50009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 -- 172.21.15.99:0/2656023647 shutdown_connections 2023-08-25T08:57:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 --2- 172.21.15.99:0/2656023647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee601024c0 0x7fee60108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 --2- 172.21.15.99:0/2656023647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee60100930 0x7fee60100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 --2- 172.21.15.99:0/2656023647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee60101b20 0x7fee60101f80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 -- 172.21.15.99:0/2656023647 >> 172.21.15.99:0/2656023647 conn(0x7fee600fbe60 msgr2=0x7fee600fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 -- 172.21.15.99:0/2656023647 shutdown_connections 2023-08-25T08:57:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.120+0000 7fee685ff700 1 -- 172.21.15.99:0/2656023647 wait complete. 2023-08-25T08:57:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.121+0000 7fee685ff700 1 Processor -- start 2023-08-25T08:57:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee685ff700 1 -- start start 2023-08-25T08:57:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee685ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee60100930 0x7fee601987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee685ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee60101b20 0x7fee60198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee685ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee601024c0 0x7fee6019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee685ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee601a0360 con 0x7fee60101b20 2023-08-25T08:57:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee685ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee601a04a0 con 0x7fee60100930 2023-08-25T08:57:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee685ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee601a0630 con 0x7fee601024c0 2023-08-25T08:57:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee6639b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee60100930 0x7fee601987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee6639b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee60100930 0x7fee601987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:45564/0 (socket says 172.21.15.99:45564) 2023-08-25T08:57:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee6639b700 1 -- 172.21.15.99:0/2657304125 learned_addr learned my addr 172.21.15.99:0/2657304125 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee65b9a700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee60101b20 0x7fee60198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee66b9c700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee601024c0 0x7fee6019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee6639b700 1 -- 172.21.15.99:0/2657304125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee601024c0 msgr2=0x7fee6019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee6639b700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee601024c0 0x7fee6019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee6639b700 1 -- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee60101b20 msgr2=0x7fee60198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.122+0000 7fee6639b700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee60101b20 0x7fee60198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.123+0000 7fee6639b700 1 -- 172.21.15.99:0/2657304125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500095f0 con 0x7fee60100930 2023-08-25T08:57:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.123+0000 7fee66b9c700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee601024c0 0x7fee6019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:57:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.123+0000 7fee6639b700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee60100930 0x7fee601987e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00e970 tx=0x7fee5c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.123+0000 7fee577fe700 1 -- 172.21.15.99:0/2657304125 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c009790 con 0x7fee60100930 2023-08-25T08:57:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.123+0000 7fee577fe700 1 -- 172.21.15.99:0/2657304125 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee5c00eec0 con 0x7fee60100930 2023-08-25T08:57:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.123+0000 7fee577fe700 1 -- 172.21.15.99:0/2657304125 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c003cf0 con 0x7fee60100930 2023-08-25T08:57:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.123+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee601a0910 con 0x7fee60100930 2023-08-25T08:57:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.123+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee601a0dd0 con 0x7fee60100930 2023-08-25T08:57:28.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.124+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6010c190 con 0x7fee60100930 2023-08-25T08:57:28.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.126+0000 7fee577fe700 1 -- 172.21.15.99:0/2657304125 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fee5c00ccd0 con 0x7fee60100930 2023-08-25T08:57:28.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.127+0000 7fee577fe700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee4c0a00a0 0x7fee4c0a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:28.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.127+0000 7fee577fe700 1 -- 172.21.15.99:0/2657304125 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fee5c014070 con 0x7fee60100930 2023-08-25T08:57:28.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.127+0000 7fee65b9a700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee4c0a00a0 0x7fee4c0a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:28.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.128+0000 7fee65b9a700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee4c0a00a0 0x7fee4c0a2560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b580 tx=0x7fee50005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.129+0000 7fee577fe700 1 -- 172.21.15.99:0/2657304125 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fee5c095380 con 0x7fee60100930 2023-08-25T08:57:28.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:28 smithi099 ceph-mon[97641]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:28 smithi099 ceph-mon[97641]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-08-25T08:57:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:28 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/271461051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:28.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:28 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3049367416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:28.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.324+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee60108410 con 0x7fee4c0a00a0 2023-08-25T08:57:28.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.337+0000 7fee577fe700 1 -- 172.21.15.99:0/2657304125 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335578 (secure 0 0 0) 0x7fee60108410 con 0x7fee4c0a00a0 2023-08-25T08:57:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:28.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.341+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee4c0a00a0 msgr2=0x7fee4c0a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.341+0000 7fee685ff700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee4c0a00a0 0x7fee4c0a2560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b580 tx=0x7fee50005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.341+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee60100930 msgr2=0x7fee601987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.341+0000 7fee685ff700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee60100930 0x7fee601987e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00e970 tx=0x7fee5c00ec80 comp rx=0 tx=0).stop 2023-08-25T08:57:28.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.342+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 shutdown_connections 2023-08-25T08:57:28.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.342+0000 7fee685ff700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee601024c0 0x7fee6019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.342+0000 7fee685ff700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fee4c0a00a0 0x7fee4c0a2560 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.342+0000 7fee685ff700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee60100930 0x7fee601987e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.342+0000 7fee685ff700 1 --2- 172.21.15.99:0/2657304125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee60101b20 0x7fee60198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.342+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 >> 172.21.15.99:0/2657304125 conn(0x7fee600fbe60 msgr2=0x7fee600fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.342+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 shutdown_connections 2023-08-25T08:57:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.342+0000 7fee685ff700 1 -- 172.21.15.99:0/2657304125 wait complete. 2023-08-25T08:57:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-08-25T08:57:28.361 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2023-08-25T08:57:26.437078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":610580,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":149,"num_read_kb":130,"num_write":276,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":279,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334952,"kb_used_data":10712,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606872,"statfs":{"total":1151940427776,"available":1151597436928,"internally_reserved":0,"allocated":10969088,"data_stored":6898908,"data_compressed":108807,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20212,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":95,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.735919"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920834+0000","last_change":"2023-08-25T08:57:19.519342+0000","last_active":"2023-08-25T08:57:21.920834+0000","last_peered":"2023-08-25T08:57:21.920834+0000","last_clean":"2023-08-25T08:57:21.920834+0000","last_became_active":"2023-08-25T08:57:19.519006+0000","last_became_peered":"2023-08-25T08:57:19.519006+0000","last_unstale":"2023-08-25T08:57:21.920834+0000","last_undegraded":"2023-08-25T08:57:21.920834+0000","last_fullsized":"2023-08-25T08:57:21.920834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:32.976833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577376+0000","last_change":"2023-08-25T08:57:19.521779+0000","last_active":"2023-08-25T08:57:21.577376+0000","last_peered":"2023-08-25T08:57:21.577376+0000","last_clean":"2023-08-25T08:57:21.577376+0000","last_became_active":"2023-08-25T08:57:19.521358+0000","last_became_peered":"2023-08-25T08:57:19.521358+0000","last_unstale":"2023-08-25T08:57:21.577376+0000","last_undegraded":"2023-08-25T08:57:21.577376+0000","last_fullsized":"2023-08-25T08:57:21.577376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:23:27.999946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577350+0000","last_change":"2023-08-25T08:57:19.531553+0000","last_active":"2023-08-25T08:57:21.577350+0000","last_peered":"2023-08-25T08:57:21.577350+0000","last_clean":"2023-08-25T08:57:21.577350+0000","last_became_active":"2023-08-25T08:57:19.531113+0000","last_became_peered":"2023-08-25T08:57:19.531113+0000","last_unstale":"2023-08-25T08:57:21.577350+0000","last_undegraded":"2023-08-25T08:57:21.577350+0000","last_fullsized":"2023-08-25T08:57:21.577350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:57.221411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.498448+0000","last_change":"2023-08-25T08:57:19.519696+0000","last_active":"2023-08-25T08:57:21.498448+0000","last_peered":"2023-08-25T08:57:21.498448+0000","last_clean":"2023-08-25T08:57:21.498448+0000","last_became_active":"2023-08-25T08:57:19.518918+0000","last_became_peered":"2023-08-25T08:57:19.518918+0000","last_unstale":"2023-08-25T08:57:21.498448+0000","last_undegraded":"2023-08-25T08:57:21.498448+0000","last_fullsized":"2023-08-25T08:57:21.498448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:44:46.112412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.861989+0000","last_change":"2023-08-25T08:57:19.520706+0000","last_active":"2023-08-25T08:57:20.861989+0000","last_peered":"2023-08-25T08:57:20.861989+0000","last_clean":"2023-08-25T08:57:20.861989+0000","last_became_active":"2023-08-25T08:57:19.520333+0000","last_became_peered":"2023-08-25T08:57:19.520333+0000","last_unstale":"2023-08-25T08:57:20.861989+0000","last_undegraded":"2023-08-25T08:57:20.861989+0000","last_fullsized":"2023-08-25T08:57:20.861989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:25.722869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577779+0000","last_change":"2023-08-25T08:57:19.520040+0000","last_active":"2023-08-25T08:57:21.577779+0000","last_peered":"2023-08-25T08:57:21.577779+0000","last_clean":"2023-08-25T08:57:21.577779+0000","last_became_active":"2023-08-25T08:57:19.519726+0000","last_became_peered":"2023-08-25T08:57:19.519726+0000","last_unstale":"2023-08-25T08:57:21.577779+0000","last_undegraded":"2023-08-25T08:57:21.577779+0000","last_fullsized":"2023-08-25T08:57:21.577779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:36.003956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578052+0000","last_change":"2023-08-25T08:57:19.518064+0000","last_active":"2023-08-25T08:57:21.578052+0000","last_peered":"2023-08-25T08:57:21.578052+0000","last_clean":"2023-08-25T08:57:21.578052+0000","last_became_active":"2023-08-25T08:57:19.517725+0000","last_became_peered":"2023-08-25T08:57:19.517725+0000","last_unstale":"2023-08-25T08:57:21.578052+0000","last_undegraded":"2023-08-25T08:57:21.578052+0000","last_fullsized":"2023-08-25T08:57:21.578052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:15:39.794457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.507687+0000","last_change":"2023-08-25T08:57:19.527429+0000","last_active":"2023-08-25T08:57:20.507687+0000","last_peered":"2023-08-25T08:57:20.507687+0000","last_clean":"2023-08-25T08:57:20.507687+0000","last_became_active":"2023-08-25T08:57:19.526370+0000","last_became_peered":"2023-08-25T08:57:19.526370+0000","last_unstale":"2023-08-25T08:57:20.507687+0000","last_undegraded":"2023-08-25T08:57:20.507687+0000","last_fullsized":"2023-08-25T08:57:20.507687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:05.241223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578087+0000","last_change":"2023-08-25T08:57:19.517136+0000","last_active":"2023-08-25T08:57:21.578087+0000","last_peered":"2023-08-25T08:57:21.578087+0000","last_clean":"2023-08-25T08:57:21.578087+0000","last_became_active":"2023-08-25T08:57:19.516852+0000","last_became_peered":"2023-08-25T08:57:19.516852+0000","last_unstale":"2023-08-25T08:57:21.578087+0000","last_undegraded":"2023-08-25T08:57:21.578087+0000","last_fullsized":"2023-08-25T08:57:21.578087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:38.248094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.579207+0000","last_change":"2023-08-25T08:57:19.522122+0000","last_active":"2023-08-25T08:57:21.579207+0000","last_peered":"2023-08-25T08:57:21.579207+0000","last_clean":"2023-08-25T08:57:21.579207+0000","last_became_active":"2023-08-25T08:57:19.520984+0000","last_became_peered":"2023-08-25T08:57:19.520984+0000","last_unstale":"2023-08-25T08:57:21.579207+0000","last_undegraded":"2023-08-25T08:57:21.579207+0000","last_fullsized":"2023-08-25T08:57:21.579207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:59.542512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578697+0000","last_change":"2023-08-25T08:57:19.531356+0000","last_active":"2023-08-25T08:57:21.578697+0000","last_peered":"2023-08-25T08:57:21.578697+0000","last_clean":"2023-08-25T08:57:21.578697+0000","last_became_active":"2023-08-25T08:57:19.531050+0000","last_became_peered":"2023-08-25T08:57:19.531050+0000","last_unstale":"2023-08-25T08:57:21.578697+0000","last_undegraded":"2023-08-25T08:57:21.578697+0000","last_fullsized":"2023-08-25T08:57:21.578697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:40:32.714726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.498416+0000","last_change":"2023-08-25T08:57:19.520091+0000","last_active":"2023-08-25T08:57:21.498416+0000","last_peered":"2023-08-25T08:57:21.498416+0000","last_clean":"2023-08-25T08:57:21.498416+0000","last_became_active":"2023-08-25T08:57:19.519356+0000","last_became_peered":"2023-08-25T08:57:19.519356+0000","last_unstale":"2023-08-25T08:57:21.498416+0000","last_undegraded":"2023-08-25T08:57:21.498416+0000","last_fullsized":"2023-08-25T08:57:21.498416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:12:05.753053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509577+0000","last_change":"2023-08-25T08:57:19.515446+0000","last_active":"2023-08-25T08:57:20.509577+0000","last_peered":"2023-08-25T08:57:20.509577+0000","last_clean":"2023-08-25T08:57:20.509577+0000","last_became_active":"2023-08-25T08:57:19.514376+0000","last_became_peered":"2023-08-25T08:57:19.514376+0000","last_unstale":"2023-08-25T08:57:20.509577+0000","last_undegraded":"2023-08-25T08:57:20.509577+0000","last_fullsized":"2023-08-25T08:57:20.509577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:23:44.549208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577354+0000","last_change":"2023-08-25T08:57:19.521629+0000","last_active":"2023-08-25T08:57:21.577354+0000","last_peered":"2023-08-25T08:57:21.577354+0000","last_clean":"2023-08-25T08:57:21.577354+0000","last_became_active":"2023-08-25T08:57:19.521307+0000","last_became_peered":"2023-08-25T08:57:19.521307+0000","last_unstale":"2023-08-25T08:57:21.577354+0000","last_undegraded":"2023-08-25T08:57:21.577354+0000","last_fullsized":"2023-08-25T08:57:21.577354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:19.552027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577515+0000","last_change":"2023-08-25T08:57:19.521670+0000","last_active":"2023-08-25T08:57:21.577515+0000","last_peered":"2023-08-25T08:57:21.577515+0000","last_clean":"2023-08-25T08:57:21.577515+0000","last_became_active":"2023-08-25T08:57:19.521325+0000","last_became_peered":"2023-08-25T08:57:19.521325+0000","last_unstale":"2023-08-25T08:57:21.577515+0000","last_undegraded":"2023-08-25T08:57:21.577515+0000","last_fullsized":"2023-08-25T08:57:21.577515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:24:27.589627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577208+0000","last_change":"2023-08-25T08:57:19.526733+0000","last_active":"2023-08-25T08:57:21.577208+0000","last_peered":"2023-08-25T08:57:21.577208+0000","last_clean":"2023-08-25T08:57:21.577208+0000","last_became_active":"2023-08-25T08:57:19.525692+0000","last_became_peered":"2023-08-25T08:57:19.525692+0000","last_unstale":"2023-08-25T08:57:21.577208+0000","last_undegraded":"2023-08-25T08:57:21.577208+0000","last_fullsized":"2023-08-25T08:57:21.577208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:58:39.725840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920886+0000","last_change":"2023-08-25T08:57:19.519289+0000","last_active":"2023-08-25T08:57:21.920886+0000","last_peered":"2023-08-25T08:57:21.920886+0000","last_clean":"2023-08-25T08:57:21.920886+0000","last_became_active":"2023-08-25T08:57:19.518584+0000","last_became_peered":"2023-08-25T08:57:19.518584+0000","last_unstale":"2023-08-25T08:57:21.920886+0000","last_undegraded":"2023-08-25T08:57:21.920886+0000","last_fullsized":"2023-08-25T08:57:21.920886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:56:36.031877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577761+0000","last_change":"2023-08-25T08:57:19.525090+0000","last_active":"2023-08-25T08:57:21.577761+0000","last_peered":"2023-08-25T08:57:21.577761+0000","last_clean":"2023-08-25T08:57:21.577761+0000","last_became_active":"2023-08-25T08:57:19.524543+0000","last_became_peered":"2023-08-25T08:57:19.524543+0000","last_unstale":"2023-08-25T08:57:21.577761+0000","last_undegraded":"2023-08-25T08:57:21.577761+0000","last_fullsized":"2023-08-25T08:57:21.577761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:41:42.783908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506702+0000","last_change":"2023-08-25T08:57:19.526761+0000","last_active":"2023-08-25T08:57:20.506702+0000","last_peered":"2023-08-25T08:57:20.506702+0000","last_clean":"2023-08-25T08:57:20.506702+0000","last_became_active":"2023-08-25T08:57:19.526421+0000","last_became_peered":"2023-08-25T08:57:19.526421+0000","last_unstale":"2023-08-25T08:57:20.506702+0000","last_undegraded":"2023-08-25T08:57:20.506702+0000","last_fullsized":"2023-08-25T08:57:20.506702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:30.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.041023+0000","last_change":"2023-08-25T08:57:18.717584+0000","last_active":"2023-08-25T08:57:24.041023+0000","last_peered":"2023-08-25T08:57:24.041023+0000","last_clean":"2023-08-25T08:57:24.041023+0000","last_became_active":"2023-08-25T08:57:18.717284+0000","last_became_peered":"2023-08-25T08:57:18.717284+0000","last_unstale":"2023-08-25T08:57:24.041023+0000","last_undegraded":"2023-08-25T08:57:24.041023+0000","last_fullsized":"2023-08-25T08:57:24.041023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:11:26.749184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508383+0000","last_change":"2023-08-25T08:57:19.527089+0000","last_active":"2023-08-25T08:57:20.508383+0000","last_peered":"2023-08-25T08:57:20.508383+0000","last_clean":"2023-08-25T08:57:20.508383+0000","last_became_active":"2023-08-25T08:57:19.526118+0000","last_became_peered":"2023-08-25T08:57:19.526118+0000","last_unstale":"2023-08-25T08:57:20.508383+0000","last_undegraded":"2023-08-25T08:57:20.508383+0000","last_fullsized":"2023-08-25T08:57:20.508383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:10:05.421256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508441+0000","last_change":"2023-08-25T08:57:18.720269+0000","last_active":"2023-08-25T08:57:20.508441+0000","last_peered":"2023-08-25T08:57:20.508441+0000","last_clean":"2023-08-25T08:57:20.508441+0000","last_became_active":"2023-08-25T08:57:18.719878+0000","last_became_peered":"2023-08-25T08:57:18.719878+0000","last_unstale":"2023-08-25T08:57:20.508441+0000","last_undegraded":"2023-08-25T08:57:20.508441+0000","last_fullsized":"2023-08-25T08:57:20.508441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:24:11.739039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577572+0000","last_change":"2023-08-25T08:57:19.531835+0000","last_active":"2023-08-25T08:57:21.577572+0000","last_peered":"2023-08-25T08:57:21.577572+0000","last_clean":"2023-08-25T08:57:21.577572+0000","last_became_active":"2023-08-25T08:57:19.531409+0000","last_became_peered":"2023-08-25T08:57:19.531409+0000","last_unstale":"2023-08-25T08:57:21.577572+0000","last_undegraded":"2023-08-25T08:57:21.577572+0000","last_fullsized":"2023-08-25T08:57:21.577572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:55:00.124607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.038085+0000","last_change":"2023-08-25T08:57:18.716847+0000","last_active":"2023-08-25T08:57:24.038085+0000","last_peered":"2023-08-25T08:57:24.038085+0000","last_clean":"2023-08-25T08:57:24.038085+0000","last_became_active":"2023-08-25T08:57:18.716416+0000","last_became_peered":"2023-08-25T08:57:18.716416+0000","last_unstale":"2023-08-25T08:57:24.038085+0000","last_undegraded":"2023-08-25T08:57:24.038085+0000","last_fullsized":"2023-08-25T08:57:24.038085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:13:17.069695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577237+0000","last_change":"2023-08-25T08:57:19.526495+0000","last_active":"2023-08-25T08:57:21.577237+0000","last_peered":"2023-08-25T08:57:21.577237+0000","last_clean":"2023-08-25T08:57:21.577237+0000","last_became_active":"2023-08-25T08:57:19.525204+0000","last_became_peered":"2023-08-25T08:57:19.525204+0000","last_unstale":"2023-08-25T08:57:21.577237+0000","last_undegraded":"2023-08-25T08:57:21.577237+0000","last_fullsized":"2023-08-25T08:57:21.577237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:52:21.853120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.862027+0000","last_change":"2023-08-25T08:57:18.715832+0000","last_active":"2023-08-25T08:57:20.862027+0000","last_peered":"2023-08-25T08:57:20.862027+0000","last_clean":"2023-08-25T08:57:20.862027+0000","last_became_active":"2023-08-25T08:57:18.714703+0000","last_became_peered":"2023-08-25T08:57:18.714703+0000","last_unstale":"2023-08-25T08:57:20.862027+0000","last_undegraded":"2023-08-25T08:57:20.862027+0000","last_fullsized":"2023-08-25T08:57:20.862027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:01:21.467556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577900+0000","last_change":"2023-08-25T08:57:19.527500+0000","last_active":"2023-08-25T08:57:21.577900+0000","last_peered":"2023-08-25T08:57:21.577900+0000","last_clean":"2023-08-25T08:57:21.577900+0000","last_became_active":"2023-08-25T08:57:19.527172+0000","last_became_peered":"2023-08-25T08:57:19.527172+0000","last_unstale":"2023-08-25T08:57:21.577900+0000","last_undegraded":"2023-08-25T08:57:21.577900+0000","last_fullsized":"2023-08-25T08:57:21.577900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:45.376591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577580+0000","last_change":"2023-08-25T08:57:18.715563+0000","last_active":"2023-08-25T08:57:21.577580+0000","last_peered":"2023-08-25T08:57:21.577580+0000","last_clean":"2023-08-25T08:57:21.577580+0000","last_became_active":"2023-08-25T08:57:18.715269+0000","last_became_peered":"2023-08-25T08:57:18.715269+0000","last_unstale":"2023-08-25T08:57:21.577580+0000","last_undegraded":"2023-08-25T08:57:21.577580+0000","last_fullsized":"2023-08-25T08:57:21.577580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:14:41.660718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.861871+0000","last_change":"2023-08-25T08:57:19.518705+0000","last_active":"2023-08-25T08:57:20.861871+0000","last_peered":"2023-08-25T08:57:20.861871+0000","last_clean":"2023-08-25T08:57:20.861871+0000","last_became_active":"2023-08-25T08:57:19.518422+0000","last_became_peered":"2023-08-25T08:57:19.518422+0000","last_unstale":"2023-08-25T08:57:20.861871+0000","last_undegraded":"2023-08-25T08:57:20.861871+0000","last_fullsized":"2023-08-25T08:57:20.861871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:52:45.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.039955+0000","last_change":"2023-08-25T08:57:18.714087+0000","last_active":"2023-08-25T08:57:24.039955+0000","last_peered":"2023-08-25T08:57:24.039955+0000","last_clean":"2023-08-25T08:57:24.039955+0000","last_became_active":"2023-08-25T08:57:18.713777+0000","last_became_peered":"2023-08-25T08:57:18.713777+0000","last_unstale":"2023-08-25T08:57:24.039955+0000","last_undegraded":"2023-08-25T08:57:24.039955+0000","last_fullsized":"2023-08-25T08:57:24.039955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:19:44.987996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577763+0000","last_change":"2023-08-25T08:57:19.518068+0000","last_active":"2023-08-25T08:57:21.577763+0000","last_peered":"2023-08-25T08:57:21.577763+0000","last_clean":"2023-08-25T08:57:21.577763+0000","last_became_active":"2023-08-25T08:57:19.517752+0000","last_became_peered":"2023-08-25T08:57:19.517752+0000","last_unstale":"2023-08-25T08:57:21.577763+0000","last_undegraded":"2023-08-25T08:57:21.577763+0000","last_fullsized":"2023-08-25T08:57:21.577763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:10.746641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509232+0000","last_change":"2023-08-25T08:57:18.713652+0000","last_active":"2023-08-25T08:57:20.509232+0000","last_peered":"2023-08-25T08:57:20.509232+0000","last_clean":"2023-08-25T08:57:20.509232+0000","last_became_active":"2023-08-25T08:57:18.713348+0000","last_became_peered":"2023-08-25T08:57:18.713348+0000","last_unstale":"2023-08-25T08:57:20.509232+0000","last_undegraded":"2023-08-25T08:57:20.509232+0000","last_fullsized":"2023-08-25T08:57:20.509232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:22.041936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.507567+0000","last_change":"2023-08-25T08:57:19.521626+0000","last_active":"2023-08-25T08:57:20.507567+0000","last_peered":"2023-08-25T08:57:20.507567+0000","last_clean":"2023-08-25T08:57:20.507567+0000","last_became_active":"2023-08-25T08:57:19.520055+0000","last_became_peered":"2023-08-25T08:57:19.520055+0000","last_unstale":"2023-08-25T08:57:20.507567+0000","last_undegraded":"2023-08-25T08:57:20.507567+0000","last_fullsized":"2023-08-25T08:57:20.507567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:12.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577370+0000","last_change":"2023-08-25T08:57:18.713735+0000","last_active":"2023-08-25T08:57:21.577370+0000","last_peered":"2023-08-25T08:57:21.577370+0000","last_clean":"2023-08-25T08:57:21.577370+0000","last_became_active":"2023-08-25T08:57:18.713421+0000","last_became_peered":"2023-08-25T08:57:18.713421+0000","last_unstale":"2023-08-25T08:57:21.577370+0000","last_undegraded":"2023-08-25T08:57:21.577370+0000","last_fullsized":"2023-08-25T08:57:21.577370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:52.319311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577707+0000","last_change":"2023-08-25T08:57:19.513137+0000","last_active":"2023-08-25T08:57:21.577707+0000","last_peered":"2023-08-25T08:57:21.577707+0000","last_clean":"2023-08-25T08:57:21.577707+0000","last_became_active":"2023-08-25T08:57:19.512804+0000","last_became_peered":"2023-08-25T08:57:19.512804+0000","last_unstale":"2023-08-25T08:57:21.577707+0000","last_undegraded":"2023-08-25T08:57:21.577707+0000","last_fullsized":"2023-08-25T08:57:21.577707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:12:45.443703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.029804+0000","last_change":"2023-08-25T08:57:18.712855+0000","last_active":"2023-08-25T08:57:24.029804+0000","last_peered":"2023-08-25T08:57:24.029804+0000","last_clean":"2023-08-25T08:57:24.029804+0000","last_became_active":"2023-08-25T08:57:18.712327+0000","last_became_peered":"2023-08-25T08:57:18.712327+0000","last_unstale":"2023-08-25T08:57:24.029804+0000","last_undegraded":"2023-08-25T08:57:24.029804+0000","last_fullsized":"2023-08-25T08:57:24.029804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:03:13.409284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578251+0000","last_change":"2023-08-25T08:57:19.526933+0000","last_active":"2023-08-25T08:57:21.578251+0000","last_peered":"2023-08-25T08:57:21.578251+0000","last_clean":"2023-08-25T08:57:21.578251+0000","last_became_active":"2023-08-25T08:57:19.524798+0000","last_became_peered":"2023-08-25T08:57:19.524798+0000","last_unstale":"2023-08-25T08:57:21.578251+0000","last_undegraded":"2023-08-25T08:57:21.578251+0000","last_fullsized":"2023-08-25T08:57:21.578251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:34:33.721743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.063068+0000","last_change":"2023-08-25T08:57:18.714182+0000","last_active":"2023-08-25T08:57:23.063068+0000","last_peered":"2023-08-25T08:57:23.063068+0000","last_clean":"2023-08-25T08:57:23.063068+0000","last_became_active":"2023-08-25T08:57:18.713872+0000","last_became_peered":"2023-08-25T08:57:18.713872+0000","last_unstale":"2023-08-25T08:57:23.063068+0000","last_undegraded":"2023-08-25T08:57:23.063068+0000","last_fullsized":"2023-08-25T08:57:23.063068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:09:56.521066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577490+0000","last_change":"2023-08-25T08:57:19.530745+0000","last_active":"2023-08-25T08:57:21.577490+0000","last_peered":"2023-08-25T08:57:21.577490+0000","last_clean":"2023-08-25T08:57:21.577490+0000","last_became_active":"2023-08-25T08:57:19.530397+0000","last_became_peered":"2023-08-25T08:57:19.530397+0000","last_unstale":"2023-08-25T08:57:21.577490+0000","last_undegraded":"2023-08-25T08:57:21.577490+0000","last_fullsized":"2023-08-25T08:57:21.577490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:50:19.404416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577970+0000","last_change":"2023-08-25T08:57:18.716995+0000","last_active":"2023-08-25T08:57:21.577970+0000","last_peered":"2023-08-25T08:57:21.577970+0000","last_clean":"2023-08-25T08:57:21.577970+0000","last_became_active":"2023-08-25T08:57:18.716672+0000","last_became_peered":"2023-08-25T08:57:18.716672+0000","last_unstale":"2023-08-25T08:57:21.577970+0000","last_undegraded":"2023-08-25T08:57:21.577970+0000","last_fullsized":"2023-08-25T08:57:21.577970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:23:53.036884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577874+0000","last_change":"2023-08-25T08:57:19.526608+0000","last_active":"2023-08-25T08:57:21.577874+0000","last_peered":"2023-08-25T08:57:21.577874+0000","last_clean":"2023-08-25T08:57:21.577874+0000","last_became_active":"2023-08-25T08:57:19.525614+0000","last_became_peered":"2023-08-25T08:57:19.525614+0000","last_unstale":"2023-08-25T08:57:21.577874+0000","last_undegraded":"2023-08-25T08:57:21.577874+0000","last_fullsized":"2023-08-25T08:57:21.577874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:10:36.808709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'14","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:25.536047+0000","last_change":"2023-08-25T08:57:18.712588+0000","last_active":"2023-08-25T08:57:25.536047+0000","last_peered":"2023-08-25T08:57:25.536047+0000","last_clean":"2023-08-25T08:57:25.536047+0000","last_became_active":"2023-08-25T08:57:18.712259+0000","last_became_peered":"2023-08-25T08:57:18.712259+0000","last_unstale":"2023-08-25T08:57:25.536047+0000","last_undegraded":"2023-08-25T08:57:25.536047+0000","last_fullsized":"2023-08-25T08:57:25.536047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:30:37.621654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18977,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":15,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.862200+0000","last_change":"2023-08-25T08:57:19.517907+0000","last_active":"2023-08-25T08:57:20.862200+0000","last_peered":"2023-08-25T08:57:20.862200+0000","last_clean":"2023-08-25T08:57:20.862200+0000","last_became_active":"2023-08-25T08:57:19.517596+0000","last_became_peered":"2023-08-25T08:57:19.517596+0000","last_unstale":"2023-08-25T08:57:20.862200+0000","last_undegraded":"2023-08-25T08:57:20.862200+0000","last_fullsized":"2023-08-25T08:57:20.862200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:24:49.829820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506561+0000","last_change":"2023-08-25T08:57:18.721564+0000","last_active":"2023-08-25T08:57:20.506561+0000","last_peered":"2023-08-25T08:57:20.506561+0000","last_clean":"2023-08-25T08:57:20.506561+0000","last_became_active":"2023-08-25T08:57:18.721235+0000","last_became_peered":"2023-08-25T08:57:18.721235+0000","last_unstale":"2023-08-25T08:57:20.506561+0000","last_undegraded":"2023-08-25T08:57:20.506561+0000","last_fullsized":"2023-08-25T08:57:20.506561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:26:55.065629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509111+0000","last_change":"2023-08-25T08:57:19.518399+0000","last_active":"2023-08-25T08:57:20.509111+0000","last_peered":"2023-08-25T08:57:20.509111+0000","last_clean":"2023-08-25T08:57:20.509111+0000","last_became_active":"2023-08-25T08:57:19.518061+0000","last_became_peered":"2023-08-25T08:57:19.518061+0000","last_unstale":"2023-08-25T08:57:20.509111+0000","last_undegraded":"2023-08-25T08:57:20.509111+0000","last_fullsized":"2023-08-25T08:57:20.509111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:35:24.119537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577553+0000","last_change":"2023-08-25T08:57:18.722020+0000","last_active":"2023-08-25T08:57:21.577553+0000","last_peered":"2023-08-25T08:57:21.577553+0000","last_clean":"2023-08-25T08:57:21.577553+0000","last_became_active":"2023-08-25T08:57:18.721707+0000","last_became_peered":"2023-08-25T08:57:18.721707+0000","last_unstale":"2023-08-25T08:57:21.577553+0000","last_undegraded":"2023-08-25T08:57:21.577553+0000","last_fullsized":"2023-08-25T08:57:21.577553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:16:27.471685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578379+0000","last_change":"2023-08-25T08:57:19.513954+0000","last_active":"2023-08-25T08:57:21.578379+0000","last_peered":"2023-08-25T08:57:21.578379+0000","last_clean":"2023-08-25T08:57:21.578379+0000","last_became_active":"2023-08-25T08:57:19.513645+0000","last_became_peered":"2023-08-25T08:57:19.513645+0000","last_unstale":"2023-08-25T08:57:21.578379+0000","last_undegraded":"2023-08-25T08:57:21.578379+0000","last_fullsized":"2023-08-25T08:57:21.578379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:27:38.030869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.498330+0000","last_change":"2023-08-25T08:57:18.711499+0000","last_active":"2023-08-25T08:57:21.498330+0000","last_peered":"2023-08-25T08:57:21.498330+0000","last_clean":"2023-08-25T08:57:21.498330+0000","last_became_active":"2023-08-25T08:57:18.711223+0000","last_became_peered":"2023-08-25T08:57:18.711223+0000","last_unstale":"2023-08-25T08:57:21.498330+0000","last_undegraded":"2023-08-25T08:57:21.498330+0000","last_fullsized":"2023-08-25T08:57:21.498330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:34:20.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577471+0000","last_change":"2023-08-25T08:57:19.518663+0000","last_active":"2023-08-25T08:57:21.577471+0000","last_peered":"2023-08-25T08:57:21.577471+0000","last_clean":"2023-08-25T08:57:21.577471+0000","last_became_active":"2023-08-25T08:57:19.518394+0000","last_became_peered":"2023-08-25T08:57:19.518394+0000","last_unstale":"2023-08-25T08:57:21.577471+0000","last_undegraded":"2023-08-25T08:57:21.577471+0000","last_fullsized":"2023-08-25T08:57:21.577471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:35:26.927923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.041844+0000","last_change":"2023-08-25T08:57:18.718554+0000","last_active":"2023-08-25T08:57:24.041844+0000","last_peered":"2023-08-25T08:57:24.041844+0000","last_clean":"2023-08-25T08:57:24.041844+0000","last_became_active":"2023-08-25T08:57:18.718283+0000","last_became_peered":"2023-08-25T08:57:18.718283+0000","last_unstale":"2023-08-25T08:57:24.041844+0000","last_undegraded":"2023-08-25T08:57:24.041844+0000","last_fullsized":"2023-08-25T08:57:24.041844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:47:23.878565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577584+0000","last_change":"2023-08-25T08:57:19.520856+0000","last_active":"2023-08-25T08:57:21.577584+0000","last_peered":"2023-08-25T08:57:21.577584+0000","last_clean":"2023-08-25T08:57:21.577584+0000","last_became_active":"2023-08-25T08:57:19.520568+0000","last_became_peered":"2023-08-25T08:57:19.520568+0000","last_unstale":"2023-08-25T08:57:21.577584+0000","last_undegraded":"2023-08-25T08:57:21.577584+0000","last_fullsized":"2023-08-25T08:57:21.577584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:26:17.967186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.862211+0000","last_change":"2023-08-25T08:57:18.715972+0000","last_active":"2023-08-25T08:57:20.862211+0000","last_peered":"2023-08-25T08:57:20.862211+0000","last_clean":"2023-08-25T08:57:20.862211+0000","last_became_active":"2023-08-25T08:57:18.715519+0000","last_became_peered":"2023-08-25T08:57:18.715519+0000","last_unstale":"2023-08-25T08:57:20.862211+0000","last_undegraded":"2023-08-25T08:57:20.862211+0000","last_fullsized":"2023-08-25T08:57:20.862211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:37:26.806786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920710+0000","last_change":"2023-08-25T08:57:19.517877+0000","last_active":"2023-08-25T08:57:21.920710+0000","last_peered":"2023-08-25T08:57:21.920710+0000","last_clean":"2023-08-25T08:57:21.920710+0000","last_became_active":"2023-08-25T08:57:19.517539+0000","last_became_peered":"2023-08-25T08:57:19.517539+0000","last_unstale":"2023-08-25T08:57:21.920710+0000","last_undegraded":"2023-08-25T08:57:21.920710+0000","last_fullsized":"2023-08-25T08:57:21.920710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:29:20.537007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577827+0000","last_change":"2023-08-25T08:57:18.713835+0000","last_active":"2023-08-25T08:57:21.577827+0000","last_peered":"2023-08-25T08:57:21.577827+0000","last_clean":"2023-08-25T08:57:21.577827+0000","last_became_active":"2023-08-25T08:57:18.713510+0000","last_became_peered":"2023-08-25T08:57:18.713510+0000","last_unstale":"2023-08-25T08:57:21.577827+0000","last_undegraded":"2023-08-25T08:57:21.577827+0000","last_fullsized":"2023-08-25T08:57:21.577827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:51.073757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.579129+0000","last_change":"2023-08-25T08:57:19.526979+0000","last_active":"2023-08-25T08:57:21.579129+0000","last_peered":"2023-08-25T08:57:21.579129+0000","last_clean":"2023-08-25T08:57:21.579129+0000","last_became_active":"2023-08-25T08:57:19.526660+0000","last_became_peered":"2023-08-25T08:57:19.526660+0000","last_unstale":"2023-08-25T08:57:21.579129+0000","last_undegraded":"2023-08-25T08:57:21.579129+0000","last_fullsized":"2023-08-25T08:57:21.579129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:23:10.052607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578722+0000","last_change":"2023-08-25T08:57:18.717952+0000","last_active":"2023-08-25T08:57:21.578722+0000","last_peered":"2023-08-25T08:57:21.578722+0000","last_clean":"2023-08-25T08:57:21.578722+0000","last_became_active":"2023-08-25T08:57:18.717643+0000","last_became_peered":"2023-08-25T08:57:18.717643+0000","last_unstale":"2023-08-25T08:57:21.578722+0000","last_undegraded":"2023-08-25T08:57:21.578722+0000","last_fullsized":"2023-08-25T08:57:21.578722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:52:22.390179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577843+0000","last_change":"2023-08-25T08:57:19.521718+0000","last_active":"2023-08-25T08:57:21.577843+0000","last_peered":"2023-08-25T08:57:21.577843+0000","last_clean":"2023-08-25T08:57:21.577843+0000","last_became_active":"2023-08-25T08:57:19.521246+0000","last_became_peered":"2023-08-25T08:57:19.521246+0000","last_unstale":"2023-08-25T08:57:21.577843+0000","last_undegraded":"2023-08-25T08:57:21.577843+0000","last_fullsized":"2023-08-25T08:57:21.577843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:36.200645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577622+0000","last_change":"2023-08-25T08:57:19.518510+0000","last_active":"2023-08-25T08:57:21.577622+0000","last_peered":"2023-08-25T08:57:21.577622+0000","last_clean":"2023-08-25T08:57:21.577622+0000","last_became_active":"2023-08-25T08:57:19.518202+0000","last_became_peered":"2023-08-25T08:57:19.518202+0000","last_unstale":"2023-08-25T08:57:21.577622+0000","last_undegraded":"2023-08-25T08:57:21.577622+0000","last_fullsized":"2023-08-25T08:57:21.577622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:00.267573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509145+0000","last_change":"2023-08-25T08:57:18.718657+0000","last_active":"2023-08-25T08:57:20.509145+0000","last_peered":"2023-08-25T08:57:20.509145+0000","last_clean":"2023-08-25T08:57:20.509145+0000","last_became_active":"2023-08-25T08:57:18.718348+0000","last_became_peered":"2023-08-25T08:57:18.718348+0000","last_unstale":"2023-08-25T08:57:20.509145+0000","last_undegraded":"2023-08-25T08:57:20.509145+0000","last_fullsized":"2023-08-25T08:57:20.509145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:24:14.753607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508148+0000","last_change":"2023-08-25T08:57:19.520997+0000","last_active":"2023-08-25T08:57:20.508148+0000","last_peered":"2023-08-25T08:57:20.508148+0000","last_clean":"2023-08-25T08:57:20.508148+0000","last_became_active":"2023-08-25T08:57:19.520695+0000","last_became_peered":"2023-08-25T08:57:19.520695+0000","last_unstale":"2023-08-25T08:57:20.508148+0000","last_undegraded":"2023-08-25T08:57:20.508148+0000","last_fullsized":"2023-08-25T08:57:20.508148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:10.661689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.498583+0000","last_change":"2023-08-25T08:57:18.714573+0000","last_active":"2023-08-25T08:57:21.498583+0000","last_peered":"2023-08-25T08:57:21.498583+0000","last_clean":"2023-08-25T08:57:21.498583+0000","last_became_active":"2023-08-25T08:57:18.714268+0000","last_became_peered":"2023-08-25T08:57:18.714268+0000","last_unstale":"2023-08-25T08:57:21.498583+0000","last_undegraded":"2023-08-25T08:57:21.498583+0000","last_fullsized":"2023-08-25T08:57:21.498583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:33.154816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577677+0000","last_change":"2023-08-25T08:56:14.362397+0000","last_active":"2023-08-25T08:57:21.577677+0000","last_peered":"2023-08-25T08:57:21.577677+0000","last_clean":"2023-08-25T08:57:21.577677+0000","last_became_active":"2023-08-25T08:56:14.361946+0000","last_became_peered":"2023-08-25T08:56:14.361946+0000","last_unstale":"2023-08-25T08:57:21.577677+0000","last_undegraded":"2023-08-25T08:57:21.577677+0000","last_fullsized":"2023-08-25T08:57:21.577677+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:53:09.022543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:53:09.022543+0000","last_clean_scrub_stamp":"2023-08-25T08:53:09.022543+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:21:51.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509008+0000","last_change":"2023-08-25T08:57:19.517547+0000","last_active":"2023-08-25T08:57:20.509008+0000","last_peered":"2023-08-25T08:57:20.509008+0000","last_clean":"2023-08-25T08:57:20.509008+0000","last_became_active":"2023-08-25T08:57:19.517221+0000","last_became_peered":"2023-08-25T08:57:19.517221+0000","last_unstale":"2023-08-25T08:57:20.509008+0000","last_undegraded":"2023-08-25T08:57:20.509008+0000","last_fullsized":"2023-08-25T08:57:20.509008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:45.891452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577918+0000","last_change":"2023-08-25T08:57:18.715769+0000","last_active":"2023-08-25T08:57:21.577918+0000","last_peered":"2023-08-25T08:57:21.577918+0000","last_clean":"2023-08-25T08:57:21.577918+0000","last_became_active":"2023-08-25T08:57:18.715399+0000","last_became_peered":"2023-08-25T08:57:18.715399+0000","last_unstale":"2023-08-25T08:57:21.577918+0000","last_undegraded":"2023-08-25T08:57:21.577918+0000","last_fullsized":"2023-08-25T08:57:21.577918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:39:15.306754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.063627+0000","last_change":"2023-08-25T08:57:18.720658+0000","last_active":"2023-08-25T08:57:23.063627+0000","last_peered":"2023-08-25T08:57:23.063627+0000","last_clean":"2023-08-25T08:57:23.063627+0000","last_became_active":"2023-08-25T08:57:18.720419+0000","last_became_peered":"2023-08-25T08:57:18.720419+0000","last_unstale":"2023-08-25T08:57:23.063627+0000","last_undegraded":"2023-08-25T08:57:23.063627+0000","last_fullsized":"2023-08-25T08:57:23.063627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:15.624027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509606+0000","last_change":"2023-08-25T08:57:19.513257+0000","last_active":"2023-08-25T08:57:20.509606+0000","last_peered":"2023-08-25T08:57:20.509606+0000","last_clean":"2023-08-25T08:57:20.509606+0000","last_became_active":"2023-08-25T08:57:19.512852+0000","last_became_peered":"2023-08-25T08:57:19.512852+0000","last_unstale":"2023-08-25T08:57:20.509606+0000","last_undegraded":"2023-08-25T08:57:20.509606+0000","last_fullsized":"2023-08-25T08:57:20.509606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:29:24.814590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.039988+0000","last_change":"2023-08-25T08:57:18.718740+0000","last_active":"2023-08-25T08:57:24.039988+0000","last_peered":"2023-08-25T08:57:24.039988+0000","last_clean":"2023-08-25T08:57:24.039988+0000","last_became_active":"2023-08-25T08:57:18.718433+0000","last_became_peered":"2023-08-25T08:57:18.718433+0000","last_unstale":"2023-08-25T08:57:24.039988+0000","last_undegraded":"2023-08-25T08:57:24.039988+0000","last_fullsized":"2023-08-25T08:57:24.039988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:55:15.935758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508534+0000","last_change":"2023-08-25T08:57:19.525867+0000","last_active":"2023-08-25T08:57:20.508534+0000","last_peered":"2023-08-25T08:57:20.508534+0000","last_clean":"2023-08-25T08:57:20.508534+0000","last_became_active":"2023-08-25T08:57:19.525492+0000","last_became_peered":"2023-08-25T08:57:19.525492+0000","last_unstale":"2023-08-25T08:57:20.508534+0000","last_undegraded":"2023-08-25T08:57:20.508534+0000","last_fullsized":"2023-08-25T08:57:20.508534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:53.064700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.498588+0000","last_change":"2023-08-25T08:57:18.711447+0000","last_active":"2023-08-25T08:57:21.498588+0000","last_peered":"2023-08-25T08:57:21.498588+0000","last_clean":"2023-08-25T08:57:21.498588+0000","last_became_active":"2023-08-25T08:57:18.711164+0000","last_became_peered":"2023-08-25T08:57:18.711164+0000","last_unstale":"2023-08-25T08:57:21.498588+0000","last_undegraded":"2023-08-25T08:57:21.498588+0000","last_fullsized":"2023-08-25T08:57:21.498588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:27:12.390451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577359+0000","last_change":"2023-08-25T08:57:19.528927+0000","last_active":"2023-08-25T08:57:21.577359+0000","last_peered":"2023-08-25T08:57:21.577359+0000","last_clean":"2023-08-25T08:57:21.577359+0000","last_became_active":"2023-08-25T08:57:19.528281+0000","last_became_peered":"2023-08-25T08:57:19.528281+0000","last_unstale":"2023-08-25T08:57:21.577359+0000","last_undegraded":"2023-08-25T08:57:21.577359+0000","last_fullsized":"2023-08-25T08:57:21.577359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:03.091434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.063187+0000","last_change":"2023-08-25T08:57:18.713474+0000","last_active":"2023-08-25T08:57:23.063187+0000","last_peered":"2023-08-25T08:57:23.063187+0000","last_clean":"2023-08-25T08:57:23.063187+0000","last_became_active":"2023-08-25T08:57:18.713147+0000","last_became_peered":"2023-08-25T08:57:18.713147+0000","last_unstale":"2023-08-25T08:57:23.063187+0000","last_undegraded":"2023-08-25T08:57:23.063187+0000","last_fullsized":"2023-08-25T08:57:23.063187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:55.688173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.861772+0000","last_change":"2023-08-25T08:57:19.515035+0000","last_active":"2023-08-25T08:57:20.861772+0000","last_peered":"2023-08-25T08:57:20.861772+0000","last_clean":"2023-08-25T08:57:20.861772+0000","last_became_active":"2023-08-25T08:57:19.514729+0000","last_became_peered":"2023-08-25T08:57:19.514729+0000","last_unstale":"2023-08-25T08:57:20.861772+0000","last_undegraded":"2023-08-25T08:57:20.861772+0000","last_fullsized":"2023-08-25T08:57:20.861772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:18.522519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.061221+0000","last_change":"2023-08-25T08:57:18.721451+0000","last_active":"2023-08-25T08:57:23.061221+0000","last_peered":"2023-08-25T08:57:23.061221+0000","last_clean":"2023-08-25T08:57:23.061221+0000","last_became_active":"2023-08-25T08:57:18.721132+0000","last_became_peered":"2023-08-25T08:57:18.721132+0000","last_unstale":"2023-08-25T08:57:23.061221+0000","last_undegraded":"2023-08-25T08:57:23.061221+0000","last_fullsized":"2023-08-25T08:57:23.061221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:48.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920596+0000","last_change":"2023-08-25T08:57:19.519649+0000","last_active":"2023-08-25T08:57:21.920596+0000","last_peered":"2023-08-25T08:57:21.920596+0000","last_clean":"2023-08-25T08:57:21.920596+0000","last_became_active":"2023-08-25T08:57:19.518907+0000","last_became_peered":"2023-08-25T08:57:19.518907+0000","last_unstale":"2023-08-25T08:57:21.920596+0000","last_undegraded":"2023-08-25T08:57:21.920596+0000","last_fullsized":"2023-08-25T08:57:21.920596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:32:55.986526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506545+0000","last_change":"2023-08-25T08:57:18.722996+0000","last_active":"2023-08-25T08:57:20.506545+0000","last_peered":"2023-08-25T08:57:20.506545+0000","last_clean":"2023-08-25T08:57:20.506545+0000","last_became_active":"2023-08-25T08:57:18.722662+0000","last_became_peered":"2023-08-25T08:57:18.722662+0000","last_unstale":"2023-08-25T08:57:20.506545+0000","last_undegraded":"2023-08-25T08:57:20.506545+0000","last_fullsized":"2023-08-25T08:57:20.506545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:30:17.722094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.507862+0000","last_change":"2023-08-25T08:57:19.529138+0000","last_active":"2023-08-25T08:57:20.507862+0000","last_peered":"2023-08-25T08:57:20.507862+0000","last_clean":"2023-08-25T08:57:20.507862+0000","last_became_active":"2023-08-25T08:57:19.527485+0000","last_became_peered":"2023-08-25T08:57:19.527485+0000","last_unstale":"2023-08-25T08:57:20.507862+0000","last_undegraded":"2023-08-25T08:57:20.507862+0000","last_fullsized":"2023-08-25T08:57:20.507862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:51.151079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.064239+0000","last_change":"2023-08-25T08:57:18.717811+0000","last_active":"2023-08-25T08:57:23.064239+0000","last_peered":"2023-08-25T08:57:23.064239+0000","last_clean":"2023-08-25T08:57:23.064239+0000","last_became_active":"2023-08-25T08:57:18.717573+0000","last_became_peered":"2023-08-25T08:57:18.717573+0000","last_unstale":"2023-08-25T08:57:23.064239+0000","last_undegraded":"2023-08-25T08:57:23.064239+0000","last_fullsized":"2023-08-25T08:57:23.064239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:13:37.354858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578921+0000","last_change":"2023-08-25T08:57:19.528271+0000","last_active":"2023-08-25T08:57:21.578921+0000","last_peered":"2023-08-25T08:57:21.578921+0000","last_clean":"2023-08-25T08:57:21.578921+0000","last_became_active":"2023-08-25T08:57:19.528017+0000","last_became_peered":"2023-08-25T08:57:19.528017+0000","last_unstale":"2023-08-25T08:57:21.578921+0000","last_undegraded":"2023-08-25T08:57:21.578921+0000","last_fullsized":"2023-08-25T08:57:21.578921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:01:34.370416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506812+0000","last_change":"2023-08-25T08:57:18.715063+0000","last_active":"2023-08-25T08:57:20.506812+0000","last_peered":"2023-08-25T08:57:20.506812+0000","last_clean":"2023-08-25T08:57:20.506812+0000","last_became_active":"2023-08-25T08:57:18.714755+0000","last_became_peered":"2023-08-25T08:57:18.714755+0000","last_unstale":"2023-08-25T08:57:20.506812+0000","last_undegraded":"2023-08-25T08:57:20.506812+0000","last_fullsized":"2023-08-25T08:57:20.506812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:08:26.104256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578831+0000","last_change":"2023-08-25T08:57:19.528557+0000","last_active":"2023-08-25T08:57:21.578831+0000","last_peered":"2023-08-25T08:57:21.578831+0000","last_clean":"2023-08-25T08:57:21.578831+0000","last_became_active":"2023-08-25T08:57:19.528284+0000","last_became_peered":"2023-08-25T08:57:19.528284+0000","last_unstale":"2023-08-25T08:57:21.578831+0000","last_undegraded":"2023-08-25T08:57:21.578831+0000","last_fullsized":"2023-08-25T08:57:21.578831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:52:50.854609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577795+0000","last_change":"2023-08-25T08:57:18.722893+0000","last_active":"2023-08-25T08:57:21.577795+0000","last_peered":"2023-08-25T08:57:21.577795+0000","last_clean":"2023-08-25T08:57:21.577795+0000","last_became_active":"2023-08-25T08:57:18.722553+0000","last_became_peered":"2023-08-25T08:57:18.722553+0000","last_unstale":"2023-08-25T08:57:21.577795+0000","last_undegraded":"2023-08-25T08:57:21.577795+0000","last_fullsized":"2023-08-25T08:57:21.577795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:25:00.205900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578471+0000","last_change":"2023-08-25T08:57:19.517975+0000","last_active":"2023-08-25T08:57:21.578471+0000","last_peered":"2023-08-25T08:57:21.578471+0000","last_clean":"2023-08-25T08:57:21.578471+0000","last_became_active":"2023-08-25T08:57:19.517715+0000","last_became_peered":"2023-08-25T08:57:19.517715+0000","last_unstale":"2023-08-25T08:57:21.578471+0000","last_undegraded":"2023-08-25T08:57:21.578471+0000","last_fullsized":"2023-08-25T08:57:21.578471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:26:59.048019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508997+0000","last_change":"2023-08-25T08:57:18.717331+0000","last_active":"2023-08-25T08:57:20.508997+0000","last_peered":"2023-08-25T08:57:20.508997+0000","last_clean":"2023-08-25T08:57:20.508997+0000","last_became_active":"2023-08-25T08:57:18.716957+0000","last_became_peered":"2023-08-25T08:57:18.716957+0000","last_unstale":"2023-08-25T08:57:20.508997+0000","last_undegraded":"2023-08-25T08:57:20.508997+0000","last_fullsized":"2023-08-25T08:57:20.508997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:21.300644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578015+0000","last_change":"2023-08-25T08:57:19.515217+0000","last_active":"2023-08-25T08:57:21.578015+0000","last_peered":"2023-08-25T08:57:21.578015+0000","last_clean":"2023-08-25T08:57:21.578015+0000","last_became_active":"2023-08-25T08:57:19.514870+0000","last_became_peered":"2023-08-25T08:57:19.514870+0000","last_unstale":"2023-08-25T08:57:21.578015+0000","last_undegraded":"2023-08-25T08:57:21.578015+0000","last_fullsized":"2023-08-25T08:57:21.578015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:17:43.197478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920528+0000","last_change":"2023-08-25T08:57:19.512984+0000","last_active":"2023-08-25T08:57:21.920528+0000","last_peered":"2023-08-25T08:57:21.920528+0000","last_clean":"2023-08-25T08:57:21.920528+0000","last_became_active":"2023-08-25T08:57:19.512644+0000","last_became_peered":"2023-08-25T08:57:19.512644+0000","last_unstale":"2023-08-25T08:57:21.920528+0000","last_undegraded":"2023-08-25T08:57:21.920528+0000","last_fullsized":"2023-08-25T08:57:21.920528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:04:51.059464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506789+0000","last_change":"2023-08-25T08:57:19.521540+0000","last_active":"2023-08-25T08:57:20.506789+0000","last_peered":"2023-08-25T08:57:20.506789+0000","last_clean":"2023-08-25T08:57:20.506789+0000","last_became_active":"2023-08-25T08:57:19.521305+0000","last_became_peered":"2023-08-25T08:57:19.521305+0000","last_unstale":"2023-08-25T08:57:20.506789+0000","last_undegraded":"2023-08-25T08:57:20.506789+0000","last_fullsized":"2023-08-25T08:57:20.506789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:44:48.837529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506686+0000","last_change":"2023-08-25T08:57:19.526830+0000","last_active":"2023-08-25T08:57:20.506686+0000","last_peered":"2023-08-25T08:57:20.506686+0000","last_clean":"2023-08-25T08:57:20.506686+0000","last_became_active":"2023-08-25T08:57:19.526396+0000","last_became_peered":"2023-08-25T08:57:19.526396+0000","last_unstale":"2023-08-25T08:57:20.506686+0000","last_undegraded":"2023-08-25T08:57:20.506686+0000","last_fullsized":"2023-08-25T08:57:20.506686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:48.876189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577326+0000","last_change":"2023-08-25T08:57:19.522557+0000","last_active":"2023-08-25T08:57:21.577326+0000","last_peered":"2023-08-25T08:57:21.577326+0000","last_clean":"2023-08-25T08:57:21.577326+0000","last_became_active":"2023-08-25T08:57:19.522076+0000","last_became_peered":"2023-08-25T08:57:19.522076+0000","last_unstale":"2023-08-25T08:57:21.577326+0000","last_undegraded":"2023-08-25T08:57:21.577326+0000","last_fullsized":"2023-08-25T08:57:21.577326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:13.619576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509444+0000","last_change":"2023-08-25T08:57:19.518894+0000","last_active":"2023-08-25T08:57:20.509444+0000","last_peered":"2023-08-25T08:57:20.509444+0000","last_clean":"2023-08-25T08:57:20.509444+0000","last_became_active":"2023-08-25T08:57:19.518599+0000","last_became_peered":"2023-08-25T08:57:19.518599+0000","last_unstale":"2023-08-25T08:57:20.509444+0000","last_undegraded":"2023-08-25T08:57:20.509444+0000","last_fullsized":"2023-08-25T08:57:20.509444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:14.148977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920810+0000","last_change":"2023-08-25T08:57:19.513584+0000","last_active":"2023-08-25T08:57:21.920810+0000","last_peered":"2023-08-25T08:57:21.920810+0000","last_clean":"2023-08-25T08:57:21.920810+0000","last_became_active":"2023-08-25T08:57:19.513187+0000","last_became_peered":"2023-08-25T08:57:19.513187+0000","last_unstale":"2023-08-25T08:57:21.920810+0000","last_undegraded":"2023-08-25T08:57:21.920810+0000","last_fullsized":"2023-08-25T08:57:21.920810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:27.836012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506864+0000","last_change":"2023-08-25T08:57:19.528534+0000","last_active":"2023-08-25T08:57:20.506864+0000","last_peered":"2023-08-25T08:57:20.506864+0000","last_clean":"2023-08-25T08:57:20.506864+0000","last_became_active":"2023-08-25T08:57:19.528186+0000","last_became_peered":"2023-08-25T08:57:19.528186+0000","last_unstale":"2023-08-25T08:57:20.506864+0000","last_undegraded":"2023-08-25T08:57:20.506864+0000","last_fullsized":"2023-08-25T08:57:20.506864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:45.431720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508530+0000","last_change":"2023-08-25T08:57:19.529932+0000","last_active":"2023-08-25T08:57:20.508530+0000","last_peered":"2023-08-25T08:57:20.508530+0000","last_clean":"2023-08-25T08:57:20.508530+0000","last_became_active":"2023-08-25T08:57:19.529581+0000","last_became_peered":"2023-08-25T08:57:19.529581+0000","last_unstale":"2023-08-25T08:57:20.508530+0000","last_undegraded":"2023-08-25T08:57:20.508530+0000","last_fullsized":"2023-08-25T08:57:20.508530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:10.394555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577532+0000","last_change":"2023-08-25T08:57:19.527777+0000","last_active":"2023-08-25T08:57:21.577532+0000","last_peered":"2023-08-25T08:57:21.577532+0000","last_clean":"2023-08-25T08:57:21.577532+0000","last_became_active":"2023-08-25T08:57:19.527515+0000","last_became_peered":"2023-08-25T08:57:19.527515+0000","last_unstale":"2023-08-25T08:57:21.577532+0000","last_undegraded":"2023-08-25T08:57:21.577532+0000","last_fullsized":"2023-08-25T08:57:21.577532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:24.616320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.507485+0000","last_change":"2023-08-25T08:57:19.522676+0000","last_active":"2023-08-25T08:57:20.507485+0000","last_peered":"2023-08-25T08:57:20.507485+0000","last_clean":"2023-08-25T08:57:20.507485+0000","last_became_active":"2023-08-25T08:57:19.522282+0000","last_became_peered":"2023-08-25T08:57:19.522282+0000","last_unstale":"2023-08-25T08:57:20.507485+0000","last_undegraded":"2023-08-25T08:57:20.507485+0000","last_fullsized":"2023-08-25T08:57:20.507485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:27:01.305079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577476+0000","last_change":"2023-08-25T08:57:19.525175+0000","last_active":"2023-08-25T08:57:21.577476+0000","last_peered":"2023-08-25T08:57:21.577476+0000","last_clean":"2023-08-25T08:57:21.577476+0000","last_became_active":"2023-08-25T08:57:19.524769+0000","last_became_peered":"2023-08-25T08:57:19.524769+0000","last_unstale":"2023-08-25T08:57:21.577476+0000","last_undegraded":"2023-08-25T08:57:21.577476+0000","last_fullsized":"2023-08-25T08:57:21.577476+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:07:03.279154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509572+0000","last_change":"2023-08-25T08:57:19.518697+0000","last_active":"2023-08-25T08:57:20.509572+0000","last_peered":"2023-08-25T08:57:20.509572+0000","last_clean":"2023-08-25T08:57:20.509572+0000","last_became_active":"2023-08-25T08:57:19.518316+0000","last_became_peered":"2023-08-25T08:57:19.518316+0000","last_unstale":"2023-08-25T08:57:20.509572+0000","last_undegraded":"2023-08-25T08:57:20.509572+0000","last_fullsized":"2023-08-25T08:57:20.509572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:21:41.551008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920593+0000","last_change":"2023-08-25T08:57:19.515293+0000","last_active":"2023-08-25T08:57:21.920593+0000","last_peered":"2023-08-25T08:57:21.920593+0000","last_clean":"2023-08-25T08:57:21.920593+0000","last_became_active":"2023-08-25T08:57:19.514883+0000","last_became_peered":"2023-08-25T08:57:19.514883+0000","last_unstale":"2023-08-25T08:57:21.920593+0000","last_undegraded":"2023-08-25T08:57:21.920593+0000","last_fullsized":"2023-08-25T08:57:21.920593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:50:41.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250603,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":20089,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":94,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":27151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":417248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1036864,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.436}]},{"osd":2,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.753},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.627}]},{"osd":3,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.373},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.36199999999999999}]},{"osd":8,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.78800000000000003}]},{"osd":9,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":426636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.66000000000000003}]},{"osd":8,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.63},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.60299999999999998}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66700000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103194,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":426444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.749}]},{"osd":1,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.503}]},{"osd":3,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80700000000000005}]},{"osd":9,"last update":"Fri Aug 25 08:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.89700000000000002}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.875}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332125,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1017390,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"min":{"1min":0.32700000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60799999999999998,"5min":0.628,"15min":0.628},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.46200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.82599999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.496},"min":{"1min":0.38800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.503},{"interface":"front","average":{"1min":0.497,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51100000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86199999999999999,"5min":0.879,"15min":0.879},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.88}]},{"osd":6,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81699999999999995}]},{"osd":8,"last update":"Fri Aug 25 08:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.89900000000000002}]},{"osd":9,"last update":"Fri Aug 25 08:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79700000000000004}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":425818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.441,"15min":0.432},"min":{"1min":0.34999999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"min":{"1min":0.40999999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.5}]},{"osd":1,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.70699999999999996,"15min":0.71799999999999997},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84799999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.101,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.57099999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.42299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.46999999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.55200000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90800000000000003,"5min":3.9500000000000002,"15min":3.9500000000000002},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.51000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.64200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Fri Aug 25 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73299999999999998,"15min":0.71899999999999997},"min":{"1min":0.60199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.51000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri Aug 25 08:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.78300000000000003}]},{"osd":9,"last update":"Fri Aug 25 08:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.75700000000000001}]},{"osd":10,"last update":"Fri Aug 25 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.97999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":427157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.33800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.626},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61699999999999999,"15min":0.56299999999999994},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81399999999999995,"5min":5.4359999999999999,"15min":5.4359999999999999},"last":0.747}]},{"osd":2,"last update":"Fri Aug 25 08:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.441,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Aug 25 08:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85199999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.46899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83399999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.42199999999999999,"15min":0.40200000000000002},"min":{"1min":0.29299999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.44700000000000001,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42999999999999999}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri Aug 25 08:56:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86399999999999999,"5min":1.107,"15min":1.107},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":1.153,"15min":1.153},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.497},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.47199999999999998}]},{"osd":8,"last update":"Fri Aug 25 08:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.71599999999999997}]},{"osd":9,"last update":"Fri Aug 25 08:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.76600000000000001}]},{"osd":10,"last update":"Fri Aug 25 08:57:13 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.83099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":425908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58299999999999996,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Aug 25 08:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.46100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":1.1919999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60999999999999999,"15min":0.56399999999999995},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Aug 25 08:55:37 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":1.157},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.46300000000000002,"15min":0.45000000000000001},"min":{"1min":0.34999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Aug 25 08:57:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.66000000000000003,"15min":0.62},"min":{"1min":0.47499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.629,"15min":0.60199999999999998},"min":{"1min":0.55400000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.2250000000000001}]},{"osd":5,"last update":"Fri Aug 25 08:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.107}]},{"osd":6,"last update":"Fri Aug 25 08:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.67900000000000005,"15min":0.65900000000000003},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.626}]},{"osd":7,"last update":"Fri Aug 25 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.77000000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68300000000000005}]},{"osd":8,"last update":"Fri Aug 25 08:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.70899999999999996}]},{"osd":9,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.407},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Aug 25 08:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.749},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.597}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":427464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79700000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":1.409,"15min":1.409},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.41699999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Aug 25 08:56:42 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.442},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Aug 25 08:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42899999999999999,"15min":0.42099999999999999},"min":{"1min":0.27100000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.376},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.55500000000000005}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91800000000000004,"5min":1.117,"15min":1.117},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":1.097,"15min":1.097},"last":0.64500000000000002}]},{"osd":8,"last update":"Fri Aug 25 08:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.505}]},{"osd":9,"last update":"Fri Aug 25 08:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.61899999999999999}]},{"osd":10,"last update":"Fri Aug 25 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.54800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":426102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41199999999999998,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.40699999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Aug 25 08:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Aug 25 08:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84899999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Aug 25 08:57:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.51400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58799999999999997},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.52900000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Aug 25 08:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.05,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.625}]},{"osd":7,"last update":"Fri Aug 25 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.56799999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.97899999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98399999999999999,"5min":1.161,"15min":1.161},"last":0.81100000000000005}]},{"osd":8,"last update":"Fri Aug 25 08:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85199999999999998}]},{"osd":9,"last update":"Fri Aug 25 08:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.95499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":425840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.438}]},{"osd":1,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.40200000000000002},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.41499999999999998,"15min":0.40699999999999997},"min":{"1min":0.33500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69499999999999995}]},{"osd":3,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.38700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77400000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60099999999999998,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.66700000000000004,"15min":0.65500000000000003},"min":{"1min":0.54100000000000004,"5min":0.433,"15min":0.433},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Aug 25 08:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.92200000000000004}]},{"osd":8,"last update":"Fri Aug 25 08:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.997}]},{"osd":9,"last update":"Fri Aug 25 08:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.96399999999999997}]},{"osd":10,"last update":"Fri Aug 25 08:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":1.032}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757288,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":1016037,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45300000000000001,"15min":0.47099999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67900000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.46700000000000003,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.495}]},{"osd":3,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.745},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61799999999999999,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Fri Aug 25 08:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95099999999999996,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71299999999999997,"15min":0.71199999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.04,"5min":1.075,"15min":1.075},"last":0.79700000000000004}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.877}]},{"osd":8,"last update":"Fri Aug 25 08:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.85999999999999999}]},{"osd":9,"last update":"Fri Aug 25 08:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.93000000000000005}]},{"osd":10,"last update":"Fri Aug 25 08:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.90400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34629,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34629,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":20555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T08:57:28.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T08:57:28.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:28 smithi157 ceph-mon[130879]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:28.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:28 smithi157 ceph-mon[130879]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-08-25T08:57:28.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:28 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/271461051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:28.383 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:28 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3049367416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:28.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:28 smithi181 ceph-mon[131296]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:57:28.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:28 smithi181 ceph-mon[131296]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-08-25T08:57:28.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:28 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/271461051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:57:28.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:28 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3049367416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.598+0000 7fe4f7bff700 1 -- 172.21.15.99:0/2148615713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0101b10 msgr2=0x7fe4f0101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.598+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/2148615713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0101b10 0x7fe4f0101f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00099c0 tx=0x7fe4e0009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.599+0000 7fe4f7bff700 1 -- 172.21.15.99:0/2148615713 shutdown_connections 2023-08-25T08:57:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.599+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/2148615713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f01024b0 0x7fe4f0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.599+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/2148615713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f0100920 0x7fe4f0100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.599+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/2148615713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0101b10 0x7fe4f0101f70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.599+0000 7fe4f7bff700 1 -- 172.21.15.99:0/2148615713 >> 172.21.15.99:0/2148615713 conn(0x7fe4f00fbe50 msgr2=0x7fe4f00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.599+0000 7fe4f7bff700 1 -- 172.21.15.99:0/2148615713 shutdown_connections 2023-08-25T08:57:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.599+0000 7fe4f7bff700 1 -- 172.21.15.99:0/2148615713 wait complete. 2023-08-25T08:57:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f7bff700 1 Processor -- start 2023-08-25T08:57:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f7bff700 1 -- start start 2023-08-25T08:57:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f7bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f0100920 0x7fe4f01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f7bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f0101b10 0x7fe4f0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f7bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f01024b0 0x7fe4f019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f7bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a0330 con 0x7fe4f01024b0 2023-08-25T08:57:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f7bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a0470 con 0x7fe4f0101b10 2023-08-25T08:57:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f7bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a0600 con 0x7fe4f0100920 2023-08-25T08:57:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f599b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f0100920 0x7fe4f01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f599b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f0100920 0x7fe4f01987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:60348/0 (socket says 172.21.15.99:60348) 2023-08-25T08:57:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f599b700 1 -- 172.21.15.99:0/1108534358 learned_addr learned my addr 172.21.15.99:0/1108534358 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.600+0000 7fe4f619c700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f01024b0 0x7fe4f019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4f519a700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f0101b10 0x7fe4f0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4f599b700 1 -- 172.21.15.99:0/1108534358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f0101b10 msgr2=0x7fe4f0198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4f599b700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f0101b10 0x7fe4f0198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4f599b700 1 -- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f01024b0 msgr2=0x7fe4f019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4f599b700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f01024b0 0x7fe4f019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4f599b700 1 -- 172.21.15.99:0/1108534358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00095f0 con 0x7fe4f0100920 2023-08-25T08:57:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4f599b700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f0100920 0x7fe4f01987b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e970 tx=0x7fe4ec00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4e6ffd700 1 -- 172.21.15.99:0/1108534358 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec009790 con 0x7fe4f0100920 2023-08-25T08:57:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4e6ffd700 1 -- 172.21.15.99:0/1108534358 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4ec00c7e0 con 0x7fe4f0100920 2023-08-25T08:57:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f01a08e0 con 0x7fe4f0100920 2023-08-25T08:57:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.601+0000 7fe4e6ffd700 1 -- 172.21.15.99:0/1108534358 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec003c70 con 0x7fe4f0100920 2023-08-25T08:57:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.602+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01a0df0 con 0x7fe4f0100920 2023-08-25T08:57:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.603+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0192f20 con 0x7fe4f0100920 2023-08-25T08:57:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.604+0000 7fe4e6ffd700 1 -- 172.21.15.99:0/1108534358 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fe4ec00cc50 con 0x7fe4f0100920 2023-08-25T08:57:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.606+0000 7fe4e6ffd700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe4dc0a0020 0x7fe4dc0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.606+0000 7fe4e6ffd700 1 -- 172.21.15.99:0/1108534358 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe4ec014070 con 0x7fe4f0100920 2023-08-25T08:57:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.606+0000 7fe4f519a700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe4dc0a0020 0x7fe4dc0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.606+0000 7fe4f519a700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe4dc0a0020 0x7fe4dc0a24e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000b580 tx=0x7fe4e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.608+0000 7fe4e6ffd700 1 -- 172.21.15.99:0/1108534358 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fe4ec095310 con 0x7fe4f0100920 2023-08-25T08:57:28.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.805+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4f002ce80 con 0x7fe4dc0a0020 2023-08-25T08:57:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.816+0000 7fe4e6ffd700 1 -- 172.21.15.99:0/1108534358 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335653 (secure 0 0 0) 0x7fe4f002ce80 con 0x7fe4dc0a0020 2023-08-25T08:57:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe4dc0a0020 msgr2=0x7fe4dc0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe4dc0a0020 0x7fe4dc0a24e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000b580 tx=0x7fe4e0005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f0100920 msgr2=0x7fe4f01987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f0100920 0x7fe4f01987b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e970 tx=0x7fe4ec00ec80 comp rx=0 tx=0).stop 2023-08-25T08:57:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 shutdown_connections 2023-08-25T08:57:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f01024b0 0x7fe4f019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f0100920 0x7fe4f01987b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe4dc0a0020 0x7fe4dc0a24e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 --2- 172.21.15.99:0/1108534358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4f0101b10 0x7fe4f0198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.820+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 >> 172.21.15.99:0/1108534358 conn(0x7fe4f00fbe50 msgr2=0x7fe4f01073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.821+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 shutdown_connections 2023-08-25T08:57:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:28.821+0000 7fe4f7bff700 1 -- 172.21.15.99:0/1108534358 wait complete. 2023-08-25T08:57:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-08-25T08:57:28.840 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2023-08-25T08:57:28.437557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613378,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":169,"num_read_kb":143,"num_write":303,"num_write_kb":2995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":245,"ondisk_log_size":245,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335048,"kb_used_data":10832,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606776,"statfs":{"total":1151940427776,"available":1151597338624,"internally_reserved":0,"allocated":11091968,"data_stored":6948256,"data_compressed":108807,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23010,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":122,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.936109"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920834+0000","last_change":"2023-08-25T08:57:19.519342+0000","last_active":"2023-08-25T08:57:21.920834+0000","last_peered":"2023-08-25T08:57:21.920834+0000","last_clean":"2023-08-25T08:57:21.920834+0000","last_became_active":"2023-08-25T08:57:19.519006+0000","last_became_peered":"2023-08-25T08:57:19.519006+0000","last_unstale":"2023-08-25T08:57:21.920834+0000","last_undegraded":"2023-08-25T08:57:21.920834+0000","last_fullsized":"2023-08-25T08:57:21.920834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:32.976833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577376+0000","last_change":"2023-08-25T08:57:19.521779+0000","last_active":"2023-08-25T08:57:21.577376+0000","last_peered":"2023-08-25T08:57:21.577376+0000","last_clean":"2023-08-25T08:57:21.577376+0000","last_became_active":"2023-08-25T08:57:19.521358+0000","last_became_peered":"2023-08-25T08:57:19.521358+0000","last_unstale":"2023-08-25T08:57:21.577376+0000","last_undegraded":"2023-08-25T08:57:21.577376+0000","last_fullsized":"2023-08-25T08:57:21.577376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:23:27.999946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577350+0000","last_change":"2023-08-25T08:57:19.531553+0000","last_active":"2023-08-25T08:57:21.577350+0000","last_peered":"2023-08-25T08:57:21.577350+0000","last_clean":"2023-08-25T08:57:21.577350+0000","last_became_active":"2023-08-25T08:57:19.531113+0000","last_became_peered":"2023-08-25T08:57:19.531113+0000","last_unstale":"2023-08-25T08:57:21.577350+0000","last_undegraded":"2023-08-25T08:57:21.577350+0000","last_fullsized":"2023-08-25T08:57:21.577350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:57.221411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.498448+0000","last_change":"2023-08-25T08:57:19.519696+0000","last_active":"2023-08-25T08:57:21.498448+0000","last_peered":"2023-08-25T08:57:21.498448+0000","last_clean":"2023-08-25T08:57:21.498448+0000","last_became_active":"2023-08-25T08:57:19.518918+0000","last_became_peered":"2023-08-25T08:57:19.518918+0000","last_unstale":"2023-08-25T08:57:21.498448+0000","last_undegraded":"2023-08-25T08:57:21.498448+0000","last_fullsized":"2023-08-25T08:57:21.498448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:44:46.112412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.861989+0000","last_change":"2023-08-25T08:57:19.520706+0000","last_active":"2023-08-25T08:57:20.861989+0000","last_peered":"2023-08-25T08:57:20.861989+0000","last_clean":"2023-08-25T08:57:20.861989+0000","last_became_active":"2023-08-25T08:57:19.520333+0000","last_became_peered":"2023-08-25T08:57:19.520333+0000","last_unstale":"2023-08-25T08:57:20.861989+0000","last_undegraded":"2023-08-25T08:57:20.861989+0000","last_fullsized":"2023-08-25T08:57:20.861989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:25.722869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577779+0000","last_change":"2023-08-25T08:57:19.520040+0000","last_active":"2023-08-25T08:57:21.577779+0000","last_peered":"2023-08-25T08:57:21.577779+0000","last_clean":"2023-08-25T08:57:21.577779+0000","last_became_active":"2023-08-25T08:57:19.519726+0000","last_became_peered":"2023-08-25T08:57:19.519726+0000","last_unstale":"2023-08-25T08:57:21.577779+0000","last_undegraded":"2023-08-25T08:57:21.577779+0000","last_fullsized":"2023-08-25T08:57:21.577779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:36.003956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578052+0000","last_change":"2023-08-25T08:57:19.518064+0000","last_active":"2023-08-25T08:57:21.578052+0000","last_peered":"2023-08-25T08:57:21.578052+0000","last_clean":"2023-08-25T08:57:21.578052+0000","last_became_active":"2023-08-25T08:57:19.517725+0000","last_became_peered":"2023-08-25T08:57:19.517725+0000","last_unstale":"2023-08-25T08:57:21.578052+0000","last_undegraded":"2023-08-25T08:57:21.578052+0000","last_fullsized":"2023-08-25T08:57:21.578052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:15:39.794457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.507687+0000","last_change":"2023-08-25T08:57:19.527429+0000","last_active":"2023-08-25T08:57:20.507687+0000","last_peered":"2023-08-25T08:57:20.507687+0000","last_clean":"2023-08-25T08:57:20.507687+0000","last_became_active":"2023-08-25T08:57:19.526370+0000","last_became_peered":"2023-08-25T08:57:19.526370+0000","last_unstale":"2023-08-25T08:57:20.507687+0000","last_undegraded":"2023-08-25T08:57:20.507687+0000","last_fullsized":"2023-08-25T08:57:20.507687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:05.241223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578087+0000","last_change":"2023-08-25T08:57:19.517136+0000","last_active":"2023-08-25T08:57:21.578087+0000","last_peered":"2023-08-25T08:57:21.578087+0000","last_clean":"2023-08-25T08:57:21.578087+0000","last_became_active":"2023-08-25T08:57:19.516852+0000","last_became_peered":"2023-08-25T08:57:19.516852+0000","last_unstale":"2023-08-25T08:57:21.578087+0000","last_undegraded":"2023-08-25T08:57:21.578087+0000","last_fullsized":"2023-08-25T08:57:21.578087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:38.248094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.579207+0000","last_change":"2023-08-25T08:57:19.522122+0000","last_active":"2023-08-25T08:57:21.579207+0000","last_peered":"2023-08-25T08:57:21.579207+0000","last_clean":"2023-08-25T08:57:21.579207+0000","last_became_active":"2023-08-25T08:57:19.520984+0000","last_became_peered":"2023-08-25T08:57:19.520984+0000","last_unstale":"2023-08-25T08:57:21.579207+0000","last_undegraded":"2023-08-25T08:57:21.579207+0000","last_fullsized":"2023-08-25T08:57:21.579207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:59.542512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578697+0000","last_change":"2023-08-25T08:57:19.531356+0000","last_active":"2023-08-25T08:57:21.578697+0000","last_peered":"2023-08-25T08:57:21.578697+0000","last_clean":"2023-08-25T08:57:21.578697+0000","last_became_active":"2023-08-25T08:57:19.531050+0000","last_became_peered":"2023-08-25T08:57:19.531050+0000","last_unstale":"2023-08-25T08:57:21.578697+0000","last_undegraded":"2023-08-25T08:57:21.578697+0000","last_fullsized":"2023-08-25T08:57:21.578697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:40:32.714726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.498416+0000","last_change":"2023-08-25T08:57:19.520091+0000","last_active":"2023-08-25T08:57:21.498416+0000","last_peered":"2023-08-25T08:57:21.498416+0000","last_clean":"2023-08-25T08:57:21.498416+0000","last_became_active":"2023-08-25T08:57:19.519356+0000","last_became_peered":"2023-08-25T08:57:19.519356+0000","last_unstale":"2023-08-25T08:57:21.498416+0000","last_undegraded":"2023-08-25T08:57:21.498416+0000","last_fullsized":"2023-08-25T08:57:21.498416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:12:05.753053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509577+0000","last_change":"2023-08-25T08:57:19.515446+0000","last_active":"2023-08-25T08:57:20.509577+0000","last_peered":"2023-08-25T08:57:20.509577+0000","last_clean":"2023-08-25T08:57:20.509577+0000","last_became_active":"2023-08-25T08:57:19.514376+0000","last_became_peered":"2023-08-25T08:57:19.514376+0000","last_unstale":"2023-08-25T08:57:20.509577+0000","last_undegraded":"2023-08-25T08:57:20.509577+0000","last_fullsized":"2023-08-25T08:57:20.509577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:23:44.549208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577354+0000","last_change":"2023-08-25T08:57:19.521629+0000","last_active":"2023-08-25T08:57:21.577354+0000","last_peered":"2023-08-25T08:57:21.577354+0000","last_clean":"2023-08-25T08:57:21.577354+0000","last_became_active":"2023-08-25T08:57:19.521307+0000","last_became_peered":"2023-08-25T08:57:19.521307+0000","last_unstale":"2023-08-25T08:57:21.577354+0000","last_undegraded":"2023-08-25T08:57:21.577354+0000","last_fullsized":"2023-08-25T08:57:21.577354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:19.552027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577515+0000","last_change":"2023-08-25T08:57:19.521670+0000","last_active":"2023-08-25T08:57:21.577515+0000","last_peered":"2023-08-25T08:57:21.577515+0000","last_clean":"2023-08-25T08:57:21.577515+0000","last_became_active":"2023-08-25T08:57:19.521325+0000","last_became_peered":"2023-08-25T08:57:19.521325+0000","last_unstale":"2023-08-25T08:57:21.577515+0000","last_undegraded":"2023-08-25T08:57:21.577515+0000","last_fullsized":"2023-08-25T08:57:21.577515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:24:27.589627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577208+0000","last_change":"2023-08-25T08:57:19.526733+0000","last_active":"2023-08-25T08:57:21.577208+0000","last_peered":"2023-08-25T08:57:21.577208+0000","last_clean":"2023-08-25T08:57:21.577208+0000","last_became_active":"2023-08-25T08:57:19.525692+0000","last_became_peered":"2023-08-25T08:57:19.525692+0000","last_unstale":"2023-08-25T08:57:21.577208+0000","last_undegraded":"2023-08-25T08:57:21.577208+0000","last_fullsized":"2023-08-25T08:57:21.577208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:58:39.725840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920886+0000","last_change":"2023-08-25T08:57:19.519289+0000","last_active":"2023-08-25T08:57:21.920886+0000","last_peered":"2023-08-25T08:57:21.920886+0000","last_clean":"2023-08-25T08:57:21.920886+0000","last_became_active":"2023-08-25T08:57:19.518584+0000","last_became_peered":"2023-08-25T08:57:19.518584+0000","last_unstale":"2023-08-25T08:57:21.920886+0000","last_undegraded":"2023-08-25T08:57:21.920886+0000","last_fullsized":"2023-08-25T08:57:21.920886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:56:36.031877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577761+0000","last_change":"2023-08-25T08:57:19.525090+0000","last_active":"2023-08-25T08:57:21.577761+0000","last_peered":"2023-08-25T08:57:21.577761+0000","last_clean":"2023-08-25T08:57:21.577761+0000","last_became_active":"2023-08-25T08:57:19.524543+0000","last_became_peered":"2023-08-25T08:57:19.524543+0000","last_unstale":"2023-08-25T08:57:21.577761+0000","last_undegraded":"2023-08-25T08:57:21.577761+0000","last_fullsized":"2023-08-25T08:57:21.577761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:41:42.783908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506702+0000","last_change":"2023-08-25T08:57:19.526761+0000","last_active":"2023-08-25T08:57:20.506702+0000","last_peered":"2023-08-25T08:57:20.506702+0000","last_clean":"2023-08-25T08:57:20.506702+0000","last_became_active":"2023-08-25T08:57:19.526421+0000","last_became_peered":"2023-08-25T08:57:19.526421+0000","last_unstale":"2023-08-25T08:57:20.506702+0000","last_undegraded":"2023-08-25T08:57:20.506702+0000","last_fullsized":"2023-08-25T08:57:20.506702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:30.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.041023+0000","last_change":"2023-08-25T08:57:18.717584+0000","last_active":"2023-08-25T08:57:24.041023+0000","last_peered":"2023-08-25T08:57:24.041023+0000","last_clean":"2023-08-25T08:57:24.041023+0000","last_became_active":"2023-08-25T08:57:18.717284+0000","last_became_peered":"2023-08-25T08:57:18.717284+0000","last_unstale":"2023-08-25T08:57:24.041023+0000","last_undegraded":"2023-08-25T08:57:24.041023+0000","last_fullsized":"2023-08-25T08:57:24.041023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:11:26.749184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508383+0000","last_change":"2023-08-25T08:57:19.527089+0000","last_active":"2023-08-25T08:57:20.508383+0000","last_peered":"2023-08-25T08:57:20.508383+0000","last_clean":"2023-08-25T08:57:20.508383+0000","last_became_active":"2023-08-25T08:57:19.526118+0000","last_became_peered":"2023-08-25T08:57:19.526118+0000","last_unstale":"2023-08-25T08:57:20.508383+0000","last_undegraded":"2023-08-25T08:57:20.508383+0000","last_fullsized":"2023-08-25T08:57:20.508383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:10:05.421256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508441+0000","last_change":"2023-08-25T08:57:18.720269+0000","last_active":"2023-08-25T08:57:20.508441+0000","last_peered":"2023-08-25T08:57:20.508441+0000","last_clean":"2023-08-25T08:57:20.508441+0000","last_became_active":"2023-08-25T08:57:18.719878+0000","last_became_peered":"2023-08-25T08:57:18.719878+0000","last_unstale":"2023-08-25T08:57:20.508441+0000","last_undegraded":"2023-08-25T08:57:20.508441+0000","last_fullsized":"2023-08-25T08:57:20.508441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:24:11.739039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577572+0000","last_change":"2023-08-25T08:57:19.531835+0000","last_active":"2023-08-25T08:57:21.577572+0000","last_peered":"2023-08-25T08:57:21.577572+0000","last_clean":"2023-08-25T08:57:21.577572+0000","last_became_active":"2023-08-25T08:57:19.531409+0000","last_became_peered":"2023-08-25T08:57:19.531409+0000","last_unstale":"2023-08-25T08:57:21.577572+0000","last_undegraded":"2023-08-25T08:57:21.577572+0000","last_fullsized":"2023-08-25T08:57:21.577572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:55:00.124607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.038085+0000","last_change":"2023-08-25T08:57:18.716847+0000","last_active":"2023-08-25T08:57:24.038085+0000","last_peered":"2023-08-25T08:57:24.038085+0000","last_clean":"2023-08-25T08:57:24.038085+0000","last_became_active":"2023-08-25T08:57:18.716416+0000","last_became_peered":"2023-08-25T08:57:18.716416+0000","last_unstale":"2023-08-25T08:57:24.038085+0000","last_undegraded":"2023-08-25T08:57:24.038085+0000","last_fullsized":"2023-08-25T08:57:24.038085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:13:17.069695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577237+0000","last_change":"2023-08-25T08:57:19.526495+0000","last_active":"2023-08-25T08:57:21.577237+0000","last_peered":"2023-08-25T08:57:21.577237+0000","last_clean":"2023-08-25T08:57:21.577237+0000","last_became_active":"2023-08-25T08:57:19.525204+0000","last_became_peered":"2023-08-25T08:57:19.525204+0000","last_unstale":"2023-08-25T08:57:21.577237+0000","last_undegraded":"2023-08-25T08:57:21.577237+0000","last_fullsized":"2023-08-25T08:57:21.577237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:52:21.853120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.862027+0000","last_change":"2023-08-25T08:57:18.715832+0000","last_active":"2023-08-25T08:57:20.862027+0000","last_peered":"2023-08-25T08:57:20.862027+0000","last_clean":"2023-08-25T08:57:20.862027+0000","last_became_active":"2023-08-25T08:57:18.714703+0000","last_became_peered":"2023-08-25T08:57:18.714703+0000","last_unstale":"2023-08-25T08:57:20.862027+0000","last_undegraded":"2023-08-25T08:57:20.862027+0000","last_fullsized":"2023-08-25T08:57:20.862027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:01:21.467556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577900+0000","last_change":"2023-08-25T08:57:19.527500+0000","last_active":"2023-08-25T08:57:21.577900+0000","last_peered":"2023-08-25T08:57:21.577900+0000","last_clean":"2023-08-25T08:57:21.577900+0000","last_became_active":"2023-08-25T08:57:19.527172+0000","last_became_peered":"2023-08-25T08:57:19.527172+0000","last_unstale":"2023-08-25T08:57:21.577900+0000","last_undegraded":"2023-08-25T08:57:21.577900+0000","last_fullsized":"2023-08-25T08:57:21.577900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:45.376591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:26.036776+0000","last_change":"2023-08-25T08:57:18.715563+0000","last_active":"2023-08-25T08:57:26.036776+0000","last_peered":"2023-08-25T08:57:26.036776+0000","last_clean":"2023-08-25T08:57:26.036776+0000","last_became_active":"2023-08-25T08:57:18.715269+0000","last_became_peered":"2023-08-25T08:57:18.715269+0000","last_unstale":"2023-08-25T08:57:26.036776+0000","last_undegraded":"2023-08-25T08:57:26.036776+0000","last_fullsized":"2023-08-25T08:57:26.036776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:14:41.660718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.861871+0000","last_change":"2023-08-25T08:57:19.518705+0000","last_active":"2023-08-25T08:57:20.861871+0000","last_peered":"2023-08-25T08:57:20.861871+0000","last_clean":"2023-08-25T08:57:20.861871+0000","last_became_active":"2023-08-25T08:57:19.518422+0000","last_became_peered":"2023-08-25T08:57:19.518422+0000","last_unstale":"2023-08-25T08:57:20.861871+0000","last_undegraded":"2023-08-25T08:57:20.861871+0000","last_fullsized":"2023-08-25T08:57:20.861871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:52:45.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.039955+0000","last_change":"2023-08-25T08:57:18.714087+0000","last_active":"2023-08-25T08:57:24.039955+0000","last_peered":"2023-08-25T08:57:24.039955+0000","last_clean":"2023-08-25T08:57:24.039955+0000","last_became_active":"2023-08-25T08:57:18.713777+0000","last_became_peered":"2023-08-25T08:57:18.713777+0000","last_unstale":"2023-08-25T08:57:24.039955+0000","last_undegraded":"2023-08-25T08:57:24.039955+0000","last_fullsized":"2023-08-25T08:57:24.039955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:19:44.987996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577763+0000","last_change":"2023-08-25T08:57:19.518068+0000","last_active":"2023-08-25T08:57:21.577763+0000","last_peered":"2023-08-25T08:57:21.577763+0000","last_clean":"2023-08-25T08:57:21.577763+0000","last_became_active":"2023-08-25T08:57:19.517752+0000","last_became_peered":"2023-08-25T08:57:19.517752+0000","last_unstale":"2023-08-25T08:57:21.577763+0000","last_undegraded":"2023-08-25T08:57:21.577763+0000","last_fullsized":"2023-08-25T08:57:21.577763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:10.746641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509232+0000","last_change":"2023-08-25T08:57:18.713652+0000","last_active":"2023-08-25T08:57:20.509232+0000","last_peered":"2023-08-25T08:57:20.509232+0000","last_clean":"2023-08-25T08:57:20.509232+0000","last_became_active":"2023-08-25T08:57:18.713348+0000","last_became_peered":"2023-08-25T08:57:18.713348+0000","last_unstale":"2023-08-25T08:57:20.509232+0000","last_undegraded":"2023-08-25T08:57:20.509232+0000","last_fullsized":"2023-08-25T08:57:20.509232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:22.041936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.507567+0000","last_change":"2023-08-25T08:57:19.521626+0000","last_active":"2023-08-25T08:57:20.507567+0000","last_peered":"2023-08-25T08:57:20.507567+0000","last_clean":"2023-08-25T08:57:20.507567+0000","last_became_active":"2023-08-25T08:57:19.520055+0000","last_became_peered":"2023-08-25T08:57:19.520055+0000","last_unstale":"2023-08-25T08:57:20.507567+0000","last_undegraded":"2023-08-25T08:57:20.507567+0000","last_fullsized":"2023-08-25T08:57:20.507567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:12.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577436+0000","last_change":"2023-08-25T08:57:18.713735+0000","last_active":"2023-08-25T08:57:21.577436+0000","last_peered":"2023-08-25T08:57:21.577436+0000","last_clean":"2023-08-25T08:57:21.577436+0000","last_became_active":"2023-08-25T08:57:18.713421+0000","last_became_peered":"2023-08-25T08:57:18.713421+0000","last_unstale":"2023-08-25T08:57:21.577436+0000","last_undegraded":"2023-08-25T08:57:21.577436+0000","last_fullsized":"2023-08-25T08:57:21.577436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:52.319311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577707+0000","last_change":"2023-08-25T08:57:19.513137+0000","last_active":"2023-08-25T08:57:21.577707+0000","last_peered":"2023-08-25T08:57:21.577707+0000","last_clean":"2023-08-25T08:57:21.577707+0000","last_became_active":"2023-08-25T08:57:19.512804+0000","last_became_peered":"2023-08-25T08:57:19.512804+0000","last_unstale":"2023-08-25T08:57:21.577707+0000","last_undegraded":"2023-08-25T08:57:21.577707+0000","last_fullsized":"2023-08-25T08:57:21.577707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:12:45.443703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.029804+0000","last_change":"2023-08-25T08:57:18.712855+0000","last_active":"2023-08-25T08:57:24.029804+0000","last_peered":"2023-08-25T08:57:24.029804+0000","last_clean":"2023-08-25T08:57:24.029804+0000","last_became_active":"2023-08-25T08:57:18.712327+0000","last_became_peered":"2023-08-25T08:57:18.712327+0000","last_unstale":"2023-08-25T08:57:24.029804+0000","last_undegraded":"2023-08-25T08:57:24.029804+0000","last_fullsized":"2023-08-25T08:57:24.029804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:03:13.409284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578251+0000","last_change":"2023-08-25T08:57:19.526933+0000","last_active":"2023-08-25T08:57:21.578251+0000","last_peered":"2023-08-25T08:57:21.578251+0000","last_clean":"2023-08-25T08:57:21.578251+0000","last_became_active":"2023-08-25T08:57:19.524798+0000","last_became_peered":"2023-08-25T08:57:19.524798+0000","last_unstale":"2023-08-25T08:57:21.578251+0000","last_undegraded":"2023-08-25T08:57:21.578251+0000","last_fullsized":"2023-08-25T08:57:21.578251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:34:33.721743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.063068+0000","last_change":"2023-08-25T08:57:18.714182+0000","last_active":"2023-08-25T08:57:23.063068+0000","last_peered":"2023-08-25T08:57:23.063068+0000","last_clean":"2023-08-25T08:57:23.063068+0000","last_became_active":"2023-08-25T08:57:18.713872+0000","last_became_peered":"2023-08-25T08:57:18.713872+0000","last_unstale":"2023-08-25T08:57:23.063068+0000","last_undegraded":"2023-08-25T08:57:23.063068+0000","last_fullsized":"2023-08-25T08:57:23.063068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:09:56.521066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577490+0000","last_change":"2023-08-25T08:57:19.530745+0000","last_active":"2023-08-25T08:57:21.577490+0000","last_peered":"2023-08-25T08:57:21.577490+0000","last_clean":"2023-08-25T08:57:21.577490+0000","last_became_active":"2023-08-25T08:57:19.530397+0000","last_became_peered":"2023-08-25T08:57:19.530397+0000","last_unstale":"2023-08-25T08:57:21.577490+0000","last_undegraded":"2023-08-25T08:57:21.577490+0000","last_fullsized":"2023-08-25T08:57:21.577490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:50:19.404416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.062180+0000","last_change":"2023-08-25T08:57:18.716995+0000","last_active":"2023-08-25T08:57:23.062180+0000","last_peered":"2023-08-25T08:57:23.062180+0000","last_clean":"2023-08-25T08:57:23.062180+0000","last_became_active":"2023-08-25T08:57:18.716672+0000","last_became_peered":"2023-08-25T08:57:18.716672+0000","last_unstale":"2023-08-25T08:57:23.062180+0000","last_undegraded":"2023-08-25T08:57:23.062180+0000","last_fullsized":"2023-08-25T08:57:23.062180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:23:53.036884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577874+0000","last_change":"2023-08-25T08:57:19.526608+0000","last_active":"2023-08-25T08:57:21.577874+0000","last_peered":"2023-08-25T08:57:21.577874+0000","last_clean":"2023-08-25T08:57:21.577874+0000","last_became_active":"2023-08-25T08:57:19.525614+0000","last_became_peered":"2023-08-25T08:57:19.525614+0000","last_unstale":"2023-08-25T08:57:21.577874+0000","last_undegraded":"2023-08-25T08:57:21.577874+0000","last_fullsized":"2023-08-25T08:57:21.577874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:10:36.808709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'14","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:25.536047+0000","last_change":"2023-08-25T08:57:18.712588+0000","last_active":"2023-08-25T08:57:25.536047+0000","last_peered":"2023-08-25T08:57:25.536047+0000","last_clean":"2023-08-25T08:57:25.536047+0000","last_became_active":"2023-08-25T08:57:18.712259+0000","last_became_peered":"2023-08-25T08:57:18.712259+0000","last_unstale":"2023-08-25T08:57:25.536047+0000","last_undegraded":"2023-08-25T08:57:25.536047+0000","last_fullsized":"2023-08-25T08:57:25.536047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:30:37.621654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18977,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":15,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.862200+0000","last_change":"2023-08-25T08:57:19.517907+0000","last_active":"2023-08-25T08:57:20.862200+0000","last_peered":"2023-08-25T08:57:20.862200+0000","last_clean":"2023-08-25T08:57:20.862200+0000","last_became_active":"2023-08-25T08:57:19.517596+0000","last_became_peered":"2023-08-25T08:57:19.517596+0000","last_unstale":"2023-08-25T08:57:20.862200+0000","last_undegraded":"2023-08-25T08:57:20.862200+0000","last_fullsized":"2023-08-25T08:57:20.862200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:24:49.829820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.039436+0000","last_change":"2023-08-25T08:57:18.721564+0000","last_active":"2023-08-25T08:57:24.039436+0000","last_peered":"2023-08-25T08:57:24.039436+0000","last_clean":"2023-08-25T08:57:24.039436+0000","last_became_active":"2023-08-25T08:57:18.721235+0000","last_became_peered":"2023-08-25T08:57:18.721235+0000","last_unstale":"2023-08-25T08:57:24.039436+0000","last_undegraded":"2023-08-25T08:57:24.039436+0000","last_fullsized":"2023-08-25T08:57:24.039436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:26:55.065629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509111+0000","last_change":"2023-08-25T08:57:19.518399+0000","last_active":"2023-08-25T08:57:20.509111+0000","last_peered":"2023-08-25T08:57:20.509111+0000","last_clean":"2023-08-25T08:57:20.509111+0000","last_became_active":"2023-08-25T08:57:19.518061+0000","last_became_peered":"2023-08-25T08:57:19.518061+0000","last_unstale":"2023-08-25T08:57:20.509111+0000","last_undegraded":"2023-08-25T08:57:20.509111+0000","last_fullsized":"2023-08-25T08:57:20.509111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:35:24.119537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.063412+0000","last_change":"2023-08-25T08:57:18.722020+0000","last_active":"2023-08-25T08:57:23.063412+0000","last_peered":"2023-08-25T08:57:23.063412+0000","last_clean":"2023-08-25T08:57:23.063412+0000","last_became_active":"2023-08-25T08:57:18.721707+0000","last_became_peered":"2023-08-25T08:57:18.721707+0000","last_unstale":"2023-08-25T08:57:23.063412+0000","last_undegraded":"2023-08-25T08:57:23.063412+0000","last_fullsized":"2023-08-25T08:57:23.063412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:16:27.471685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578379+0000","last_change":"2023-08-25T08:57:19.513954+0000","last_active":"2023-08-25T08:57:21.578379+0000","last_peered":"2023-08-25T08:57:21.578379+0000","last_clean":"2023-08-25T08:57:21.578379+0000","last_became_active":"2023-08-25T08:57:19.513645+0000","last_became_peered":"2023-08-25T08:57:19.513645+0000","last_unstale":"2023-08-25T08:57:21.578379+0000","last_undegraded":"2023-08-25T08:57:21.578379+0000","last_fullsized":"2023-08-25T08:57:21.578379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:27:38.030869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:26.042928+0000","last_change":"2023-08-25T08:57:18.711499+0000","last_active":"2023-08-25T08:57:26.042928+0000","last_peered":"2023-08-25T08:57:26.042928+0000","last_clean":"2023-08-25T08:57:26.042928+0000","last_became_active":"2023-08-25T08:57:18.711223+0000","last_became_peered":"2023-08-25T08:57:18.711223+0000","last_unstale":"2023-08-25T08:57:26.042928+0000","last_undegraded":"2023-08-25T08:57:26.042928+0000","last_fullsized":"2023-08-25T08:57:26.042928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:34:20.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577471+0000","last_change":"2023-08-25T08:57:19.518663+0000","last_active":"2023-08-25T08:57:21.577471+0000","last_peered":"2023-08-25T08:57:21.577471+0000","last_clean":"2023-08-25T08:57:21.577471+0000","last_became_active":"2023-08-25T08:57:19.518394+0000","last_became_peered":"2023-08-25T08:57:19.518394+0000","last_unstale":"2023-08-25T08:57:21.577471+0000","last_undegraded":"2023-08-25T08:57:21.577471+0000","last_fullsized":"2023-08-25T08:57:21.577471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:35:26.927923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.041844+0000","last_change":"2023-08-25T08:57:18.718554+0000","last_active":"2023-08-25T08:57:24.041844+0000","last_peered":"2023-08-25T08:57:24.041844+0000","last_clean":"2023-08-25T08:57:24.041844+0000","last_became_active":"2023-08-25T08:57:18.718283+0000","last_became_peered":"2023-08-25T08:57:18.718283+0000","last_unstale":"2023-08-25T08:57:24.041844+0000","last_undegraded":"2023-08-25T08:57:24.041844+0000","last_fullsized":"2023-08-25T08:57:24.041844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:47:23.878565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577584+0000","last_change":"2023-08-25T08:57:19.520856+0000","last_active":"2023-08-25T08:57:21.577584+0000","last_peered":"2023-08-25T08:57:21.577584+0000","last_clean":"2023-08-25T08:57:21.577584+0000","last_became_active":"2023-08-25T08:57:19.520568+0000","last_became_peered":"2023-08-25T08:57:19.520568+0000","last_unstale":"2023-08-25T08:57:21.577584+0000","last_undegraded":"2023-08-25T08:57:21.577584+0000","last_fullsized":"2023-08-25T08:57:21.577584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:26:17.967186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.862211+0000","last_change":"2023-08-25T08:57:18.715972+0000","last_active":"2023-08-25T08:57:20.862211+0000","last_peered":"2023-08-25T08:57:20.862211+0000","last_clean":"2023-08-25T08:57:20.862211+0000","last_became_active":"2023-08-25T08:57:18.715519+0000","last_became_peered":"2023-08-25T08:57:18.715519+0000","last_unstale":"2023-08-25T08:57:20.862211+0000","last_undegraded":"2023-08-25T08:57:20.862211+0000","last_fullsized":"2023-08-25T08:57:20.862211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:37:26.806786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920710+0000","last_change":"2023-08-25T08:57:19.517877+0000","last_active":"2023-08-25T08:57:21.920710+0000","last_peered":"2023-08-25T08:57:21.920710+0000","last_clean":"2023-08-25T08:57:21.920710+0000","last_became_active":"2023-08-25T08:57:19.517539+0000","last_became_peered":"2023-08-25T08:57:19.517539+0000","last_unstale":"2023-08-25T08:57:21.920710+0000","last_undegraded":"2023-08-25T08:57:21.920710+0000","last_fullsized":"2023-08-25T08:57:21.920710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:29:20.537007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577827+0000","last_change":"2023-08-25T08:57:18.713835+0000","last_active":"2023-08-25T08:57:21.577827+0000","last_peered":"2023-08-25T08:57:21.577827+0000","last_clean":"2023-08-25T08:57:21.577827+0000","last_became_active":"2023-08-25T08:57:18.713510+0000","last_became_peered":"2023-08-25T08:57:18.713510+0000","last_unstale":"2023-08-25T08:57:21.577827+0000","last_undegraded":"2023-08-25T08:57:21.577827+0000","last_fullsized":"2023-08-25T08:57:21.577827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:51.073757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.579129+0000","last_change":"2023-08-25T08:57:19.526979+0000","last_active":"2023-08-25T08:57:21.579129+0000","last_peered":"2023-08-25T08:57:21.579129+0000","last_clean":"2023-08-25T08:57:21.579129+0000","last_became_active":"2023-08-25T08:57:19.526660+0000","last_became_peered":"2023-08-25T08:57:19.526660+0000","last_unstale":"2023-08-25T08:57:21.579129+0000","last_undegraded":"2023-08-25T08:57:21.579129+0000","last_fullsized":"2023-08-25T08:57:21.579129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:23:10.052607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578722+0000","last_change":"2023-08-25T08:57:18.717952+0000","last_active":"2023-08-25T08:57:21.578722+0000","last_peered":"2023-08-25T08:57:21.578722+0000","last_clean":"2023-08-25T08:57:21.578722+0000","last_became_active":"2023-08-25T08:57:18.717643+0000","last_became_peered":"2023-08-25T08:57:18.717643+0000","last_unstale":"2023-08-25T08:57:21.578722+0000","last_undegraded":"2023-08-25T08:57:21.578722+0000","last_fullsized":"2023-08-25T08:57:21.578722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:52:22.390179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577843+0000","last_change":"2023-08-25T08:57:19.521718+0000","last_active":"2023-08-25T08:57:21.577843+0000","last_peered":"2023-08-25T08:57:21.577843+0000","last_clean":"2023-08-25T08:57:21.577843+0000","last_became_active":"2023-08-25T08:57:19.521246+0000","last_became_peered":"2023-08-25T08:57:19.521246+0000","last_unstale":"2023-08-25T08:57:21.577843+0000","last_undegraded":"2023-08-25T08:57:21.577843+0000","last_fullsized":"2023-08-25T08:57:21.577843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:36.200645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577622+0000","last_change":"2023-08-25T08:57:19.518510+0000","last_active":"2023-08-25T08:57:21.577622+0000","last_peered":"2023-08-25T08:57:21.577622+0000","last_clean":"2023-08-25T08:57:21.577622+0000","last_became_active":"2023-08-25T08:57:19.518202+0000","last_became_peered":"2023-08-25T08:57:19.518202+0000","last_unstale":"2023-08-25T08:57:21.577622+0000","last_undegraded":"2023-08-25T08:57:21.577622+0000","last_fullsized":"2023-08-25T08:57:21.577622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:00.267573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509145+0000","last_change":"2023-08-25T08:57:18.718657+0000","last_active":"2023-08-25T08:57:20.509145+0000","last_peered":"2023-08-25T08:57:20.509145+0000","last_clean":"2023-08-25T08:57:20.509145+0000","last_became_active":"2023-08-25T08:57:18.718348+0000","last_became_peered":"2023-08-25T08:57:18.718348+0000","last_unstale":"2023-08-25T08:57:20.509145+0000","last_undegraded":"2023-08-25T08:57:20.509145+0000","last_fullsized":"2023-08-25T08:57:20.509145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:24:14.753607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508148+0000","last_change":"2023-08-25T08:57:19.520997+0000","last_active":"2023-08-25T08:57:20.508148+0000","last_peered":"2023-08-25T08:57:20.508148+0000","last_clean":"2023-08-25T08:57:20.508148+0000","last_became_active":"2023-08-25T08:57:19.520695+0000","last_became_peered":"2023-08-25T08:57:19.520695+0000","last_unstale":"2023-08-25T08:57:20.508148+0000","last_undegraded":"2023-08-25T08:57:20.508148+0000","last_fullsized":"2023-08-25T08:57:20.508148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:10.661689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.040324+0000","last_change":"2023-08-25T08:57:18.714573+0000","last_active":"2023-08-25T08:57:24.040324+0000","last_peered":"2023-08-25T08:57:24.040324+0000","last_clean":"2023-08-25T08:57:24.040324+0000","last_became_active":"2023-08-25T08:57:18.714268+0000","last_became_peered":"2023-08-25T08:57:18.714268+0000","last_unstale":"2023-08-25T08:57:24.040324+0000","last_undegraded":"2023-08-25T08:57:24.040324+0000","last_fullsized":"2023-08-25T08:57:24.040324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:33.154816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577677+0000","last_change":"2023-08-25T08:56:14.362397+0000","last_active":"2023-08-25T08:57:21.577677+0000","last_peered":"2023-08-25T08:57:21.577677+0000","last_clean":"2023-08-25T08:57:21.577677+0000","last_became_active":"2023-08-25T08:56:14.361946+0000","last_became_peered":"2023-08-25T08:56:14.361946+0000","last_unstale":"2023-08-25T08:57:21.577677+0000","last_undegraded":"2023-08-25T08:57:21.577677+0000","last_fullsized":"2023-08-25T08:57:21.577677+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:53:09.022543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:53:09.022543+0000","last_clean_scrub_stamp":"2023-08-25T08:53:09.022543+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:21:51.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509008+0000","last_change":"2023-08-25T08:57:19.517547+0000","last_active":"2023-08-25T08:57:20.509008+0000","last_peered":"2023-08-25T08:57:20.509008+0000","last_clean":"2023-08-25T08:57:20.509008+0000","last_became_active":"2023-08-25T08:57:19.517221+0000","last_became_peered":"2023-08-25T08:57:19.517221+0000","last_unstale":"2023-08-25T08:57:20.509008+0000","last_undegraded":"2023-08-25T08:57:20.509008+0000","last_fullsized":"2023-08-25T08:57:20.509008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:45.891452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577918+0000","last_change":"2023-08-25T08:57:18.715769+0000","last_active":"2023-08-25T08:57:21.577918+0000","last_peered":"2023-08-25T08:57:21.577918+0000","last_clean":"2023-08-25T08:57:21.577918+0000","last_became_active":"2023-08-25T08:57:18.715399+0000","last_became_peered":"2023-08-25T08:57:18.715399+0000","last_unstale":"2023-08-25T08:57:21.577918+0000","last_undegraded":"2023-08-25T08:57:21.577918+0000","last_fullsized":"2023-08-25T08:57:21.577918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:39:15.306754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.063627+0000","last_change":"2023-08-25T08:57:18.720658+0000","last_active":"2023-08-25T08:57:23.063627+0000","last_peered":"2023-08-25T08:57:23.063627+0000","last_clean":"2023-08-25T08:57:23.063627+0000","last_became_active":"2023-08-25T08:57:18.720419+0000","last_became_peered":"2023-08-25T08:57:18.720419+0000","last_unstale":"2023-08-25T08:57:23.063627+0000","last_undegraded":"2023-08-25T08:57:23.063627+0000","last_fullsized":"2023-08-25T08:57:23.063627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:15.624027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509606+0000","last_change":"2023-08-25T08:57:19.513257+0000","last_active":"2023-08-25T08:57:20.509606+0000","last_peered":"2023-08-25T08:57:20.509606+0000","last_clean":"2023-08-25T08:57:20.509606+0000","last_became_active":"2023-08-25T08:57:19.512852+0000","last_became_peered":"2023-08-25T08:57:19.512852+0000","last_unstale":"2023-08-25T08:57:20.509606+0000","last_undegraded":"2023-08-25T08:57:20.509606+0000","last_fullsized":"2023-08-25T08:57:20.509606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:29:24.814590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.039988+0000","last_change":"2023-08-25T08:57:18.718740+0000","last_active":"2023-08-25T08:57:24.039988+0000","last_peered":"2023-08-25T08:57:24.039988+0000","last_clean":"2023-08-25T08:57:24.039988+0000","last_became_active":"2023-08-25T08:57:18.718433+0000","last_became_peered":"2023-08-25T08:57:18.718433+0000","last_unstale":"2023-08-25T08:57:24.039988+0000","last_undegraded":"2023-08-25T08:57:24.039988+0000","last_fullsized":"2023-08-25T08:57:24.039988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:55:15.935758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508534+0000","last_change":"2023-08-25T08:57:19.525867+0000","last_active":"2023-08-25T08:57:20.508534+0000","last_peered":"2023-08-25T08:57:20.508534+0000","last_clean":"2023-08-25T08:57:20.508534+0000","last_became_active":"2023-08-25T08:57:19.525492+0000","last_became_peered":"2023-08-25T08:57:19.525492+0000","last_unstale":"2023-08-25T08:57:20.508534+0000","last_undegraded":"2023-08-25T08:57:20.508534+0000","last_fullsized":"2023-08-25T08:57:20.508534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:53.064700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.062378+0000","last_change":"2023-08-25T08:57:18.711447+0000","last_active":"2023-08-25T08:57:23.062378+0000","last_peered":"2023-08-25T08:57:23.062378+0000","last_clean":"2023-08-25T08:57:23.062378+0000","last_became_active":"2023-08-25T08:57:18.711164+0000","last_became_peered":"2023-08-25T08:57:18.711164+0000","last_unstale":"2023-08-25T08:57:23.062378+0000","last_undegraded":"2023-08-25T08:57:23.062378+0000","last_fullsized":"2023-08-25T08:57:23.062378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:27:12.390451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577359+0000","last_change":"2023-08-25T08:57:19.528927+0000","last_active":"2023-08-25T08:57:21.577359+0000","last_peered":"2023-08-25T08:57:21.577359+0000","last_clean":"2023-08-25T08:57:21.577359+0000","last_became_active":"2023-08-25T08:57:19.528281+0000","last_became_peered":"2023-08-25T08:57:19.528281+0000","last_unstale":"2023-08-25T08:57:21.577359+0000","last_undegraded":"2023-08-25T08:57:21.577359+0000","last_fullsized":"2023-08-25T08:57:21.577359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:03.091434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.063187+0000","last_change":"2023-08-25T08:57:18.713474+0000","last_active":"2023-08-25T08:57:23.063187+0000","last_peered":"2023-08-25T08:57:23.063187+0000","last_clean":"2023-08-25T08:57:23.063187+0000","last_became_active":"2023-08-25T08:57:18.713147+0000","last_became_peered":"2023-08-25T08:57:18.713147+0000","last_unstale":"2023-08-25T08:57:23.063187+0000","last_undegraded":"2023-08-25T08:57:23.063187+0000","last_fullsized":"2023-08-25T08:57:23.063187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:55.688173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.861772+0000","last_change":"2023-08-25T08:57:19.515035+0000","last_active":"2023-08-25T08:57:20.861772+0000","last_peered":"2023-08-25T08:57:20.861772+0000","last_clean":"2023-08-25T08:57:20.861772+0000","last_became_active":"2023-08-25T08:57:19.514729+0000","last_became_peered":"2023-08-25T08:57:19.514729+0000","last_unstale":"2023-08-25T08:57:20.861772+0000","last_undegraded":"2023-08-25T08:57:20.861772+0000","last_fullsized":"2023-08-25T08:57:20.861772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:18.522519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.061221+0000","last_change":"2023-08-25T08:57:18.721451+0000","last_active":"2023-08-25T08:57:23.061221+0000","last_peered":"2023-08-25T08:57:23.061221+0000","last_clean":"2023-08-25T08:57:23.061221+0000","last_became_active":"2023-08-25T08:57:18.721132+0000","last_became_peered":"2023-08-25T08:57:18.721132+0000","last_unstale":"2023-08-25T08:57:23.061221+0000","last_undegraded":"2023-08-25T08:57:23.061221+0000","last_fullsized":"2023-08-25T08:57:23.061221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:48.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920596+0000","last_change":"2023-08-25T08:57:19.519649+0000","last_active":"2023-08-25T08:57:21.920596+0000","last_peered":"2023-08-25T08:57:21.920596+0000","last_clean":"2023-08-25T08:57:21.920596+0000","last_became_active":"2023-08-25T08:57:19.518907+0000","last_became_peered":"2023-08-25T08:57:19.518907+0000","last_unstale":"2023-08-25T08:57:21.920596+0000","last_undegraded":"2023-08-25T08:57:21.920596+0000","last_fullsized":"2023-08-25T08:57:21.920596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:32:55.986526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506598+0000","last_change":"2023-08-25T08:57:18.722996+0000","last_active":"2023-08-25T08:57:20.506598+0000","last_peered":"2023-08-25T08:57:20.506598+0000","last_clean":"2023-08-25T08:57:20.506598+0000","last_became_active":"2023-08-25T08:57:18.722662+0000","last_became_peered":"2023-08-25T08:57:18.722662+0000","last_unstale":"2023-08-25T08:57:20.506598+0000","last_undegraded":"2023-08-25T08:57:20.506598+0000","last_fullsized":"2023-08-25T08:57:20.506598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:30:17.722094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.507862+0000","last_change":"2023-08-25T08:57:19.529138+0000","last_active":"2023-08-25T08:57:20.507862+0000","last_peered":"2023-08-25T08:57:20.507862+0000","last_clean":"2023-08-25T08:57:20.507862+0000","last_became_active":"2023-08-25T08:57:19.527485+0000","last_became_peered":"2023-08-25T08:57:19.527485+0000","last_unstale":"2023-08-25T08:57:20.507862+0000","last_undegraded":"2023-08-25T08:57:20.507862+0000","last_fullsized":"2023-08-25T08:57:20.507862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:51.151079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:23.064239+0000","last_change":"2023-08-25T08:57:18.717811+0000","last_active":"2023-08-25T08:57:23.064239+0000","last_peered":"2023-08-25T08:57:23.064239+0000","last_clean":"2023-08-25T08:57:23.064239+0000","last_became_active":"2023-08-25T08:57:18.717573+0000","last_became_peered":"2023-08-25T08:57:18.717573+0000","last_unstale":"2023-08-25T08:57:23.064239+0000","last_undegraded":"2023-08-25T08:57:23.064239+0000","last_fullsized":"2023-08-25T08:57:23.064239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:13:37.354858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578921+0000","last_change":"2023-08-25T08:57:19.528271+0000","last_active":"2023-08-25T08:57:21.578921+0000","last_peered":"2023-08-25T08:57:21.578921+0000","last_clean":"2023-08-25T08:57:21.578921+0000","last_became_active":"2023-08-25T08:57:19.528017+0000","last_became_peered":"2023-08-25T08:57:19.528017+0000","last_unstale":"2023-08-25T08:57:21.578921+0000","last_undegraded":"2023-08-25T08:57:21.578921+0000","last_fullsized":"2023-08-25T08:57:21.578921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:01:34.370416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:24.040183+0000","last_change":"2023-08-25T08:57:18.715063+0000","last_active":"2023-08-25T08:57:24.040183+0000","last_peered":"2023-08-25T08:57:24.040183+0000","last_clean":"2023-08-25T08:57:24.040183+0000","last_became_active":"2023-08-25T08:57:18.714755+0000","last_became_peered":"2023-08-25T08:57:18.714755+0000","last_unstale":"2023-08-25T08:57:24.040183+0000","last_undegraded":"2023-08-25T08:57:24.040183+0000","last_fullsized":"2023-08-25T08:57:24.040183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:08:26.104256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578831+0000","last_change":"2023-08-25T08:57:19.528557+0000","last_active":"2023-08-25T08:57:21.578831+0000","last_peered":"2023-08-25T08:57:21.578831+0000","last_clean":"2023-08-25T08:57:21.578831+0000","last_became_active":"2023-08-25T08:57:19.528284+0000","last_became_peered":"2023-08-25T08:57:19.528284+0000","last_unstale":"2023-08-25T08:57:21.578831+0000","last_undegraded":"2023-08-25T08:57:21.578831+0000","last_fullsized":"2023-08-25T08:57:21.578831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:52:50.854609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577843+0000","last_change":"2023-08-25T08:57:18.722893+0000","last_active":"2023-08-25T08:57:21.577843+0000","last_peered":"2023-08-25T08:57:21.577843+0000","last_clean":"2023-08-25T08:57:21.577843+0000","last_became_active":"2023-08-25T08:57:18.722553+0000","last_became_peered":"2023-08-25T08:57:18.722553+0000","last_unstale":"2023-08-25T08:57:21.577843+0000","last_undegraded":"2023-08-25T08:57:21.577843+0000","last_fullsized":"2023-08-25T08:57:21.577843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:25:00.205900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578471+0000","last_change":"2023-08-25T08:57:19.517975+0000","last_active":"2023-08-25T08:57:21.578471+0000","last_peered":"2023-08-25T08:57:21.578471+0000","last_clean":"2023-08-25T08:57:21.578471+0000","last_became_active":"2023-08-25T08:57:19.517715+0000","last_became_peered":"2023-08-25T08:57:19.517715+0000","last_unstale":"2023-08-25T08:57:21.578471+0000","last_undegraded":"2023-08-25T08:57:21.578471+0000","last_fullsized":"2023-08-25T08:57:21.578471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:26:59.048019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508997+0000","last_change":"2023-08-25T08:57:18.717331+0000","last_active":"2023-08-25T08:57:20.508997+0000","last_peered":"2023-08-25T08:57:20.508997+0000","last_clean":"2023-08-25T08:57:20.508997+0000","last_became_active":"2023-08-25T08:57:18.716957+0000","last_became_peered":"2023-08-25T08:57:18.716957+0000","last_unstale":"2023-08-25T08:57:20.508997+0000","last_undegraded":"2023-08-25T08:57:20.508997+0000","last_fullsized":"2023-08-25T08:57:20.508997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:21.300644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.578015+0000","last_change":"2023-08-25T08:57:19.515217+0000","last_active":"2023-08-25T08:57:21.578015+0000","last_peered":"2023-08-25T08:57:21.578015+0000","last_clean":"2023-08-25T08:57:21.578015+0000","last_became_active":"2023-08-25T08:57:19.514870+0000","last_became_peered":"2023-08-25T08:57:19.514870+0000","last_unstale":"2023-08-25T08:57:21.578015+0000","last_undegraded":"2023-08-25T08:57:21.578015+0000","last_fullsized":"2023-08-25T08:57:21.578015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:17:43.197478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920528+0000","last_change":"2023-08-25T08:57:19.512984+0000","last_active":"2023-08-25T08:57:21.920528+0000","last_peered":"2023-08-25T08:57:21.920528+0000","last_clean":"2023-08-25T08:57:21.920528+0000","last_became_active":"2023-08-25T08:57:19.512644+0000","last_became_peered":"2023-08-25T08:57:19.512644+0000","last_unstale":"2023-08-25T08:57:21.920528+0000","last_undegraded":"2023-08-25T08:57:21.920528+0000","last_fullsized":"2023-08-25T08:57:21.920528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:04:51.059464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506789+0000","last_change":"2023-08-25T08:57:19.521540+0000","last_active":"2023-08-25T08:57:20.506789+0000","last_peered":"2023-08-25T08:57:20.506789+0000","last_clean":"2023-08-25T08:57:20.506789+0000","last_became_active":"2023-08-25T08:57:19.521305+0000","last_became_peered":"2023-08-25T08:57:19.521305+0000","last_unstale":"2023-08-25T08:57:20.506789+0000","last_undegraded":"2023-08-25T08:57:20.506789+0000","last_fullsized":"2023-08-25T08:57:20.506789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:44:48.837529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506686+0000","last_change":"2023-08-25T08:57:19.526830+0000","last_active":"2023-08-25T08:57:20.506686+0000","last_peered":"2023-08-25T08:57:20.506686+0000","last_clean":"2023-08-25T08:57:20.506686+0000","last_became_active":"2023-08-25T08:57:19.526396+0000","last_became_peered":"2023-08-25T08:57:19.526396+0000","last_unstale":"2023-08-25T08:57:20.506686+0000","last_undegraded":"2023-08-25T08:57:20.506686+0000","last_fullsized":"2023-08-25T08:57:20.506686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:48.876189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577326+0000","last_change":"2023-08-25T08:57:19.522557+0000","last_active":"2023-08-25T08:57:21.577326+0000","last_peered":"2023-08-25T08:57:21.577326+0000","last_clean":"2023-08-25T08:57:21.577326+0000","last_became_active":"2023-08-25T08:57:19.522076+0000","last_became_peered":"2023-08-25T08:57:19.522076+0000","last_unstale":"2023-08-25T08:57:21.577326+0000","last_undegraded":"2023-08-25T08:57:21.577326+0000","last_fullsized":"2023-08-25T08:57:21.577326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:13.619576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509444+0000","last_change":"2023-08-25T08:57:19.518894+0000","last_active":"2023-08-25T08:57:20.509444+0000","last_peered":"2023-08-25T08:57:20.509444+0000","last_clean":"2023-08-25T08:57:20.509444+0000","last_became_active":"2023-08-25T08:57:19.518599+0000","last_became_peered":"2023-08-25T08:57:19.518599+0000","last_unstale":"2023-08-25T08:57:20.509444+0000","last_undegraded":"2023-08-25T08:57:20.509444+0000","last_fullsized":"2023-08-25T08:57:20.509444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:14.148977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920810+0000","last_change":"2023-08-25T08:57:19.513584+0000","last_active":"2023-08-25T08:57:21.920810+0000","last_peered":"2023-08-25T08:57:21.920810+0000","last_clean":"2023-08-25T08:57:21.920810+0000","last_became_active":"2023-08-25T08:57:19.513187+0000","last_became_peered":"2023-08-25T08:57:19.513187+0000","last_unstale":"2023-08-25T08:57:21.920810+0000","last_undegraded":"2023-08-25T08:57:21.920810+0000","last_fullsized":"2023-08-25T08:57:21.920810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:27.836012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.506864+0000","last_change":"2023-08-25T08:57:19.528534+0000","last_active":"2023-08-25T08:57:20.506864+0000","last_peered":"2023-08-25T08:57:20.506864+0000","last_clean":"2023-08-25T08:57:20.506864+0000","last_became_active":"2023-08-25T08:57:19.528186+0000","last_became_peered":"2023-08-25T08:57:19.528186+0000","last_unstale":"2023-08-25T08:57:20.506864+0000","last_undegraded":"2023-08-25T08:57:20.506864+0000","last_fullsized":"2023-08-25T08:57:20.506864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:45.431720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.508530+0000","last_change":"2023-08-25T08:57:19.529932+0000","last_active":"2023-08-25T08:57:20.508530+0000","last_peered":"2023-08-25T08:57:20.508530+0000","last_clean":"2023-08-25T08:57:20.508530+0000","last_became_active":"2023-08-25T08:57:19.529581+0000","last_became_peered":"2023-08-25T08:57:19.529581+0000","last_unstale":"2023-08-25T08:57:20.508530+0000","last_undegraded":"2023-08-25T08:57:20.508530+0000","last_fullsized":"2023-08-25T08:57:20.508530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:10.394555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577532+0000","last_change":"2023-08-25T08:57:19.527777+0000","last_active":"2023-08-25T08:57:21.577532+0000","last_peered":"2023-08-25T08:57:21.577532+0000","last_clean":"2023-08-25T08:57:21.577532+0000","last_became_active":"2023-08-25T08:57:19.527515+0000","last_became_peered":"2023-08-25T08:57:19.527515+0000","last_unstale":"2023-08-25T08:57:21.577532+0000","last_undegraded":"2023-08-25T08:57:21.577532+0000","last_fullsized":"2023-08-25T08:57:21.577532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:24.616320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.507485+0000","last_change":"2023-08-25T08:57:19.522676+0000","last_active":"2023-08-25T08:57:20.507485+0000","last_peered":"2023-08-25T08:57:20.507485+0000","last_clean":"2023-08-25T08:57:20.507485+0000","last_became_active":"2023-08-25T08:57:19.522282+0000","last_became_peered":"2023-08-25T08:57:19.522282+0000","last_unstale":"2023-08-25T08:57:20.507485+0000","last_undegraded":"2023-08-25T08:57:20.507485+0000","last_fullsized":"2023-08-25T08:57:20.507485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:27:01.305079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.577476+0000","last_change":"2023-08-25T08:57:19.525175+0000","last_active":"2023-08-25T08:57:21.577476+0000","last_peered":"2023-08-25T08:57:21.577476+0000","last_clean":"2023-08-25T08:57:21.577476+0000","last_became_active":"2023-08-25T08:57:19.524769+0000","last_became_peered":"2023-08-25T08:57:19.524769+0000","last_unstale":"2023-08-25T08:57:21.577476+0000","last_undegraded":"2023-08-25T08:57:21.577476+0000","last_fullsized":"2023-08-25T08:57:21.577476+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:07:03.279154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:20.509572+0000","last_change":"2023-08-25T08:57:19.518697+0000","last_active":"2023-08-25T08:57:20.509572+0000","last_peered":"2023-08-25T08:57:20.509572+0000","last_clean":"2023-08-25T08:57:20.509572+0000","last_became_active":"2023-08-25T08:57:19.518316+0000","last_became_peered":"2023-08-25T08:57:19.518316+0000","last_unstale":"2023-08-25T08:57:20.509572+0000","last_undegraded":"2023-08-25T08:57:20.509572+0000","last_fullsized":"2023-08-25T08:57:20.509572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:21:41.551008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-08-25T08:57:21.920593+0000","last_change":"2023-08-25T08:57:19.515293+0000","last_active":"2023-08-25T08:57:21.920593+0000","last_peered":"2023-08-25T08:57:21.920593+0000","last_clean":"2023-08-25T08:57:21.920593+0000","last_became_active":"2023-08-25T08:57:19.514883+0000","last_became_peered":"2023-08-25T08:57:19.514883+0000","last_unstale":"2023-08-25T08:57:21.920593+0000","last_undegraded":"2023-08-25T08:57:21.920593+0000","last_fullsized":"2023-08-25T08:57:21.920593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:50:41.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250603,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22887,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":121,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":72403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449102,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":426400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1036864,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.436}]},{"osd":2,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.753},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.627}]},{"osd":3,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.373},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.36199999999999999}]},{"osd":8,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.78800000000000003}]},{"osd":9,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":426636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.66000000000000003}]},{"osd":8,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.63},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.60299999999999998}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66700000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103194,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":426444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.749}]},{"osd":1,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.503}]},{"osd":3,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Aug 25 08:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80700000000000005}]},{"osd":9,"last update":"Fri Aug 25 08:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.89700000000000002}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.875}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332125,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1017390,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"min":{"1min":0.32700000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60799999999999998,"5min":0.628,"15min":0.628},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.46200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.82599999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.496},"min":{"1min":0.38800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.503},{"interface":"front","average":{"1min":0.497,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51100000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86199999999999999,"5min":0.879,"15min":0.879},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.88}]},{"osd":6,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81699999999999995}]},{"osd":8,"last update":"Fri Aug 25 08:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.89900000000000002}]},{"osd":9,"last update":"Fri Aug 25 08:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79700000000000004}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":425818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.441,"15min":0.432},"min":{"1min":0.34999999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"min":{"1min":0.40999999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.5}]},{"osd":1,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.70699999999999996,"15min":0.71799999999999997},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84799999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.101,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.57099999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.42299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.46999999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.55200000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90800000000000003,"5min":3.9500000000000002,"15min":3.9500000000000002},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.51000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.64200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Fri Aug 25 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73299999999999998,"15min":0.71899999999999997},"min":{"1min":0.60199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.51000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri Aug 25 08:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.78300000000000003}]},{"osd":9,"last update":"Fri Aug 25 08:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.75700000000000001}]},{"osd":10,"last update":"Fri Aug 25 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.97999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":427157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.33800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.626},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61699999999999999,"15min":0.56299999999999994},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81399999999999995,"5min":5.4359999999999999,"15min":5.4359999999999999},"last":0.747}]},{"osd":2,"last update":"Fri Aug 25 08:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.441,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Aug 25 08:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85199999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.46899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83399999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.42199999999999999,"15min":0.40200000000000002},"min":{"1min":0.29299999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.44700000000000001,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42999999999999999}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri Aug 25 08:56:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86399999999999999,"5min":1.107,"15min":1.107},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":1.153,"15min":1.153},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.497},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.47199999999999998}]},{"osd":8,"last update":"Fri Aug 25 08:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.71599999999999997}]},{"osd":9,"last update":"Fri Aug 25 08:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.76600000000000001}]},{"osd":10,"last update":"Fri Aug 25 08:57:13 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.83099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":427296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58299999999999996,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Aug 25 08:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.46100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60999999999999999,"15min":0.56399999999999995},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Aug 25 08:55:37 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.46300000000000002,"15min":0.45000000000000001},"min":{"1min":0.34999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Aug 25 08:57:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.66000000000000003,"15min":0.62},"min":{"1min":0.47499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.629,"15min":0.60199999999999998},"min":{"1min":0.55400000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Aug 25 08:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.65000000000000002,"15min":0.626},"min":{"1min":0.497,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Aug 25 08:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.67900000000000005,"15min":0.65900000000000003},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Aug 25 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.77000000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68300000000000005}]},{"osd":8,"last update":"Fri Aug 25 08:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.65600000000000003}]},{"osd":9,"last update":"Fri Aug 25 08:56:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.70599999999999996}]},{"osd":10,"last update":"Fri Aug 25 08:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.82399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":463843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79700000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":1.409,"15min":1.409},"last":0.629},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.41699999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Aug 25 08:56:42 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.75}]},{"osd":5,"last update":"Fri Aug 25 08:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42899999999999999,"15min":0.42099999999999999},"min":{"1min":0.27100000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40300000000000002}]},{"osd":6,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91800000000000004,"5min":1.117,"15min":1.117},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":1.097,"15min":1.097},"last":0.64500000000000002}]},{"osd":8,"last update":"Fri Aug 25 08:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66300000000000003}]},{"osd":9,"last update":"Fri Aug 25 08:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73799999999999999}]},{"osd":10,"last update":"Fri Aug 25 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":426102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41199999999999998,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.40699999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Aug 25 08:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Aug 25 08:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84899999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Aug 25 08:57:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.51400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58799999999999997},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.52900000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Aug 25 08:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.05,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.625}]},{"osd":7,"last update":"Fri Aug 25 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.56799999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.97899999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98399999999999999,"5min":1.161,"15min":1.161},"last":0.81100000000000005}]},{"osd":8,"last update":"Fri Aug 25 08:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85199999999999998}]},{"osd":9,"last update":"Fri Aug 25 08:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.95499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920813,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":426462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.438}]},{"osd":1,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.40200000000000002},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.41499999999999998,"15min":0.40699999999999997},"min":{"1min":0.33500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69499999999999995}]},{"osd":3,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.38700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77400000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60099999999999998,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Fri Aug 25 08:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.66700000000000004,"15min":0.65500000000000003},"min":{"1min":0.54100000000000004,"5min":0.433,"15min":0.433},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Aug 25 08:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.92200000000000004}]},{"osd":8,"last update":"Fri Aug 25 08:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.997}]},{"osd":9,"last update":"Fri Aug 25 08:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.96399999999999997}]},{"osd":10,"last update":"Fri Aug 25 08:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":1.032}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757289,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1017844,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.622}]},{"osd":2,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45300000000000001,"15min":0.47099999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67900000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.46700000000000003,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61799999999999999,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95099999999999996,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71299999999999997,"15min":0.71199999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.04,"5min":1.075,"15min":1.075},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Aug 25 08:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.65700000000000003}]},{"osd":9,"last update":"Fri Aug 25 08:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri Aug 25 08:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.754},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34629,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34629,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":20555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T08:57:28.853 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T08:57:28.854 INFO:tasks.cephadm:Setting up client nodes... 2023-08-25T08:57:28.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-08-25T08:57:29.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:29 smithi099 ceph-mon[97641]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:57:29.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:29 smithi099 ceph-mon[97641]: pgmap v270: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2023-08-25T08:57:29.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:29 smithi099 ceph-mon[97641]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:57:29.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:29 smithi157 ceph-mon[130879]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:57:29.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:29 smithi157 ceph-mon[130879]: pgmap v270: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2023-08-25T08:57:29.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:29 smithi157 ceph-mon[130879]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:57:29.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:29 smithi181 ceph-mon[131296]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:57:29.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:29 smithi181 ceph-mon[131296]: pgmap v270: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2023-08-25T08:57:29.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:29 smithi181 ceph-mon[131296]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:57:29.596 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:31.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 -- 172.21.15.99:0/2246095715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb028101be0 msgr2=0x7fb028102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:31.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 --2- 172.21.15.99:0/2246095715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb028101be0 0x7fb028102040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0180099c0 tx=0x7fb018009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 -- 172.21.15.99:0/2246095715 shutdown_connections 2023-08-25T08:57:31.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 --2- 172.21.15.99:0/2246095715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb028102580 0x7fb028108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 --2- 172.21.15.99:0/2246095715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0281009e0 0x7fb028100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 --2- 172.21.15.99:0/2246095715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb028101be0 0x7fb028102040 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 -- 172.21.15.99:0/2246095715 >> 172.21.15.99:0/2246095715 conn(0x7fb0280fbfa0 msgr2=0x7fb0280fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 -- 172.21.15.99:0/2246095715 shutdown_connections 2023-08-25T08:57:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.131+0000 7fb0307b0700 1 -- 172.21.15.99:0/2246095715 wait complete. 2023-08-25T08:57:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.132+0000 7fb0307b0700 1 Processor -- start 2023-08-25T08:57:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.132+0000 7fb0307b0700 1 -- start start 2023-08-25T08:57:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.132+0000 7fb0307b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0281009e0 0x7fb0281988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.132+0000 7fb0307b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb028101be0 0x7fb028198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.132+0000 7fb0307b0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb028102580 0x7fb02819fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.132+0000 7fb0307b0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0281a0470 con 0x7fb0281009e0 2023-08-25T08:57:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.132+0000 7fb0307b0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0281a05b0 con 0x7fb028102580 2023-08-25T08:57:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.132+0000 7fb0307b0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0281a0740 con 0x7fb028101be0 2023-08-25T08:57:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02dd4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb028101be0 0x7fb028198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02e54c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0281009e0 0x7fb0281988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02dd4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb028101be0 0x7fb028198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:60372/0 (socket says 172.21.15.99:60372) 2023-08-25T08:57:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02dd4b700 1 -- 172.21.15.99:0/218613797 learned_addr learned my addr 172.21.15.99:0/218613797 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02ed4d700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb028102580 0x7fb02819fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02e54c700 1 -- 172.21.15.99:0/218613797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb028101be0 msgr2=0x7fb028198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02e54c700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb028101be0 0x7fb028198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02e54c700 1 -- 172.21.15.99:0/218613797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb028102580 msgr2=0x7fb02819fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02e54c700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb028102580 0x7fb02819fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02e54c700 1 -- 172.21.15.99:0/218613797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180095f0 con 0x7fb0281009e0 2023-08-25T08:57:31.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02dd4b700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb028101be0 0x7fb028198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:31.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.133+0000 7fb02e54c700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0281009e0 0x7fb0281988e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb02400e9f0 tx=0x7fb02400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:31.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.134+0000 7fb01f7fe700 1 -- 172.21.15.99:0/218613797 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024004220 con 0x7fb0281009e0 2023-08-25T08:57:31.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.134+0000 7fb01f7fe700 1 -- 172.21.15.99:0/218613797 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb024004380 con 0x7fb0281009e0 2023-08-25T08:57:31.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.134+0000 7fb01f7fe700 1 -- 172.21.15.99:0/218613797 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024003bb0 con 0x7fb0281009e0 2023-08-25T08:57:31.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.134+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0281a0a20 con 0x7fb0281009e0 2023-08-25T08:57:31.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.134+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0281a0f20 con 0x7fb0281009e0 2023-08-25T08:57:31.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.135+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02810c230 con 0x7fb0281009e0 2023-08-25T08:57:31.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.137+0000 7fb01f7fe700 1 -- 172.21.15.99:0/218613797 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb024003d10 con 0x7fb0281009e0 2023-08-25T08:57:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.138+0000 7fb01f7fe700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb0140a0060 0x7fb0140a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.138+0000 7fb01f7fe700 1 -- 172.21.15.99:0/218613797 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb024014070 con 0x7fb0281009e0 2023-08-25T08:57:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.138+0000 7fb02dd4b700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb0140a0060 0x7fb0140a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.139+0000 7fb02dd4b700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb0140a0060 0x7fb0140a2520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb018004800 tx=0x7fb01801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.142+0000 7fb01f7fe700 1 -- 172.21.15.99:0/218613797 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb0240951a0 con 0x7fb0281009e0 2023-08-25T08:57:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.420+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb02810c470 con 0x7fb0281009e0 2023-08-25T08:57:31.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.424+0000 7fb01f7fe700 1 -- 172.21.15.99:0/218613797 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb0240987c0 con 0x7fb0281009e0 2023-08-25T08:57:31.425 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2023-08-25T08:57:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD7bOhk6DwiGRAA4I0i2kRyoPAAByx97rH+sg== 2023-08-25T08:57:31.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb0140a0060 msgr2=0x7fb0140a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb0140a0060 0x7fb0140a2520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb018004800 tx=0x7fb01801a040 comp rx=0 tx=0).stop 2023-08-25T08:57:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0281009e0 msgr2=0x7fb0281988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0281009e0 0x7fb0281988e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb02400e9f0 tx=0x7fb02400ed00 comp rx=0 tx=0).stop 2023-08-25T08:57:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 shutdown_connections 2023-08-25T08:57:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb0140a0060 0x7fb0140a2520 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb028102580 0x7fb02819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0281009e0 0x7fb0281988e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 --2- 172.21.15.99:0/218613797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb028101be0 0x7fb028198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 >> 172.21.15.99:0/218613797 conn(0x7fb0280fbfa0 msgr2=0x7fb0280fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 shutdown_connections 2023-08-25T08:57:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:31.427+0000 7fb0307b0700 1 -- 172.21.15.99:0/218613797 wait complete. 2023-08-25T08:57:31.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:31 smithi181 ceph-mon[131296]: pgmap v271: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2023-08-25T08:57:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:31 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/218613797' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-25T08:57:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:31 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/218613797' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-25T08:57:31.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:31 smithi099 ceph-mon[97641]: pgmap v271: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2023-08-25T08:57:31.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:31 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/218613797' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-25T08:57:31.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:31 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/218613797' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-25T08:57:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:31 smithi157 ceph-mon[130879]: pgmap v271: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2023-08-25T08:57:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:31 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/218613797' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-25T08:57:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:31 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/218613797' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-25T08:57:31.972 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:57:31.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-08-25T08:57:31.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-25T08:57:32.071 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-25T08:57:32.072 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-08-25T08:57:32.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph mgr dump --format=json 2023-08-25T08:57:32.828 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:33.683 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:33 smithi099 ceph-mon[97641]: pgmap v272: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2023-08-25T08:57:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:33 smithi157 ceph-mon[130879]: pgmap v272: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2023-08-25T08:57:33.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:33 smithi181 ceph-mon[131296]: pgmap v272: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2023-08-25T08:57:34.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3310640336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4101300 msgr2=0x7f1ec410d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:34.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3310640336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4101300 0x7f1ec410d810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00b0a0 tx=0x7f1ebc00b3b0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3310640336 shutdown_connections 2023-08-25T08:57:34.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3310640336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4101300 0x7f1ec410d810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3310640336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec4106940 0x7f1ec4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3310640336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ec4100940 0x7f1ec4100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3310640336 >> 172.21.15.99:0/3310640336 conn(0x7f1ec40fc410 msgr2=0x7f1ec40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:34.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3310640336 shutdown_connections 2023-08-25T08:57:34.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.356+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3310640336 wait complete. 2023-08-25T08:57:34.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.357+0000 7f1ec92ba700 1 Processor -- start 2023-08-25T08:57:34.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.357+0000 7f1ec92ba700 1 -- start start 2023-08-25T08:57:34.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec92ba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ec4100940 0x7f1ec4198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:34.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec92ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec4101300 0x7f1ec41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:34.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec92ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4106940 0x7f1ec419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:34.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec92ba700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec419dd20 con 0x7f1ec4106940 2023-08-25T08:57:34.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec92ba700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec419de90 con 0x7f1ec4100940 2023-08-25T08:57:34.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec92ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec419e050 con 0x7f1ec4101300 2023-08-25T08:57:34.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4106940 0x7f1ec419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:34.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4106940 0x7f1ec419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39050/0 (socket says 172.21.15.99:39050) 2023-08-25T08:57:34.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec4101300 0x7f1ec41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:34.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec37fe700 1 -- 172.21.15.99:0/3099880883 learned_addr learned my addr 172.21.15.99:0/3099880883 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:34.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec2ffd700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ec4100940 0x7f1ec4198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:34.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec37fe700 1 -- 172.21.15.99:0/3099880883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec4101300 msgr2=0x7f1ec41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:34.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec37fe700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec4101300 0x7f1ec41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec37fe700 1 -- 172.21.15.99:0/3099880883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ec4100940 msgr2=0x7f1ec4198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:34.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec37fe700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ec4100940 0x7f1ec4198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec37fe700 1 -- 172.21.15.99:0/3099880883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc009d00 con 0x7f1ec4106940 2023-08-25T08:57:34.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.358+0000 7f1ec27fc700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec4101300 0x7f1ec41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:34.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.359+0000 7f1ec37fe700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4106940 0x7f1ec419d710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009fd0 tx=0x7f1ebc00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:34.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.359+0000 7f1eabfff700 1 -- 172.21.15.99:0/3099880883 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00e040 con 0x7f1ec4106940 2023-08-25T08:57:34.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.359+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec419e2d0 con 0x7f1ec4106940 2023-08-25T08:57:34.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.359+0000 7f1eabfff700 1 -- 172.21.15.99:0/3099880883 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ebc0094c0 con 0x7f1ec4106940 2023-08-25T08:57:34.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.359+0000 7f1eabfff700 1 -- 172.21.15.99:0/3099880883 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0052a0 con 0x7f1ec4106940 2023-08-25T08:57:34.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.359+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec41a5b90 con 0x7f1ec4106940 2023-08-25T08:57:34.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.360+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec404e620 con 0x7f1ec4106940 2023-08-25T08:57:34.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.362+0000 7f1eabfff700 1 -- 172.21.15.99:0/3099880883 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f1ebc01d470 con 0x7f1ec4106940 2023-08-25T08:57:34.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.363+0000 7f1eabfff700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f1eac0a0060 0x7f1eac0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:34.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.363+0000 7f1eabfff700 1 -- 172.21.15.99:0/3099880883 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ebc0cc460 con 0x7f1ec4106940 2023-08-25T08:57:34.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.363+0000 7f1ec2ffd700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f1eac0a0060 0x7f1eac0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:34.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.364+0000 7f1ec2ffd700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f1eac0a0060 0x7f1eac0a2520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009be0 tx=0x7f1eb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:34.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.367+0000 7f1eabfff700 1 -- 172.21.15.99:0/3099880883 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1ebc0961d0 con 0x7f1ec4106940 2023-08-25T08:57:34.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.623+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1ec4110a80 con 0x7f1ec4106940 2023-08-25T08:57:34.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.628+0000 7f1eabfff700 1 -- 172.21.15.99:0/3099880883 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+258098 (secure 0 0 0) 0x7f1ebc0997f0 con 0x7f1ec4106940 2023-08-25T08:57:34.634 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:34.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f1eac0a0060 msgr2=0x7f1eac0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:34.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f1eac0a0060 0x7f1eac0a2520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009be0 tx=0x7f1eb4009260 comp rx=0 tx=0).stop 2023-08-25T08:57:34.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4106940 msgr2=0x7f1ec419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:34.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4106940 0x7f1ec419d710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009fd0 tx=0x7f1ebc00bce0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 shutdown_connections 2023-08-25T08:57:34.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ec4106940 0x7f1ec419d710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f1eac0a0060 0x7f1eac0a2520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ec4100940 0x7f1ec4198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 --2- 172.21.15.99:0/3099880883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec4101300 0x7f1ec41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:34.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 >> 172.21.15.99:0/3099880883 conn(0x7f1ec40fc410 msgr2=0x7f1ec410bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:34.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 shutdown_connections 2023-08-25T08:57:34.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:34.633+0000 7f1ec92ba700 1 -- 172.21.15.99:0/3099880883 wait complete. 2023-08-25T08:57:35.102 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4068336524},{"type":"v1","addr":"172.21.15.99:6801","nonce":4068336524}]},"active_addr":"172.21.15.99:6801/4068336524","active_change":"2023-08-25T08:50:54.351758+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.99:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3418985282}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":2892690228}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1861696476}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":90449605}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":266181746}]}]} 2023-08-25T08:57:35.108 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-08-25T08:57:35.109 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-08-25T08:57:35.109 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd dump --format=json 2023-08-25T08:57:35.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:35 smithi099 ceph-mon[97641]: pgmap v273: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2023-08-25T08:57:35.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:35 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3099880883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-25T08:57:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:35 smithi157 ceph-mon[130879]: pgmap v273: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2023-08-25T08:57:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:35 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3099880883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-25T08:57:35.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:35 smithi181 ceph-mon[131296]: pgmap v273: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2023-08-25T08:57:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:35 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3099880883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-25T08:57:35.834 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:37.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.353+0000 7f271b198700 1 -- 172.21.15.99:0/1534818419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714100940 msgr2=0x7f2714100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:37.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.353+0000 7f271b198700 1 --2- 172.21.15.99:0/1534818419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714100940 0x7f2714100dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2700009990 tx=0x7f2700009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.354+0000 7f271b198700 1 -- 172.21.15.99:0/1534818419 shutdown_connections 2023-08-25T08:57:37.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.354+0000 7f271b198700 1 --2- 172.21.15.99:0/1534818419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2714101300 0x7f271410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.354+0000 7f271b198700 1 --2- 172.21.15.99:0/1534818419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2714106940 0x7f2714106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.354+0000 7f271b198700 1 --2- 172.21.15.99:0/1534818419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714100940 0x7f2714100dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.354+0000 7f271b198700 1 -- 172.21.15.99:0/1534818419 >> 172.21.15.99:0/1534818419 conn(0x7f27140fc410 msgr2=0x7f27140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:37.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.354+0000 7f271b198700 1 -- 172.21.15.99:0/1534818419 shutdown_connections 2023-08-25T08:57:37.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.354+0000 7f271b198700 1 -- 172.21.15.99:0/1534818419 wait complete. 2023-08-25T08:57:37.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.355+0000 7f271b198700 1 Processor -- start 2023-08-25T08:57:37.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.355+0000 7f271b198700 1 -- start start 2023-08-25T08:57:37.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.355+0000 7f271b198700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2714100940 0x7f271419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:37.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.355+0000 7f271b198700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2714101300 0x7f271419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:37.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.355+0000 7f271b198700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714106940 0x7f2714198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:37.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.355+0000 7f271b198700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27141992d0 con 0x7f2714106940 2023-08-25T08:57:37.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.355+0000 7f271b198700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2714199440 con 0x7f2714101300 2023-08-25T08:57:37.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.355+0000 7f271b198700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2714199600 con 0x7f2714100940 2023-08-25T08:57:37.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714106940 0x7f2714198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:37.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714106940 0x7f2714198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44480/0 (socket says 172.21.15.99:44480) 2023-08-25T08:57:37.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 -- 172.21.15.99:0/1000157384 learned_addr learned my addr 172.21.15.99:0/1000157384 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:37.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2718f34700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2714100940 0x7f271419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:37.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2713fff700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2714101300 0x7f271419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:37.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 -- 172.21.15.99:0/1000157384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2714100940 msgr2=0x7f271419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:37.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2714100940 0x7f271419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 -- 172.21.15.99:0/1000157384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2714101300 msgr2=0x7f271419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:37.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2714101300 0x7f271419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 -- 172.21.15.99:0/1000157384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27000095f0 con 0x7f2714106940 2023-08-25T08:57:37.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2719735700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714106940 0x7f2714198b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f270c00ef30 tx=0x7f270c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:37.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.356+0000 7f2711ffb700 1 -- 172.21.15.99:0/1000157384 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c018070 con 0x7f2714106940 2023-08-25T08:57:37.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.357+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2714199880 con 0x7f2714106940 2023-08-25T08:57:37.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.357+0000 7f2711ffb700 1 -- 172.21.15.99:0/1000157384 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f270c00f040 con 0x7f2714106940 2023-08-25T08:57:37.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.357+0000 7f2711ffb700 1 -- 172.21.15.99:0/1000157384 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c00cd00 con 0x7f2714106940 2023-08-25T08:57:37.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.357+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27141a5ab0 con 0x7f2714106940 2023-08-25T08:57:37.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.358+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271404e620 con 0x7f2714106940 2023-08-25T08:57:37.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.359+0000 7f2711ffb700 1 -- 172.21.15.99:0/1000157384 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f270c003a00 con 0x7f2714106940 2023-08-25T08:57:37.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.362+0000 7f2711ffb700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f270409ffe0 0x7f27040a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:37.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.362+0000 7f2711ffb700 1 -- 172.21.15.99:0/1000157384 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f270c0cae80 con 0x7f2714106940 2023-08-25T08:57:37.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.362+0000 7f2718f34700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f270409ffe0 0x7f27040a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:37.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.362+0000 7f2718f34700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f270409ffe0 0x7f27040a24a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2708009be0 tx=0x7f2708009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:37.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.366+0000 7f2711ffb700 1 -- 172.21.15.99:0/1000157384 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f270c094bf0 con 0x7f2714106940 2023-08-25T08:57:37.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.566+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2714199fe0 con 0x7f2714106940 2023-08-25T08:57:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.568+0000 7f2711ffb700 1 -- 172.21.15.99:0/1000157384 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f270c098210 con 0x7f2714106940 2023-08-25T08:57:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:37.569 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:20.500911+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:57:37.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f270409ffe0 msgr2=0x7f27040a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:37.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f270409ffe0 0x7f27040a24a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2708009be0 tx=0x7f2708009260 comp rx=0 tx=0).stop 2023-08-25T08:57:37.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714106940 msgr2=0x7f2714198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:37.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714106940 0x7f2714198b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f270c00ef30 tx=0x7f270c00c3a0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 shutdown_connections 2023-08-25T08:57:37.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2714106940 0x7f2714198b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2714100940 0x7f271419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f270409ffe0 0x7f27040a24a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 --2- 172.21.15.99:0/1000157384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2714101300 0x7f271419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:37.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.570+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 >> 172.21.15.99:0/1000157384 conn(0x7f27140fc410 msgr2=0x7f271410bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:37.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.571+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 shutdown_connections 2023-08-25T08:57:37.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:37.571+0000 7f271b198700 1 -- 172.21.15.99:0/1000157384 wait complete. 2023-08-25T08:57:37.770 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:37 smithi099 ceph-mon[97641]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2023-08-25T08:57:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:37 smithi157 ceph-mon[130879]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2023-08-25T08:57:37.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:37 smithi181 ceph-mon[131296]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2023-08-25T08:57:38.109 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-08-25T08:57:38.109 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd dump --format=json 2023-08-25T08:57:38.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:38 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1000157384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:38.832 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:38.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:38 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1000157384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:38.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:38 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1000157384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:39.802 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:39 smithi099 ceph-mon[97641]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-08-25T08:57:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:39 smithi157 ceph-mon[130879]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-08-25T08:57:39.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:39 smithi181 ceph-mon[131296]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-08-25T08:57:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 -- 172.21.15.99:0/1931287131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc106940 msgr2=0x7f89dc106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 --2- 172.21.15.99:0/1931287131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc106940 0x7f89dc106d20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009990 tx=0x7f89cc009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 -- 172.21.15.99:0/1931287131 shutdown_connections 2023-08-25T08:57:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 --2- 172.21.15.99:0/1931287131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc101300 0x7f89dc10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 --2- 172.21.15.99:0/1931287131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc106940 0x7f89dc106d20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 --2- 172.21.15.99:0/1931287131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89dc100940 0x7f89dc100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 -- 172.21.15.99:0/1931287131 >> 172.21.15.99:0/1931287131 conn(0x7f89dc0fc410 msgr2=0x7f89dc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:40.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 -- 172.21.15.99:0/1931287131 shutdown_connections 2023-08-25T08:57:40.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.445+0000 7f89e22fa700 1 -- 172.21.15.99:0/1931287131 wait complete. 2023-08-25T08:57:40.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.446+0000 7f89e22fa700 1 Processor -- start 2023-08-25T08:57:40.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.446+0000 7f89e22fa700 1 -- start start 2023-08-25T08:57:40.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89e22fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc100940 0x7f89dc19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:40.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89e22fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89dc101300 0x7f89dc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:40.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89e22fa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc106940 0x7f89dc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:40.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89e22fa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc199260 con 0x7f89dc100940 2023-08-25T08:57:40.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89e22fa700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc1993d0 con 0x7f89dc106940 2023-08-25T08:57:40.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89e22fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc199590 con 0x7f89dc101300 2023-08-25T08:57:40.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc100940 0x7f89dc19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:40.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89dc101300 0x7f89dc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:40.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc100940 0x7f89dc19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44494/0 (socket says 172.21.15.99:44494) 2023-08-25T08:57:40.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89dbfff700 1 -- 172.21.15.99:0/675003778 learned_addr learned my addr 172.21.15.99:0/675003778 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:40.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89e0897700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc106940 0x7f89dc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:40.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89dbfff700 1 -- 172.21.15.99:0/675003778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89dc101300 msgr2=0x7f89dc19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:40.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89dbfff700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89dc101300 0x7f89dc19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89dbfff700 1 -- 172.21.15.99:0/675003778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc106940 msgr2=0x7f89dc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:40.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89dbfff700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc106940 0x7f89dc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89dbfff700 1 -- 172.21.15.99:0/675003778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0095f0 con 0x7f89dc100940 2023-08-25T08:57:40.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.447+0000 7f89e0897700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc106940 0x7f89dc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:40.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.448+0000 7f89dbfff700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc100940 0x7f89dc19eac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f89cc005910 tx=0x7f89cc005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:40.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.448+0000 7f89d97fa700 1 -- 172.21.15.99:0/675003778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc01d070 con 0x7f89dc100940 2023-08-25T08:57:40.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.448+0000 7f89d97fa700 1 -- 172.21.15.99:0/675003778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89cc022470 con 0x7f89dc100940 2023-08-25T08:57:40.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.448+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc199810 con 0x7f89dc100940 2023-08-25T08:57:40.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.448+0000 7f89d97fa700 1 -- 172.21.15.99:0/675003778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc017650 con 0x7f89dc100940 2023-08-25T08:57:40.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.448+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc1a5a40 con 0x7f89dc100940 2023-08-25T08:57:40.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.449+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc04e620 con 0x7f89dc100940 2023-08-25T08:57:40.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.451+0000 7f89d97fa700 1 -- 172.21.15.99:0/675003778 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f89cc0225e0 con 0x7f89dc100940 2023-08-25T08:57:40.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.452+0000 7f89d97fa700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f89c40a0060 0x7f89c40a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:40.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.453+0000 7f89d97fa700 1 -- 172.21.15.99:0/675003778 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89cc0cc5e0 con 0x7f89dc100940 2023-08-25T08:57:40.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.453+0000 7f89db7fe700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f89c40a0060 0x7f89c40a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:40.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.453+0000 7f89db7fe700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f89c40a0060 0x7f89c40a2520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f89dc19a2e0 tx=0x7f89d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:40.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.456+0000 7f89d97fa700 1 -- 172.21.15.99:0/675003778 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f89cc096350 con 0x7f89dc100940 2023-08-25T08:57:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.658+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89dc0008d0 con 0x7f89dc100940 2023-08-25T08:57:40.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.659+0000 7f89d97fa700 1 -- 172.21.15.99:0/675003778 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f89cc099970 con 0x7f89dc100940 2023-08-25T08:57:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:57:40.661 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:20.500911+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:57:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f89c40a0060 msgr2=0x7f89c40a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f89c40a0060 0x7f89c40a2520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f89dc19a2e0 tx=0x7f89d0009260 comp rx=0 tx=0).stop 2023-08-25T08:57:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc100940 msgr2=0x7f89dc19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:40.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc100940 0x7f89dc19eac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f89cc005910 tx=0x7f89cc005720 comp rx=0 tx=0).stop 2023-08-25T08:57:40.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 shutdown_connections 2023-08-25T08:57:40.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f89c40a0060 0x7f89c40a2520 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc106940 0x7f89dc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89dc100940 0x7f89dc19eac0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 --2- 172.21.15.99:0/675003778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89dc101300 0x7f89dc19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 >> 172.21.15.99:0/675003778 conn(0x7f89dc0fc410 msgr2=0x7f89dc10bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 shutdown_connections 2023-08-25T08:57:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:40.662+0000 7f89e22fa700 1 -- 172.21.15.99:0/675003778 wait complete. 2023-08-25T08:57:41.209 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-08-25T08:57:41.211 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-08-25T08:57:41.211 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-08-25T08:57:41.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-08-25T08:57:41.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-08-25T08:57:41.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-08-25T08:57:41.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-08-25T08:57:41.214 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-08-25T08:57:41.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-08-25T08:57:41.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-08-25T08:57:41.216 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-08-25T08:57:41.217 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-08-25T08:57:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:41 smithi099 ceph-mon[97641]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2023-08-25T08:57:41.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:41 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/675003778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:41 smithi157 ceph-mon[130879]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2023-08-25T08:57:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:41 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/675003778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:41.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:41 smithi181 ceph-mon[131296]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2023-08-25T08:57:41.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:41 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/675003778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:57:43.403 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.420 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.430 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.446 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.455 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.467 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.481 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.505 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.522 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.533 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:43 smithi099 ceph-mon[97641]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2023-08-25T08:57:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:43 smithi157 ceph-mon[130879]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2023-08-25T08:57:43.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:43 smithi181 ceph-mon[131296]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2023-08-25T08:57:45.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:45 smithi099 ceph-mon[97641]: pgmap v278: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-08-25T08:57:45.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:45 smithi157 ceph-mon[130879]: pgmap v278: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-08-25T08:57:45.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:45 smithi181 ceph-mon[131296]: pgmap v278: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-08-25T08:57:47.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:47 smithi099 ceph-mon[97641]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-08-25T08:57:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:47 smithi157 ceph-mon[130879]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-08-25T08:57:47.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:47 smithi181 ceph-mon[131296]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-08-25T08:57:48.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.680+0000 7f91e3801700 1 -- 172.21.15.99:0/4001762619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc106940 msgr2=0x7f91dc106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:48.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.680+0000 7f91e3801700 1 --2- 172.21.15.99:0/4001762619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc106940 0x7f91dc106d20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009990 tx=0x7f91d8009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:48.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.681+0000 7f91e3801700 1 -- 172.21.15.99:0/4001762619 shutdown_connections 2023-08-25T08:57:48.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.681+0000 7f91e3801700 1 --2- 172.21.15.99:0/4001762619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91dc101300 0x7f91dc10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:48.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.681+0000 7f91e3801700 1 --2- 172.21.15.99:0/4001762619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc106940 0x7f91dc106d20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:48.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.681+0000 7f91e3801700 1 --2- 172.21.15.99:0/4001762619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc100940 0x7f91dc100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:48.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.681+0000 7f91e3801700 1 -- 172.21.15.99:0/4001762619 >> 172.21.15.99:0/4001762619 conn(0x7f91dc0fc410 msgr2=0x7f91dc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:48.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.681+0000 7f91e3801700 1 -- 172.21.15.99:0/4001762619 shutdown_connections 2023-08-25T08:57:48.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.681+0000 7f91e3801700 1 -- 172.21.15.99:0/4001762619 wait complete. 2023-08-25T08:57:48.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.682+0000 7f91e3801700 1 Processor -- start 2023-08-25T08:57:48.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.682+0000 7f91e3801700 1 -- start start 2023-08-25T08:57:48.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e3801700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc100940 0x7f91dc072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:48.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e3801700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc101300 0x7f91dc073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:49.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e3801700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91dc106940 0x7f91dc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:49.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e3801700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1a4ed0 con 0x7f91dc100940 2023-08-25T08:57:49.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e3801700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1a5040 con 0x7f91dc101300 2023-08-25T08:57:49.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e3801700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1a5200 con 0x7f91dc106940 2023-08-25T08:57:49.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc100940 0x7f91dc072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:49.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc100940 0x7f91dc072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50316/0 (socket says 172.21.15.99:50316) 2023-08-25T08:57:49.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 -- 172.21.15.99:0/3840340165 learned_addr learned my addr 172.21.15.99:0/3840340165 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e1d9e700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91dc106940 0x7f91dc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e0d9c700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc101300 0x7f91dc073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 -- 172.21.15.99:0/3840340165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91dc106940 msgr2=0x7f91dc0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91dc106940 0x7f91dc0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 -- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc101300 msgr2=0x7f91dc073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc101300 0x7f91dc073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 -- 172.21.15.99:0/3840340165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80095f0 con 0x7f91dc100940 2023-08-25T08:57:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e0d9c700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc101300 0x7f91dc073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:49.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.683+0000 7f91e159d700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc100940 0x7f91dc072c20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91d80095c0 tx=0x7f91d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:49.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.684+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d801d070 con 0x7f91dc100940 2023-08-25T08:57:49.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.684+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc1a5480 con 0x7f91dc100940 2023-08-25T08:57:49.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.684+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91d8022470 con 0x7f91dc100940 2023-08-25T08:57:49.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.684+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8017650 con 0x7f91dc100940 2023-08-25T08:57:49.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.684+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc1a58a0 con 0x7f91dc100940 2023-08-25T08:57:49.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.685+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91dc04e620 con 0x7f91dc100940 2023-08-25T08:57:49.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.687+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f91d80225e0 con 0x7f91dc100940 2023-08-25T08:57:49.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.688+0000 7f91d27fc700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91c809ffe0 0x7f91c80a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:49.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.688+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91d80cd170 con 0x7f91dc100940 2023-08-25T08:57:49.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.688+0000 7f91e0d9c700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91c809ffe0 0x7f91c80a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:49.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.688+0000 7f91d27fc700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] conn(0x7f91c80a97c0 0x7f91c80abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:49.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.688+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 --> [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91c80ac290 con 0x7f91c80a97c0 2023-08-25T08:57:49.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.688+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d80cd4e0 con 0x7f91dc100940 2023-08-25T08:57:49.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.689+0000 7f91e1d9e700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] conn(0x7f91c80a97c0 0x7f91c80abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:49.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.689+0000 7f91e0d9c700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91c809ffe0 0x7f91c80a24a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f91dc074410 tx=0x7f91cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:49.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.689+0000 7f91e1d9e700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] conn(0x7f91c80a97c0 0x7f91c80abbe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:49.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.690+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 <== osd.4 v2:172.21.15.157:6808/488498023 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f91c80ac290 con 0x7f91c80a97c0 2023-08-25T08:57:49.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.707+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 --> [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f91dc110c40 con 0x7f91c80a97c0 2023-08-25T08:57:49.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.708+0000 7f91d27fc700 1 -- 172.21.15.99:0/3840340165 <== osd.4 v2:172.21.15.157:6808/488498023 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f91d400f040 con 0x7f91c80a97c0 2023-08-25T08:57:49.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.708+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] conn(0x7f91c80a97c0 msgr2=0x7f91c80abbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:49.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.708+0000 7f91e3801700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] conn(0x7f91c80a97c0 0x7f91c80abbe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:49.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.708+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91c809ffe0 msgr2=0x7f91c80a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:49.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.708+0000 7f91e3801700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91c809ffe0 0x7f91c80a24a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f91dc074410 tx=0x7f91cc009260 comp rx=0 tx=0).stop 2023-08-25T08:57:49.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.708+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc100940 msgr2=0x7f91dc072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:49.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.708+0000 7f91e3801700 1 --2- 172.21.15.99:0/3840340165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91dc100940 0x7f91dc072c20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91d80095c0 tx=0x7f91d8004730 comp rx=0 tx=0).stop 2023-08-25T08:57:49.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.708+0000 7f91e1d9e700 1 -- 172.21.15.99:0/3840340165 reap_dead start 2023-08-25T08:57:49.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.709+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 shutdown_connections 2023-08-25T08:57:49.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.709+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 >> 172.21.15.99:0/3840340165 conn(0x7f91dc0fc410 msgr2=0x7f91dc0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:49.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.709+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 shutdown_connections 2023-08-25T08:57:49.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:48.709+0000 7f91e3801700 1 -- 172.21.15.99:0/3840340165 wait complete. 2023-08-25T08:57:49.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:49 smithi099 ceph-mon[97641]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-08-25T08:57:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:49 smithi157 ceph-mon[130879]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-08-25T08:57:49.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:49 smithi181 ceph-mon[131296]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-08-25T08:57:51.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 -- 172.21.15.99:0/3071910937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184100940 msgr2=0x7f9184100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 --2- 172.21.15.99:0/3071910937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184100940 0x7f9184100dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9170009990 tx=0x7f9170009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 -- 172.21.15.99:0/3071910937 shutdown_connections 2023-08-25T08:57:51.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 --2- 172.21.15.99:0/3071910937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184101300 0x7f918410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 --2- 172.21.15.99:0/3071910937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9184106940 0x7f9184106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 --2- 172.21.15.99:0/3071910937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184100940 0x7f9184100dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 -- 172.21.15.99:0/3071910937 >> 172.21.15.99:0/3071910937 conn(0x7f91840fc410 msgr2=0x7f91840fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:51.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 -- 172.21.15.99:0/3071910937 shutdown_connections 2023-08-25T08:57:51.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.982+0000 7f918ad97700 1 -- 172.21.15.99:0/3071910937 wait complete. 2023-08-25T08:57:51.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.983+0000 7f918ad97700 1 Processor -- start 2023-08-25T08:57:51.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.983+0000 7f918ad97700 1 -- start start 2023-08-25T08:57:51.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f918ad97700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184100940 0x7f9184072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f918ad97700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9184101300 0x7f9184073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f918ad97700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184106940 0x7f91840736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f918ad97700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841a4eb0 con 0x7f9184106940 2023-08-25T08:57:51.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f918ad97700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841a5020 con 0x7f9184100940 2023-08-25T08:57:51.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f918ad97700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841a51e0 con 0x7f9184101300 2023-08-25T08:57:51.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f9189334700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184106940 0x7f91840736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f9189334700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184106940 0x7f91840736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50324/0 (socket says 172.21.15.99:50324) 2023-08-25T08:57:51.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f9189334700 1 -- 172.21.15.99:0/3854992509 learned_addr learned my addr 172.21.15.99:0/3854992509 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:51.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f9188b33700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184100940 0x7f9184072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.984+0000 7f9183fff700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9184101300 0x7f9184073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.985+0000 7f9189334700 1 -- 172.21.15.99:0/3854992509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9184101300 msgr2=0x7f9184073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.985+0000 7f9189334700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9184101300 0x7f9184073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.985+0000 7f9189334700 1 -- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184100940 msgr2=0x7f9184072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.985+0000 7f9189334700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184100940 0x7f9184072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.985+0000 7f9189334700 1 -- 172.21.15.99:0/3854992509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700095f0 con 0x7f9184106940 2023-08-25T08:57:51.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.985+0000 7f9183fff700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9184101300 0x7f9184073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:51.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.985+0000 7f9188b33700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184100940 0x7f9184072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:51.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.985+0000 7f9189334700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184106940 0x7f91840736e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f917c007ee0 tx=0x7f917c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:51.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.986+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c017070 con 0x7f9184106940 2023-08-25T08:57:51.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.986+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917c00f040 con 0x7f9184106940 2023-08-25T08:57:51.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.986+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91841a54c0 con 0x7f9184106940 2023-08-25T08:57:51.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.986+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c005580 con 0x7f9184106940 2023-08-25T08:57:51.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.986+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841a59c0 con 0x7f9184106940 2023-08-25T08:57:51.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.988+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f918404e620 con 0x7f9184106940 2023-08-25T08:57:51.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.990+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f917c0057a0 con 0x7f9184106940 2023-08-25T08:57:51.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.992+0000 7f9181ffb700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91740a0060 0x7f91740a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.992+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f917c0ca9f0 con 0x7f9184106940 2023-08-25T08:57:51.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.992+0000 7f9188b33700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91740a0060 0x7f91740a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.992+0000 7f9181ffb700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] conn(0x7f91740a9840 0x7f91740abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.992+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 --> [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91740ac310 con 0x7f91740a9840 2023-08-25T08:57:51.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.992+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f917c0cf2a0 con 0x7f9184106940 2023-08-25T08:57:51.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.993+0000 7f9183fff700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] conn(0x7f91740a9840 0x7f91740abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.993+0000 7f9188b33700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91740a0060 0x7f91740a2520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9178005950 tx=0x7f917800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:51.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.993+0000 7f9183fff700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] conn(0x7f91740a9840 0x7f91740abc60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:51.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:50.994+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 <== osd.7 v2:172.21.15.157:6816/3957644340 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f91740ac310 con 0x7f91740a9840 2023-08-25T08:57:51.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.010+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 --> [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9184110c80 con 0x7f91740a9840 2023-08-25T08:57:51.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.011+0000 7f9181ffb700 1 -- 172.21.15.99:0/3854992509 <== osd.7 v2:172.21.15.157:6816/3957644340 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f917001e040 con 0x7f91740a9840 2023-08-25T08:57:51.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] conn(0x7f91740a9840 msgr2=0x7f91740abc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] conn(0x7f91740a9840 0x7f91740abc60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91740a0060 msgr2=0x7f91740a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f91740a0060 0x7f91740a2520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9178005950 tx=0x7f917800b410 comp rx=0 tx=0).stop 2023-08-25T08:57:51.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184106940 msgr2=0x7f91840736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 --2- 172.21.15.99:0/3854992509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9184106940 0x7f91840736e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f917c007ee0 tx=0x7f917c00d3b0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f9189334700 1 -- 172.21.15.99:0/3854992509 reap_dead start 2023-08-25T08:57:51.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 shutdown_connections 2023-08-25T08:57:51.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 >> 172.21.15.99:0/3854992509 conn(0x7f91840fc410 msgr2=0x7f91840fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:51.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 shutdown_connections 2023-08-25T08:57:51.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.012+0000 7f918ad97700 1 -- 172.21.15.99:0/3854992509 wait complete. 2023-08-25T08:57:51.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 -- 172.21.15.99:0/983283089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f70106940 msgr2=0x7f7f70106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 --2- 172.21.15.99:0/983283089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f70106940 0x7f7f70106d20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009990 tx=0x7f7f60009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 -- 172.21.15.99:0/983283089 shutdown_connections 2023-08-25T08:57:51.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 --2- 172.21.15.99:0/983283089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f70101300 0x7f7f7010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 --2- 172.21.15.99:0/983283089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f70106940 0x7f7f70106d20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 --2- 172.21.15.99:0/983283089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f70100940 0x7f7f70100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 -- 172.21.15.99:0/983283089 >> 172.21.15.99:0/983283089 conn(0x7f7f700fc410 msgr2=0x7f7f700fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 -- 172.21.15.99:0/983283089 shutdown_connections 2023-08-25T08:57:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.698+0000 7f7f76a3b700 1 -- 172.21.15.99:0/983283089 wait complete. 2023-08-25T08:57:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.700+0000 7f7f76a3b700 1 Processor -- start 2023-08-25T08:57:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.700+0000 7f7f76a3b700 1 -- start start 2023-08-25T08:57:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.700+0000 7f7f76a3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f70100940 0x7f7f70072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.700+0000 7f7f76a3b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f70101300 0x7f7f70073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.700+0000 7f7f76a3b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f70106940 0x7f7f700736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.700+0000 7f7f76a3b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701a4f10 con 0x7f7f70106940 2023-08-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.700+0000 7f7f76a3b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701a5080 con 0x7f7f70101300 2023-08-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f76a3b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701a5240 con 0x7f7f70100940 2023-08-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f70100940 0x7f7f70072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f74fd8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f70106940 0x7f7f700736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f70100940 0x7f7f70072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:38542/0 (socket says 172.21.15.99:38542) 2023-08-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f67fff700 1 -- 172.21.15.99:0/1635441707 learned_addr learned my addr 172.21.15.99:0/1635441707 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:51.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f6ffff700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f70101300 0x7f7f70073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f74fd8700 1 -- 172.21.15.99:0/1635441707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f70100940 msgr2=0x7f7f70072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f74fd8700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f70100940 0x7f7f70072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f74fd8700 1 -- 172.21.15.99:0/1635441707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f70101300 msgr2=0x7f7f70073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f74fd8700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f70101300 0x7f7f70073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f74fd8700 1 -- 172.21.15.99:0/1635441707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600095f0 con 0x7f7f70106940 2023-08-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f67fff700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f70100940 0x7f7f70072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.701+0000 7f7f74fd8700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f70106940 0x7f7f700736e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f68007f10 tx=0x7f7f6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.702+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68009ea0 con 0x7f7f70106940 2023-08-25T08:57:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.702+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f701a5520 con 0x7f7f70106940 2023-08-25T08:57:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.702+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f6800f040 con 0x7f7f70106940 2023-08-25T08:57:51.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.702+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68004350 con 0x7f7f70106940 2023-08-25T08:57:51.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.702+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f701a59d0 con 0x7f7f70106940 2023-08-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.703+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f7004e620 con 0x7f7f70106940 2023-08-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.705+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f7f6800db20 con 0x7f7f70106940 2023-08-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.707+0000 7f7f6dffb700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7f5c09ffe0 0x7f7f5c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.707+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7f680cb7a0 con 0x7f7f70106940 2023-08-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.707+0000 7f7f67fff700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7f5c09ffe0 0x7f7f5c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.707+0000 7f7f6dffb700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] conn(0x7f7f5c0a97c0 0x7f7f5c0abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.707+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 --> [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f5c0ac290 con 0x7f7f5c0a97c0 2023-08-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.707+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f680cbc40 con 0x7f7f70106940 2023-08-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.707+0000 7f7f67fff700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7f5c09ffe0 0x7f7f5c0a24a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f60006010 tx=0x7f7f6000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.708+0000 7f7f6ffff700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] conn(0x7f7f5c0a97c0 0x7f7f5c0abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.708+0000 7f7f6ffff700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] conn(0x7f7f5c0a97c0 0x7f7f5c0abbe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:51.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.709+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 <== osd.2 v2:172.21.15.181:6800/4155797556 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7f5c0ac290 con 0x7f7f5c0a97c0 2023-08-25T08:57:51.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.725+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 --> [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f70110c20 con 0x7f7f5c0a97c0 2023-08-25T08:57:51.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.726+0000 7f7f6dffb700 1 -- 172.21.15.99:0/1635441707 <== osd.2 v2:172.21.15.181:6800/4155797556 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f5800bd30 con 0x7f7f5c0a97c0 2023-08-25T08:57:51.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.726+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 >> [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] conn(0x7f7f5c0a97c0 msgr2=0x7f7f5c0abbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.726+0000 7f7f76a3b700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] conn(0x7f7f5c0a97c0 0x7f7f5c0abbe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7f5c09ffe0 msgr2=0x7f7f5c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f76a3b700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7f5c09ffe0 0x7f7f5c0a24a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f60006010 tx=0x7f7f6000b540 comp rx=0 tx=0).stop 2023-08-25T08:57:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f70106940 msgr2=0x7f7f700736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f76a3b700 1 --2- 172.21.15.99:0/1635441707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f70106940 0x7f7f700736e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f68007f10 tx=0x7f7f6800d3b0 comp rx=0 tx=0).stop 2023-08-25T08:57:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f74fd8700 1 -- 172.21.15.99:0/1635441707 reap_dead start 2023-08-25T08:57:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 shutdown_connections 2023-08-25T08:57:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 >> 172.21.15.99:0/1635441707 conn(0x7f7f700fc410 msgr2=0x7f7f700fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 shutdown_connections 2023-08-25T08:57:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:51.727+0000 7f7f76a3b700 1 -- 172.21.15.99:0/1635441707 wait complete. 2023-08-25T08:57:51.749 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:51 smithi099 ceph-mon[97641]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T08:57:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:51 smithi157 ceph-mon[130879]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T08:57:51.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:51 smithi181 ceph-mon[131296]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T08:57:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.237+0000 7f74e2d1e700 1 -- 172.21.15.99:0/623167289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74dc106940 msgr2=0x7f74dc106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.237+0000 7f74e2d1e700 1 --2- 172.21.15.99:0/623167289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74dc106940 0x7f74dc106d20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009990 tx=0x7f74c8009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.238+0000 7f74e2d1e700 1 -- 172.21.15.99:0/623167289 shutdown_connections 2023-08-25T08:57:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.238+0000 7f74e2d1e700 1 --2- 172.21.15.99:0/623167289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74dc101300 0x7f74dc10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.238+0000 7f74e2d1e700 1 --2- 172.21.15.99:0/623167289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74dc106940 0x7f74dc106d20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.238+0000 7f74e2d1e700 1 --2- 172.21.15.99:0/623167289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74dc100940 0x7f74dc100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.238+0000 7f74e2d1e700 1 -- 172.21.15.99:0/623167289 >> 172.21.15.99:0/623167289 conn(0x7f74dc0fc410 msgr2=0x7f74dc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.238+0000 7f74e2d1e700 1 -- 172.21.15.99:0/623167289 shutdown_connections 2023-08-25T08:57:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.238+0000 7f74e2d1e700 1 -- 172.21.15.99:0/623167289 wait complete. 2023-08-25T08:57:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.240+0000 7f74e2d1e700 1 Processor -- start 2023-08-25T08:57:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.240+0000 7f74e2d1e700 1 -- start start 2023-08-25T08:57:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e2d1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74dc100940 0x7f74dc10a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e2d1e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74dc101300 0x7f74dc10a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e2d1e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74dc106940 0x7f74dc1b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e2d1e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc1b1d00 con 0x7f74dc100940 2023-08-25T08:57:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e2d1e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc1b1e40 con 0x7f74dc101300 2023-08-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e2d1e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc1b1f80 con 0x7f74dc106940 2023-08-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e12bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74dc106940 0x7f74dc1b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e12bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74dc106940 0x7f74dc1b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:38556/0 (socket says 172.21.15.99:38556) 2023-08-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.241+0000 7f74e12bb700 1 -- 172.21.15.99:0/3452672414 learned_addr learned my addr 172.21.15.99:0/3452672414 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.242+0000 7f74e12bb700 1 -- 172.21.15.99:0/3452672414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74dc101300 msgr2=0x7f74dc10a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.242+0000 7f74e12bb700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74dc101300 0x7f74dc10a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.242+0000 7f74e12bb700 1 -- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74dc100940 msgr2=0x7f74dc10a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.242+0000 7f74e12bb700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74dc100940 0x7f74dc10a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.242+0000 7f74e12bb700 1 -- 172.21.15.99:0/3452672414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80095f0 con 0x7f74dc106940 2023-08-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.243+0000 7f74e12bb700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74dc106940 0x7f74dc1b17c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f74d400ef30 tx=0x7f74d400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.243+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4018070 con 0x7f74dc106940 2023-08-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.244+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74d400f040 con 0x7f74dc106940 2023-08-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.244+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004020 con 0x7f74dc106940 2023-08-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.244+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc1b2200 con 0x7f74dc106940 2023-08-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.244+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc1b26b0 con 0x7f74dc106940 2023-08-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.247+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74c0000ff0 con 0x7f74dc106940 2023-08-25T08:57:52.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.248+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f74d4004ad0 con 0x7f74dc106940 2023-08-25T08:57:52.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.249+0000 7f74d9ffb700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74cc0a01b0 0x7f74cc0a2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:52.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.249+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f74d40cad80 con 0x7f74dc106940 2023-08-25T08:57:52.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.250+0000 7f74d9ffb700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] conn(0x7f74cc0a9990 0x7f74cc0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:52.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.250+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 --> [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74cc0ac460 con 0x7f74cc0a9990 2023-08-25T08:57:52.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.250+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d40cd3a0 con 0x7f74dc106940 2023-08-25T08:57:52.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.250+0000 7f74e0aba700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74cc0a01b0 0x7f74cc0a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:52.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.250+0000 7f74dbfff700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] conn(0x7f74cc0a9990 0x7f74cc0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.251+0000 7f74e0aba700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74cc0a01b0 0x7f74cc0a2670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f74c80098e0 tx=0x7f74c8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.251+0000 7f74dbfff700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] conn(0x7f74cc0a9990 0x7f74cc0abdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.251+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 <== osd.0 v2:172.21.15.99:6802/3269986106 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f74cc0ac460 con 0x7f74cc0a9990 2023-08-25T08:57:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.268+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 --> [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f74c0002d70 con 0x7f74cc0a9990 2023-08-25T08:57:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.268+0000 7f74d9ffb700 1 -- 172.21.15.99:0/3452672414 <== osd.0 v2:172.21.15.99:6802/3269986106 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f74c0002d70 con 0x7f74cc0a9990 2023-08-25T08:57:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] conn(0x7f74cc0a9990 msgr2=0x7f74cc0abdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] conn(0x7f74cc0a9990 0x7f74cc0abdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74cc0a01b0 msgr2=0x7f74cc0a2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74cc0a01b0 0x7f74cc0a2670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f74c80098e0 tx=0x7f74c8005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74dc106940 msgr2=0x7f74dc1b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 --2- 172.21.15.99:0/3452672414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74dc106940 0x7f74dc1b17c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f74d400ef30 tx=0x7f74d400c450 comp rx=0 tx=0).stop 2023-08-25T08:57:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e12bb700 1 -- 172.21.15.99:0/3452672414 reap_dead start 2023-08-25T08:57:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 shutdown_connections 2023-08-25T08:57:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 >> 172.21.15.99:0/3452672414 conn(0x7f74dc0fc410 msgr2=0x7f74dc10bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 shutdown_connections 2023-08-25T08:57:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.269+0000 7f74e2d1e700 1 -- 172.21.15.99:0/3452672414 wait complete. 2023-08-25T08:57:52.779 INFO:teuthology.orchestra.run.smithi099.stdout:219043332131 2023-08-25T08:57:52.779 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-08-25T08:57:53.001 INFO:teuthology.orchestra.run.smithi099.stdout:141733920818 2023-08-25T08:57:53.002 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-08-25T08:57:53.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 -- 172.21.15.99:0/1280358007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054106940 msgr2=0x7f0054106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 --2- 172.21.15.99:0/1280358007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054106940 0x7f0054106d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00480098f0 tx=0x7f0048009c00 comp rx=0 tx=0).stop 2023-08-25T08:57:53.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 -- 172.21.15.99:0/1280358007 shutdown_connections 2023-08-25T08:57:53.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 --2- 172.21.15.99:0/1280358007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054101300 0x7f005410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 --2- 172.21.15.99:0/1280358007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054106940 0x7f0054106d20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 --2- 172.21.15.99:0/1280358007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054100940 0x7f0054100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 -- 172.21.15.99:0/1280358007 >> 172.21.15.99:0/1280358007 conn(0x7f00540fc410 msgr2=0x7f00540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:53.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 -- 172.21.15.99:0/1280358007 shutdown_connections 2023-08-25T08:57:53.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.799+0000 7f005af14700 1 -- 172.21.15.99:0/1280358007 wait complete. 2023-08-25T08:57:53.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.800+0000 7f005af14700 1 Processor -- start 2023-08-25T08:57:53.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.800+0000 7f005af14700 1 -- start start 2023-08-25T08:57:53.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f005af14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054100940 0x7f005419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f005af14700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054101300 0x7f005419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f005af14700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054106940 0x7f00541968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f005af14700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0054196f40 con 0x7f0054106940 2023-08-25T08:57:53.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f005af14700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00541970b0 con 0x7f0054101300 2023-08-25T08:57:53.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f005af14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0054197270 con 0x7f0054100940 2023-08-25T08:57:53.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f0058cb0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054100940 0x7f005419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f00594b1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054106940 0x7f00541968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f0058cb0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054100940 0x7f005419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:38570/0 (socket says 172.21.15.99:38570) 2023-08-25T08:57:53.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f0058cb0700 1 -- 172.21.15.99:0/1917735038 learned_addr learned my addr 172.21.15.99:0/1917735038 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:53.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f0053fff700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054101300 0x7f005419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f0058cb0700 1 -- 172.21.15.99:0/1917735038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054101300 msgr2=0x7f005419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f0058cb0700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054101300 0x7f005419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f0058cb0700 1 -- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054106940 msgr2=0x7f00541968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.801+0000 7f0058cb0700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054106940 0x7f00541968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.802+0000 7f0058cb0700 1 -- 172.21.15.99:0/1917735038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c009d00 con 0x7f0054100940 2023-08-25T08:57:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.802+0000 7f00594b1700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054106940 0x7f00541968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:53.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.802+0000 7f0058cb0700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054100940 0x7f005419c850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0048009230 tx=0x7f004800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.802+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004801c070 con 0x7f0054100940 2023-08-25T08:57:53.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.802+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00480058e0 con 0x7f0054100940 2023-08-25T08:57:53.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.802+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0048009520 con 0x7f0054100940 2023-08-25T08:57:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.802+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00480038e0 con 0x7f0054100940 2023-08-25T08:57:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.802+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0054072b50 con 0x7f0054100940 2023-08-25T08:57:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.803+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f005404e620 con 0x7f0054100940 2023-08-25T08:57:53.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.805+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f0048020030 con 0x7f0054100940 2023-08-25T08:57:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.807+0000 7f0051ffb700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f00440a0060 0x7f00440a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.807+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00480cc0d0 con 0x7f0054100940 2023-08-25T08:57:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.807+0000 7f0053fff700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f00440a0060 0x7f00440a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.807+0000 7f0051ffb700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] conn(0x7f00440a9840 0x7f00440abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.807+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 --> [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00440ac310 con 0x7f00440a9840 2023-08-25T08:57:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.807+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00480ce6b0 con 0x7f0054100940 2023-08-25T08:57:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.807+0000 7f00594b1700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] conn(0x7f00440a9840 0x7f00440abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.808+0000 7f0053fff700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f00440a0060 0x7f00440a2520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f00540fed70 tx=0x7f0040009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.808+0000 7f00594b1700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] conn(0x7f00440a9840 0x7f00440abc60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.809+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 <== osd.6 v2:172.21.15.99:6818/2954530435 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f00440ac310 con 0x7f00440a9840 2023-08-25T08:57:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.827+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 --> [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f00540fd3d0 con 0x7f00440a9840 2023-08-25T08:57:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.828+0000 7f0051ffb700 1 -- 172.21.15.99:0/1917735038 <== osd.6 v2:172.21.15.99:6818/2954530435 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f00540fd3d0 con 0x7f00440a9840 2023-08-25T08:57:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.828+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] conn(0x7f00440a9840 msgr2=0x7f00440abc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.828+0000 7f005af14700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] conn(0x7f00440a9840 0x7f00440abc60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.828+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f00440a0060 msgr2=0x7f00440a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.828+0000 7f005af14700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f00440a0060 0x7f00440a2520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f00540fed70 tx=0x7f0040009260 comp rx=0 tx=0).stop 2023-08-25T08:57:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.828+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054100940 msgr2=0x7f005419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.829+0000 7f005af14700 1 --2- 172.21.15.99:0/1917735038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054100940 0x7f005419c850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0048009230 tx=0x7f004800fa00 comp rx=0 tx=0).stop 2023-08-25T08:57:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.829+0000 7f00594b1700 1 -- 172.21.15.99:0/1917735038 reap_dead start 2023-08-25T08:57:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.829+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 shutdown_connections 2023-08-25T08:57:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.829+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 >> 172.21.15.99:0/1917735038 conn(0x7f00540fc410 msgr2=0x7f00540fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.829+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 shutdown_connections 2023-08-25T08:57:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.829+0000 7f005af14700 1 -- 172.21.15.99:0/1917735038 wait complete. 2023-08-25T08:57:53.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 -- 172.21.15.99:0/1513966617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b8106940 msgr2=0x7fc1b8106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 --2- 172.21.15.99:0/1513966617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b8106940 0x7fc1b8106d20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009990 tx=0x7fc1a8009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 -- 172.21.15.99:0/1513966617 shutdown_connections 2023-08-25T08:57:53.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 --2- 172.21.15.99:0/1513966617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8101300 0x7fc1b810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 --2- 172.21.15.99:0/1513966617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b8106940 0x7fc1b8106d20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 --2- 172.21.15.99:0/1513966617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8100940 0x7fc1b8100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 -- 172.21.15.99:0/1513966617 >> 172.21.15.99:0/1513966617 conn(0x7fc1b80fc410 msgr2=0x7fc1b80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:53.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 -- 172.21.15.99:0/1513966617 shutdown_connections 2023-08-25T08:57:53.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.861+0000 7fc1bce41700 1 -- 172.21.15.99:0/1513966617 wait complete. 2023-08-25T08:57:53.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.862+0000 7fc1bce41700 1 Processor -- start 2023-08-25T08:57:53.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1bce41700 1 -- start start 2023-08-25T08:57:53.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1bce41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8100940 0x7fc1b810d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1bce41700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8101300 0x7fc1b810d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1bce41700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b8106940 0x7fc1b810a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1bce41700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b810ace0 con 0x7fc1b8101300 2023-08-25T08:57:53.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1bce41700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b810ae20 con 0x7fc1b8106940 2023-08-25T08:57:53.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1bce41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b810afe0 con 0x7fc1b8100940 2023-08-25T08:57:53.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1b5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8101300 0x7fc1b810d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1b5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8101300 0x7fc1b810d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50404/0 (socket says 172.21.15.99:50404) 2023-08-25T08:57:53.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1b5d9b700 1 -- 172.21.15.99:0/2164240886 learned_addr learned my addr 172.21.15.99:0/2164240886 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:53.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1b6d9d700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b8106940 0x7fc1b810a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.863+0000 7fc1b659c700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8100940 0x7fc1b810d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.864+0000 7fc1b5d9b700 1 -- 172.21.15.99:0/2164240886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8100940 msgr2=0x7fc1b810d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.864+0000 7fc1b5d9b700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8100940 0x7fc1b810d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.864+0000 7fc1b5d9b700 1 -- 172.21.15.99:0/2164240886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b8106940 msgr2=0x7fc1b810a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.864+0000 7fc1b5d9b700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b8106940 0x7fc1b810a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.864+0000 7fc1b5d9b700 1 -- 172.21.15.99:0/2164240886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac009520 con 0x7fc1b8101300 2023-08-25T08:57:53.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.864+0000 7fc1b659c700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b8100940 0x7fc1b810d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:53.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.864+0000 7fc1b5d9b700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8101300 0x7fc1b810d800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00eb30 tx=0x7fc1ac00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.865+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac009880 con 0x7fc1b8101300 2023-08-25T08:57:53.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.865+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1ac0044e0 con 0x7fc1b8101300 2023-08-25T08:57:53.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.865+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a80095f0 con 0x7fc1b8101300 2023-08-25T08:57:53.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.865+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac003d80 con 0x7fc1b8101300 2023-08-25T08:57:53.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.865+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b8071720 con 0x7fc1b8101300 2023-08-25T08:57:53.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.866+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1b80747a0 con 0x7fc1b8101300 2023-08-25T08:57:53.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.868+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc1ac003ee0 con 0x7fc1b8101300 2023-08-25T08:57:53.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.870+0000 7fc1a77fe700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc1a00a0060 0x7fc1a00a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.870+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc1ac014070 con 0x7fc1b8101300 2023-08-25T08:57:53.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.870+0000 7fc1b659c700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc1a00a0060 0x7fc1a00a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.870+0000 7fc1a77fe700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] conn(0x7fc1a00a9840 0x7fc1a00abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.870+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 --> [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1a00ac310 con 0x7fc1a00a9840 2023-08-25T08:57:53.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.870+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1ac0cbc20 con 0x7fc1b8101300 2023-08-25T08:57:53.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.870+0000 7fc1b6d9d700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] conn(0x7fc1a00a9840 0x7fc1a00abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.871+0000 7fc1b659c700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc1a00a0060 0x7fc1a00a2520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800b2e0 tx=0x7fc1a8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.871+0000 7fc1b6d9d700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] conn(0x7fc1a00a9840 0x7fc1a00abc60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.872+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 <== osd.3 v2:172.21.15.99:6810/2656518905 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc1a00ac310 con 0x7fc1a00a9840 2023-08-25T08:57:53.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.888+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 --> [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1b804e620 con 0x7fc1a00a9840 2023-08-25T08:57:53.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.889+0000 7fc1a77fe700 1 -- 172.21.15.99:0/2164240886 <== osd.3 v2:172.21.15.99:6810/2656518905 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc1b804e620 con 0x7fc1a00a9840 2023-08-25T08:57:53.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.889+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] conn(0x7fc1a00a9840 msgr2=0x7fc1a00abc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.889+0000 7fc1bce41700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] conn(0x7fc1a00a9840 0x7fc1a00abc60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.889+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc1a00a0060 msgr2=0x7fc1a00a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.889+0000 7fc1bce41700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc1a00a0060 0x7fc1a00a2520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800b2e0 tx=0x7fc1a8005fd0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.889+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8101300 msgr2=0x7fc1b810d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.889+0000 7fc1bce41700 1 --2- 172.21.15.99:0/2164240886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8101300 0x7fc1b810d800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00eb30 tx=0x7fc1ac00ee40 comp rx=0 tx=0).stop 2023-08-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.889+0000 7fc1b6d9d700 1 -- 172.21.15.99:0/2164240886 reap_dead start 2023-08-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.890+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 shutdown_connections 2023-08-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.890+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 >> 172.21.15.99:0/2164240886 conn(0x7fc1b80fc410 msgr2=0x7fc1b80fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.890+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 shutdown_connections 2023-08-25T08:57:53.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:52.890+0000 7fc1bce41700 1 -- 172.21.15.99:0/2164240886 wait complete. 2023-08-25T08:57:53.736 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:53 smithi099 ceph-mon[97641]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-08-25T08:57:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.841+0000 7fa9f801d700 1 -- 172.21.15.99:0/441005288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0100950 msgr2=0x7fa9f0100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.841+0000 7fa9f801d700 1 --2- 172.21.15.99:0/441005288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0100950 0x7fa9f0100dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00098f0 tx=0x7fa9e0009c00 comp rx=0 tx=0).stop 2023-08-25T08:57:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.841+0000 7fa9f801d700 1 -- 172.21.15.99:0/441005288 shutdown_connections 2023-08-25T08:57:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.841+0000 7fa9f801d700 1 --2- 172.21.15.99:0/441005288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f0101310 0x7fa9f010d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.841+0000 7fa9f801d700 1 --2- 172.21.15.99:0/441005288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f0106950 0x7fa9f0106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.841+0000 7fa9f801d700 1 --2- 172.21.15.99:0/441005288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0100950 0x7fa9f0100dd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.841+0000 7fa9f801d700 1 -- 172.21.15.99:0/441005288 >> 172.21.15.99:0/441005288 conn(0x7fa9f00fc440 msgr2=0x7fa9f00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.842+0000 7fa9f801d700 1 -- 172.21.15.99:0/441005288 shutdown_connections 2023-08-25T08:57:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.842+0000 7fa9f801d700 1 -- 172.21.15.99:0/441005288 wait complete. 2023-08-25T08:57:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.843+0000 7fa9f801d700 1 Processor -- start 2023-08-25T08:57:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.843+0000 7fa9f801d700 1 -- start start 2023-08-25T08:57:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.844+0000 7fa9f801d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f0100950 0x7fa9f006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.844+0000 7fa9f801d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f0101310 0x7fa9f006b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.844+0000 7fa9f801d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0106950 0x7fa9f006b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.844+0000 7fa9f801d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00730f0 con 0x7fa9f0100950 2023-08-25T08:57:53.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.844+0000 7fa9f801d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0073260 con 0x7fa9f0106950 2023-08-25T08:57:53.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.844+0000 7fa9f801d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0073420 con 0x7fa9f0101310 2023-08-25T08:57:53.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f0101310 0x7fa9f006b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f0101310 0x7fa9f006b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:38602/0 (socket says 172.21.15.99:38602) 2023-08-25T08:57:53.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 -- 172.21.15.99:0/4093927582 learned_addr learned my addr 172.21.15.99:0/4093927582 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:53.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f65ba700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0106950 0x7fa9f006b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 -- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0106950 msgr2=0x7fa9f006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0106950 0x7fa9f006b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 -- 172.21.15.99:0/4093927582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f0100950 msgr2=0x7fa9f006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f0100950 0x7fa9f006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 -- 172.21.15.99:0/4093927582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0009520 con 0x7fa9f0101310 2023-08-25T08:57:53.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f65ba700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0106950 0x7fa9f006b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:57:53.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.845+0000 7fa9f55b8700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f0101310 0x7fa9f006b1d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009810 tx=0x7fa9e000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.846+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e001c070 con 0x7fa9f0101310 2023-08-25T08:57:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.846+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9e0005040 con 0x7fa9f0101310 2023-08-25T08:57:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.846+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0004ac0 con 0x7fa9f0101310 2023-08-25T08:57:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.846+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f00736a0 con 0x7fa9f0101310 2023-08-25T08:57:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.846+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f0073af0 con 0x7fa9f0101310 2023-08-25T08:57:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.847+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9f004e620 con 0x7fa9f0101310 2023-08-25T08:57:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.850+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa9e0020070 con 0x7fa9f0101310 2023-08-25T08:57:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.852+0000 7fa9e6ffd700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9dc0a0060 0x7fa9dc0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.852+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa9e00cc8f0 con 0x7fa9f0101310 2023-08-25T08:57:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.852+0000 7fa9f5db9700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9dc0a0060 0x7fa9dc0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.853+0000 7fa9e6ffd700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] conn(0x7fa9dc0a9840 0x7fa9dc0abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.853+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 --> [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9dc0ac310 con 0x7fa9dc0a9840 2023-08-25T08:57:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.853+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9e00ccc60 con 0x7fa9f0101310 2023-08-25T08:57:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.853+0000 7fa9f65ba700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] conn(0x7fa9dc0a9840 0x7fa9dc0abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:53.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.853+0000 7fa9f5db9700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9dc0a0060 0x7fa9dc0a2520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0097b0 tx=0x7fa9ec006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.854+0000 7fa9f65ba700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] conn(0x7fa9dc0a9840 0x7fa9dc0abc60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:53.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.855+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 <== osd.1 v2:172.21.15.157:6800/2722577134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa9dc0ac310 con 0x7fa9dc0a9840 2023-08-25T08:57:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:53 smithi157 ceph-mon[130879]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-08-25T08:57:53.888 INFO:teuthology.orchestra.run.smithi099.stdout:34359738437 2023-08-25T08:57:53.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-08-25T08:57:53.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.889+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 --> [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9f0110c80 con 0x7fa9dc0a9840 2023-08-25T08:57:53.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.889+0000 7fa9e6ffd700 1 -- 172.21.15.99:0/4093927582 <== osd.1 v2:172.21.15.157:6800/2722577134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa9e800f040 con 0x7fa9dc0a9840 2023-08-25T08:57:53.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] conn(0x7fa9dc0a9840 msgr2=0x7fa9dc0abc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] conn(0x7fa9dc0a9840 0x7fa9dc0abc60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:53.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9dc0a0060 msgr2=0x7fa9dc0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9dc0a0060 0x7fa9dc0a2520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0097b0 tx=0x7fa9ec006d20 comp rx=0 tx=0).stop 2023-08-25T08:57:53.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f0101310 msgr2=0x7fa9f006b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:53.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 --2- 172.21.15.99:0/4093927582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f0101310 0x7fa9f006b1d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009810 tx=0x7fa9e000fa70 comp rx=0 tx=0).stop 2023-08-25T08:57:53.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f65ba700 1 -- 172.21.15.99:0/4093927582 reap_dead start 2023-08-25T08:57:53.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 shutdown_connections 2023-08-25T08:57:53.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 >> 172.21.15.99:0/4093927582 conn(0x7fa9f00fc440 msgr2=0x7fa9f00fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:53.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.890+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 shutdown_connections 2023-08-25T08:57:53.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:53.891+0000 7fa9f801d700 1 -- 172.21.15.99:0/4093927582 wait complete. 2023-08-25T08:57:53.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:53 smithi181 ceph-mon[131296]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-08-25T08:57:54.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.100+0000 7ff996e3a700 1 -- 172.21.15.99:0/3587301930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990106950 msgr2=0x7ff990106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.100+0000 7ff996e3a700 1 --2- 172.21.15.99:0/3587301930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990106950 0x7ff990106d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff984009990 tx=0x7ff984009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.100+0000 7ff996e3a700 1 -- 172.21.15.99:0/3587301930 shutdown_connections 2023-08-25T08:57:54.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.100+0000 7ff996e3a700 1 --2- 172.21.15.99:0/3587301930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990101310 0x7ff99010d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.100+0000 7ff996e3a700 1 --2- 172.21.15.99:0/3587301930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990106950 0x7ff990106d30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.100+0000 7ff996e3a700 1 --2- 172.21.15.99:0/3587301930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff990100950 0x7ff990100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.100+0000 7ff996e3a700 1 -- 172.21.15.99:0/3587301930 >> 172.21.15.99:0/3587301930 conn(0x7ff9900fc440 msgr2=0x7ff9900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:54.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.101+0000 7ff996e3a700 1 -- 172.21.15.99:0/3587301930 shutdown_connections 2023-08-25T08:57:54.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.101+0000 7ff996e3a700 1 -- 172.21.15.99:0/3587301930 wait complete. 2023-08-25T08:57:54.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.102+0000 7ff996e3a700 1 Processor -- start 2023-08-25T08:57:54.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.102+0000 7ff996e3a700 1 -- start start 2023-08-25T08:57:54.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.103+0000 7ff996e3a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990100950 0x7ff990072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.103+0000 7ff996e3a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff990101310 0x7ff990073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.103+0000 7ff996e3a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990106950 0x7ff9900736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.103+0000 7ff996e3a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a4e90 con 0x7ff990101310 2023-08-25T08:57:54.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.103+0000 7ff996e3a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a5000 con 0x7ff990100950 2023-08-25T08:57:54.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.103+0000 7ff996e3a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a51c0 con 0x7ff990106950 2023-08-25T08:57:54.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.103+0000 7ff98ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff990101310 0x7ff990073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff98ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff990101310 0x7ff990073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50444/0 (socket says 172.21.15.99:50444) 2023-08-25T08:57:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff98ffff700 1 -- 172.21.15.99:0/2779777360 learned_addr learned my addr 172.21.15.99:0/2779777360 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff9953d7700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990106950 0x7ff9900736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff9953d7700 1 -- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990100950 msgr2=0x7ff990072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff994bd6700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990100950 0x7ff990072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff9953d7700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990100950 0x7ff990072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff9953d7700 1 -- 172.21.15.99:0/2779777360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff990101310 msgr2=0x7ff990073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff9953d7700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff990101310 0x7ff990073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff9953d7700 1 -- 172.21.15.99:0/2779777360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9840095f0 con 0x7ff990106950 2023-08-25T08:57:54.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.104+0000 7ff994bd6700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990100950 0x7ff990072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:57:54.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.105+0000 7ff9953d7700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990106950 0x7ff9900736e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff98800efc0 tx=0x7ff98800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.105+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988018070 con 0x7ff990106950 2023-08-25T08:57:54.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.105+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9901a5440 con 0x7ff990106950 2023-08-25T08:57:54.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.105+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98800f040 con 0x7ff990106950 2023-08-25T08:57:54.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.105+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800cdf0 con 0x7ff990106950 2023-08-25T08:57:54.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.105+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9901a5940 con 0x7ff990106950 2023-08-25T08:57:54.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.106+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff99004e620 con 0x7ff990106950 2023-08-25T08:57:54.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.109+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ff988007500 con 0x7ff990106950 2023-08-25T08:57:54.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.111+0000 7ff98dffb700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9800a0060 0x7ff9800a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.112+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff9880cada0 con 0x7ff990106950 2023-08-25T08:57:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.112+0000 7ff994bd6700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9800a0060 0x7ff9800a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.112+0000 7ff98dffb700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] conn(0x7ff9800a9840 0x7ff9800abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.112+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 --> [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9800ac310 con 0x7ff9800a9840 2023-08-25T08:57:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.112+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9880cd4b0 con 0x7ff990106950 2023-08-25T08:57:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.112+0000 7ff98ffff700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] conn(0x7ff9800a9840 0x7ff9800abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.113+0000 7ff994bd6700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9800a0060 0x7ff9800a2520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b2e0 tx=0x7ff984005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.113+0000 7ff98ffff700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] conn(0x7ff9800a9840 0x7ff9800abc60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.114+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 <== osd.10 v2:172.21.15.157:6824/3880870693 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff9800ac310 con 0x7ff9800a9840 2023-08-25T08:57:54.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.131+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 --> [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff990110c30 con 0x7ff9800a9840 2023-08-25T08:57:54.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff98dffb700 1 -- 172.21.15.99:0/2779777360 <== osd.10 v2:172.21.15.157:6824/3880870693 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff97c009e90 con 0x7ff9800a9840 2023-08-25T08:57:54.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] conn(0x7ff9800a9840 msgr2=0x7ff9800abc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] conn(0x7ff9800a9840 0x7ff9800abc60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9800a0060 msgr2=0x7ff9800a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9800a0060 0x7ff9800a2520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b2e0 tx=0x7ff984005fb0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990106950 msgr2=0x7ff9900736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 --2- 172.21.15.99:0/2779777360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990106950 0x7ff9900736e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff98800efc0 tx=0x7ff98800c3a0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff9953d7700 1 -- 172.21.15.99:0/2779777360 reap_dead start 2023-08-25T08:57:54.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 shutdown_connections 2023-08-25T08:57:54.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 >> 172.21.15.99:0/2779777360 conn(0x7ff9900fc440 msgr2=0x7ff9900fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:54.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 shutdown_connections 2023-08-25T08:57:54.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.132+0000 7ff996e3a700 1 -- 172.21.15.99:0/2779777360 wait complete. 2023-08-25T08:57:54.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 -- 172.21.15.99:0/297450052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c072d50 msgr2=0x7fe78c073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 --2- 172.21.15.99:0/297450052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c072d50 0x7fe78c073130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe788009990 tx=0x7fe788009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 -- 172.21.15.99:0/297450052 shutdown_connections 2023-08-25T08:57:54.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 --2- 172.21.15.99:0/297450052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe78c10d180 0x7fe78c10f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 --2- 172.21.15.99:0/297450052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c072d50 0x7fe78c073130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 --2- 172.21.15.99:0/297450052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe78c073670 0x7fe78c10cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 -- 172.21.15.99:0/297450052 >> 172.21.15.99:0/297450052 conn(0x7fe78c0fc410 msgr2=0x7fe78c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:54.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 -- 172.21.15.99:0/297450052 shutdown_connections 2023-08-25T08:57:54.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.041+0000 7fe793925700 1 -- 172.21.15.99:0/297450052 wait complete. 2023-08-25T08:57:54.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe793925700 1 Processor -- start 2023-08-25T08:57:54.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe793925700 1 -- start start 2023-08-25T08:57:54.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe793925700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe78c072d50 0x7fe78c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe793925700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c073670 0x7fe78c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe793925700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe78c10d180 0x7fe78c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe793925700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c199150 con 0x7fe78c10d180 2023-08-25T08:57:54.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe793925700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c1992c0 con 0x7fe78c072d50 2023-08-25T08:57:54.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe793925700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c199480 con 0x7fe78c073670 2023-08-25T08:57:54.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe790ec0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c073670 0x7fe78c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe7916c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe78c072d50 0x7fe78c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe7916c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe78c072d50 0x7fe78c19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51094/0 (socket says 172.21.15.99:51094) 2023-08-25T08:57:54.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe7916c1700 1 -- 172.21.15.99:0/3865287772 learned_addr learned my addr 172.21.15.99:0/3865287772 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:54.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.046+0000 7fe791ec2700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe78c10d180 0x7fe78c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7916c1700 1 -- 172.21.15.99:0/3865287772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c073670 msgr2=0x7fe78c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7916c1700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c073670 0x7fe78c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7916c1700 1 -- 172.21.15.99:0/3865287772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe78c10d180 msgr2=0x7fe78c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7916c1700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe78c10d180 0x7fe78c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7916c1700 1 -- 172.21.15.99:0/3865287772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7880095f0 con 0x7fe78c072d50 2023-08-25T08:57:54.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7916c1700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe78c072d50 0x7fe78c19ea60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe788005910 tx=0x7fe788004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78801d070 con 0x7fe78c072d50 2023-08-25T08:57:54.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe788022470 con 0x7fe78c072d50 2023-08-25T08:57:54.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788017650 con 0x7fe78c072d50 2023-08-25T08:57:54.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c199700 con 0x7fe78c072d50 2023-08-25T08:57:54.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c1009a0 con 0x7fe78c072d50 2023-08-25T08:57:54.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.048+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe78c112df0 con 0x7fe78c072d50 2023-08-25T08:57:54.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.053+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fe7880225e0 con 0x7fe78c072d50 2023-08-25T08:57:54.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.056+0000 7fe7827fc700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe7780a0060 0x7fe7780a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.056+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7880cc290 con 0x7fe78c072d50 2023-08-25T08:57:54.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.057+0000 7fe790ec0700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe7780a0060 0x7fe7780a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.057+0000 7fe7827fc700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] conn(0x7fe7780a9840 0x7fe7780abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.057+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 --> [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7780ac310 con 0x7fe7780a9840 2023-08-25T08:57:54.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.057+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7880cc730 con 0x7fe78c072d50 2023-08-25T08:57:54.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.057+0000 7fe790ec0700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe7780a0060 0x7fe7780a2520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe78c10e820 tx=0x7fe77c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.058+0000 7fe791ec2700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] conn(0x7fe7780a9840 0x7fe7780abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.059+0000 7fe791ec2700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] conn(0x7fe7780a9840 0x7fe7780abc60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.060+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 <== osd.5 v2:172.21.15.181:6808/1806112824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe7780ac310 con 0x7fe7780a9840 2023-08-25T08:57:54.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.077+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 --> [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe78c0fe350 con 0x7fe7780a9840 2023-08-25T08:57:54.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe7827fc700 1 -- 172.21.15.99:0/3865287772 <== osd.5 v2:172.21.15.181:6808/1806112824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe78c0fe350 con 0x7fe7780a9840 2023-08-25T08:57:54.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 >> [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] conn(0x7fe7780a9840 msgr2=0x7fe7780abc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe793925700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] conn(0x7fe7780a9840 0x7fe7780abc60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe7780a0060 msgr2=0x7fe7780a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe793925700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fe7780a0060 0x7fe7780a2520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe78c10e820 tx=0x7fe77c009260 comp rx=0 tx=0).stop 2023-08-25T08:57:54.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe78c072d50 msgr2=0x7fe78c19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe793925700 1 --2- 172.21.15.99:0/3865287772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe78c072d50 0x7fe78c19ea60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe788005910 tx=0x7fe788004710 comp rx=0 tx=0).stop 2023-08-25T08:57:54.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe791ec2700 1 -- 172.21.15.99:0/3865287772 reap_dead start 2023-08-25T08:57:54.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 shutdown_connections 2023-08-25T08:57:54.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.078+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 >> 172.21.15.99:0/3865287772 conn(0x7fe78c0fc410 msgr2=0x7fe78c10f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:54.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.079+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 shutdown_connections 2023-08-25T08:57:54.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.079+0000 7fe793925700 1 -- 172.21.15.99:0/3865287772 wait complete. 2023-08-25T08:57:54.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 -- 172.21.15.99:0/3738638821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad0105390 msgr2=0x7f2ad0105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 --2- 172.21.15.99:0/3738638821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad0105390 0x7f2ad0105810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40099c0 tx=0x7f2ac4009cd0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 -- 172.21.15.99:0/3738638821 shutdown_connections 2023-08-25T08:57:54.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 --2- 172.21.15.99:0/3738638821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad0105d50 0x7f2ad010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 --2- 172.21.15.99:0/3738638821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad00688e0 0x7f2ad0068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 --2- 172.21.15.99:0/3738638821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad0105390 0x7f2ad0105810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 -- 172.21.15.99:0/3738638821 >> 172.21.15.99:0/3738638821 conn(0x7f2ad0076920 msgr2=0x7f2ad0076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 -- 172.21.15.99:0/3738638821 shutdown_connections 2023-08-25T08:57:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.336+0000 7f2ad59cc700 1 -- 172.21.15.99:0/3738638821 wait complete. 2023-08-25T08:57:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.337+0000 7f2ad59cc700 1 Processor -- start 2023-08-25T08:57:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.337+0000 7f2ad59cc700 1 -- start start 2023-08-25T08:57:54.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.337+0000 7f2ad59cc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad00688e0 0x7f2ad00ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.337+0000 7f2ad59cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad0105390 0x7f2ad00ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ad59cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad0105d50 0x7f2ad01a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ad59cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0100700 con 0x7f2ad0105d50 2023-08-25T08:57:54.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ad59cc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01a51e0 con 0x7f2ad00688e0 2023-08-25T08:57:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ad59cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01a5350 con 0x7f2ad0105390 2023-08-25T08:57:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad0105390 0x7f2ad00ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad0105390 0x7f2ad00ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:38654/0 (socket says 172.21.15.99:38654) 2023-08-25T08:57:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1911567255 learned_addr learned my addr 172.21.15.99:0/1911567255 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2acf7fe700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad0105d50 0x7f2ad01a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2aceffd700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad00688e0 0x7f2ad00ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1911567255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad00688e0 msgr2=0x7f2ad00ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad00688e0 0x7f2ad00ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad0105d50 msgr2=0x7f2ad01a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad0105d50 0x7f2ad01a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1911567255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40095f0 con 0x7f2ad0105390 2023-08-25T08:57:54.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2acf7fe700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad0105d50 0x7f2ad01a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:57:54.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.338+0000 7f2ace7fc700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad0105390 0x7f2ad00ffe10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40098e0 tx=0x7f2ac400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.339+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac401d070 con 0x7f2ad0105390 2023-08-25T08:57:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.339+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad01a5570 con 0x7f2ad0105390 2023-08-25T08:57:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.339+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac4005030 con 0x7f2ad0105390 2023-08-25T08:57:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.339+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40049c0 con 0x7f2ad0105390 2023-08-25T08:57:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.339+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad01a5980 con 0x7f2ad0105390 2023-08-25T08:57:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.340+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ad0110ba0 con 0x7f2ad0105390 2023-08-25T08:57:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.342+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f2ac40051a0 con 0x7f2ad0105390 2023-08-25T08:57:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.343+0000 7f2ad49ca700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2ab809ffe0 0x7f2ab80a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.343+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2ac40cc730 con 0x7f2ad0105390 2023-08-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.343+0000 7f2aceffd700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2ab809ffe0 0x7f2ab80a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.343+0000 7f2ad49ca700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] conn(0x7f2ab80a97c0 0x7f2ab80abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.343+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 --> [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ab80ac290 con 0x7f2ab80a97c0 2023-08-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.343+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac40ccaa0 con 0x7f2ad0105390 2023-08-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.344+0000 7f2acf7fe700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] conn(0x7f2ab80a97c0 0x7f2ab80abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.344+0000 7f2aceffd700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2ab809ffe0 0x7f2ab80a24a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009c60 tx=0x7f2ac0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.344+0000 7f2acf7fe700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] conn(0x7f2ab80a97c0 0x7f2ab80abbe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.345+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 <== osd.8 v2:172.21.15.181:6816/411715472 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2ab80ac290 con 0x7f2ab80a97c0 2023-08-25T08:57:54.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.361+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 --> [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2ad004e620 con 0x7f2ab80a97c0 2023-08-25T08:57:54.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.362+0000 7f2ad49ca700 1 -- 172.21.15.99:0/1911567255 <== osd.8 v2:172.21.15.181:6816/411715472 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2ad004e620 con 0x7f2ab80a97c0 2023-08-25T08:57:54.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.362+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 >> [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] conn(0x7f2ab80a97c0 msgr2=0x7f2ab80abbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.362+0000 7f2ad59cc700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] conn(0x7f2ab80a97c0 0x7f2ab80abbe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.362+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2ab809ffe0 msgr2=0x7f2ab80a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.362+0000 7f2ad59cc700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f2ab809ffe0 0x7f2ab80a24a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009c60 tx=0x7f2ac0009260 comp rx=0 tx=0).stop 2023-08-25T08:57:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.362+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad0105390 msgr2=0x7f2ad00ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.362+0000 7f2ad59cc700 1 --2- 172.21.15.99:0/1911567255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad0105390 0x7f2ad00ffe10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40098e0 tx=0x7f2ac400f690 comp rx=0 tx=0).stop 2023-08-25T08:57:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.362+0000 7f2acf7fe700 1 -- 172.21.15.99:0/1911567255 reap_dead start 2023-08-25T08:57:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.363+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 shutdown_connections 2023-08-25T08:57:54.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.363+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 >> 172.21.15.99:0/1911567255 conn(0x7f2ad0076920 msgr2=0x7f2ad00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:54.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.363+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 shutdown_connections 2023-08-25T08:57:54.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.363+0000 7f2ad59cc700 1 -- 172.21.15.99:0/1911567255 wait complete. 2023-08-25T08:57:54.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 -- 172.21.15.99:0/2235777074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40101300 msgr2=0x7f9b4010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 --2- 172.21.15.99:0/2235777074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40101300 0x7f9b4010d810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800b0a0 tx=0x7f9b3800b3b0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 -- 172.21.15.99:0/2235777074 shutdown_connections 2023-08-25T08:57:54.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 --2- 172.21.15.99:0/2235777074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40101300 0x7f9b4010d810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 --2- 172.21.15.99:0/2235777074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b40106940 0x7f9b40106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 --2- 172.21.15.99:0/2235777074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b40100940 0x7f9b40100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 -- 172.21.15.99:0/2235777074 >> 172.21.15.99:0/2235777074 conn(0x7f9b400fc410 msgr2=0x7f9b400fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:54.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 -- 172.21.15.99:0/2235777074 shutdown_connections 2023-08-25T08:57:54.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.803+0000 7f9b4776c700 1 -- 172.21.15.99:0/2235777074 wait complete. 2023-08-25T08:57:54.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.804+0000 7f9b4776c700 1 Processor -- start 2023-08-25T08:57:54.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.804+0000 7f9b4776c700 1 -- start start 2023-08-25T08:57:54.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b4776c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40100940 0x7f9b40198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b4776c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b40101300 0x7f9b40199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b4776c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b40106940 0x7f9b4019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b4776c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4019dcf0 con 0x7f9b40100940 2023-08-25T08:57:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b4776c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4019de60 con 0x7f9b40106940 2023-08-25T08:57:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b4776c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4019e020 con 0x7f9b40101300 2023-08-25T08:57:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45508700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40100940 0x7f9b40198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45508700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40100940 0x7f9b40198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50498/0 (socket says 172.21.15.99:50498) 2023-08-25T08:57:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45508700 1 -- 172.21.15.99:0/1669806268 learned_addr learned my addr 172.21.15.99:0/1669806268 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b44d07700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b40101300 0x7f9b40199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45d09700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b40106940 0x7f9b4019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45508700 1 -- 172.21.15.99:0/1669806268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b40101300 msgr2=0x7f9b40199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45508700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b40101300 0x7f9b40199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45508700 1 -- 172.21.15.99:0/1669806268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b40106940 msgr2=0x7f9b4019d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45508700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b40106940 0x7f9b4019d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.805+0000 7f9b45508700 1 -- 172.21.15.99:0/1669806268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38009d00 con 0x7f9b40100940 2023-08-25T08:57:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.806+0000 7f9b45508700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40100940 0x7f9b40198d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e9a0 tx=0x7f9b3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.806+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c009790 con 0x7f9b40100940 2023-08-25T08:57:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.806+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4019e300 con 0x7f9b40100940 2023-08-25T08:57:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.806+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b3c00cbd0 con 0x7f9b40100940 2023-08-25T08:57:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.806+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c018680 con 0x7f9b40100940 2023-08-25T08:57:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.806+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b401a5b60 con 0x7f9b40100940 2023-08-25T08:57:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.807+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b40110c80 con 0x7f9b40100940 2023-08-25T08:57:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.809+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f9b3c00cd40 con 0x7f9b40100940 2023-08-25T08:57:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.810+0000 7f9b367fc700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9b2c0a0060 0x7f9b2c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.810+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9b3c014070 con 0x7f9b40100940 2023-08-25T08:57:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.810+0000 7f9b44d07700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9b2c0a0060 0x7f9b2c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.811+0000 7f9b367fc700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] conn(0x7f9b2c0a9840 0x7f9b2c0abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.811+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 --> [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b2c0ac310 con 0x7f9b2c0a9840 2023-08-25T08:57:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.811+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b3c0cbbe0 con 0x7f9b40100940 2023-08-25T08:57:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.811+0000 7f9b44d07700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9b2c0a0060 0x7f9b2c0a2520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b4019a350 tx=0x7f9b3000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.811+0000 7f9b45d09700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] conn(0x7f9b2c0a9840 0x7f9b2c0abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:54.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.811+0000 7f9b45d09700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] conn(0x7f9b2c0a9840 0x7f9b2c0abc60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:54.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.812+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 <== osd.11 v2:172.21.15.181:6824/1448660331 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9b2c0ac310 con 0x7f9b2c0a9840 2023-08-25T08:57:54.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.829+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 --> [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9b4004e620 con 0x7f9b2c0a9840 2023-08-25T08:57:54.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.830+0000 7f9b367fc700 1 -- 172.21.15.99:0/1669806268 <== osd.11 v2:172.21.15.181:6824/1448660331 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9b4004e620 con 0x7f9b2c0a9840 2023-08-25T08:57:54.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.830+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 >> [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] conn(0x7f9b2c0a9840 msgr2=0x7f9b2c0abc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.830+0000 7f9b4776c700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] conn(0x7f9b2c0a9840 0x7f9b2c0abc60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.830+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9b2c0a0060 msgr2=0x7f9b2c0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.830+0000 7f9b4776c700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9b2c0a0060 0x7f9b2c0a2520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b4019a350 tx=0x7f9b3000b410 comp rx=0 tx=0).stop 2023-08-25T08:57:54.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.831+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40100940 msgr2=0x7f9b40198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:54.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.831+0000 7f9b4776c700 1 --2- 172.21.15.99:0/1669806268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b40100940 0x7f9b40198d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e9a0 tx=0x7f9b3c00ecb0 comp rx=0 tx=0).stop 2023-08-25T08:57:54.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.831+0000 7f9b45d09700 1 -- 172.21.15.99:0/1669806268 reap_dead start 2023-08-25T08:57:54.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.831+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 shutdown_connections 2023-08-25T08:57:54.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.831+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 >> 172.21.15.99:0/1669806268 conn(0x7f9b400fc410 msgr2=0x7f9b4010bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:54.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.831+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 shutdown_connections 2023-08-25T08:57:54.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:54.831+0000 7f9b4776c700 1 -- 172.21.15.99:0/1669806268 wait complete. 2023-08-25T08:57:54.980 INFO:teuthology.orchestra.run.smithi099.stdout:55834574912 2023-08-25T08:57:54.980 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-08-25T08:57:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-08-25T08:57:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-08-25T08:57:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2023-08-25T08:57:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2023-08-25T08:57:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-08-25T08:57:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-08-25T08:57:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-08-25T08:57:55.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2023-08-25T08:57:55.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2023-08-25T08:57:55.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-08-25T08:57:55.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:55 smithi099 ceph-mon[97641]: pgmap v283: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-08-25T08:57:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-08-25T08:57:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-08-25T08:57:55.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2023-08-25T08:57:55.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2023-08-25T08:57:55.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-08-25T08:57:55.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-08-25T08:57:55.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-08-25T08:57:55.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2023-08-25T08:57:55.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2023-08-25T08:57:55.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-08-25T08:57:55.381 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:55 smithi157 ceph-mon[130879]: pgmap v283: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-08-25T08:57:55.387 INFO:teuthology.orchestra.run.smithi099.stdout:77309411386 2023-08-25T08:57:55.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-08-25T08:57:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-08-25T08:57:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-08-25T08:57:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2023-08-25T08:57:55.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2023-08-25T08:57:55.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-08-25T08:57:55.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-08-25T08:57:55.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-08-25T08:57:55.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2023-08-25T08:57:55.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2023-08-25T08:57:55.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-08-25T08:57:55.496 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:55 smithi181 ceph-mon[131296]: pgmap v283: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-08-25T08:57:56.056 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-08-25T08:57:56.056 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-08-25T08:57:56.057 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2023-08-25T08:57:56.057 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2023-08-25T08:57:56.057 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-08-25T08:57:56.057 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-08-25T08:57:56.057 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-08-25T08:57:56.057 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2023-08-25T08:57:56.057 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2023-08-25T08:57:56.058 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-08-25T08:57:56.058 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:56 smithi099 ceph-mon[97641]: osdmap e84: 12 total, 12 up, 12 in 2023-08-25T08:57:56.310 INFO:teuthology.orchestra.run.smithi099.stdout:335007449108 2023-08-25T08:57:56.310 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-08-25T08:57:56.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-08-25T08:57:56.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-08-25T08:57:56.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2023-08-25T08:57:56.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2023-08-25T08:57:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-08-25T08:57:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-08-25T08:57:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-08-25T08:57:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2023-08-25T08:57:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2023-08-25T08:57:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-08-25T08:57:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:56 smithi157 ceph-mon[130879]: osdmap e84: 12 total, 12 up, 12 in 2023-08-25T08:57:56.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-08-25T08:57:56.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-08-25T08:57:56.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2023-08-25T08:57:56.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2023-08-25T08:57:56.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-08-25T08:57:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-08-25T08:57:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-08-25T08:57:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2023-08-25T08:57:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2023-08-25T08:57:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-08-25T08:57:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:56 smithi181 ceph-mon[131296]: osdmap e84: 12 total, 12 up, 12 in 2023-08-25T08:57:56.803 INFO:teuthology.orchestra.run.smithi099.stdout:249108103201 2023-08-25T08:57:56.803 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-08-25T08:57:56.814 INFO:teuthology.orchestra.run.smithi099.stdout:107374182453 2023-08-25T08:57:56.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-08-25T08:57:56.816 INFO:teuthology.orchestra.run.smithi099.stdout:304942678038 2023-08-25T08:57:56.817 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-08-25T08:57:56.935 INFO:teuthology.orchestra.run.smithi099.stdout:197568495657 2023-08-25T08:57:56.936 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-08-25T08:57:57.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:57 smithi099 ceph-mon[97641]: osdmap e85: 12 total, 12 up, 12 in 2023-08-25T08:57:57.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:57 smithi099 ceph-mon[97641]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-08-25T08:57:57.355 INFO:teuthology.orchestra.run.smithi099.stdout:163208757295 2023-08-25T08:57:57.356 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-08-25T08:57:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:57 smithi157 ceph-mon[130879]: osdmap e85: 12 total, 12 up, 12 in 2023-08-25T08:57:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:57 smithi157 ceph-mon[130879]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-08-25T08:57:57.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.410+0000 7f7120914700 1 -- 172.21.15.99:0/1584105273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c0735e0 msgr2=0x7f711c111410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:57.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.410+0000 7f7120914700 1 --2- 172.21.15.99:0/1584105273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c0735e0 0x7f711c111410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f710c009990 tx=0x7f710c009ca0 comp rx=0 tx=0).stop 2023-08-25T08:57:57.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.413+0000 7f7120914700 1 -- 172.21.15.99:0/1584105273 shutdown_connections 2023-08-25T08:57:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.413+0000 7f7120914700 1 --2- 172.21.15.99:0/1584105273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f711c111950 0x7f711c113d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.413+0000 7f7120914700 1 --2- 172.21.15.99:0/1584105273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c072cc0 0x7f711c0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.413+0000 7f7120914700 1 --2- 172.21.15.99:0/1584105273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c0735e0 0x7f711c111410 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.413+0000 7f7120914700 1 -- 172.21.15.99:0/1584105273 >> 172.21.15.99:0/1584105273 conn(0x7f711c0fc410 msgr2=0x7f711c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.413+0000 7f7120914700 1 -- 172.21.15.99:0/1584105273 shutdown_connections 2023-08-25T08:57:57.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.414+0000 7f7120914700 1 -- 172.21.15.99:0/1584105273 wait complete. 2023-08-25T08:57:57.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.415+0000 7f7120914700 1 Processor -- start 2023-08-25T08:57:57.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.415+0000 7f7120914700 1 -- start start 2023-08-25T08:57:57.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.416+0000 7f7120914700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c072cc0 0x7f711c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.416+0000 7f7120914700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f711c0735e0 0x7f711c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.416+0000 7f7120914700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c111950 0x7f711c1a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.416+0000 7f7120914700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c1a2220 con 0x7f711c111950 2023-08-25T08:57:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.416+0000 7f7120914700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c1a2390 con 0x7f711c072cc0 2023-08-25T08:57:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.416+0000 7f7120914700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c1a2550 con 0x7f711c0735e0 2023-08-25T08:57:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.417+0000 7f711a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c072cc0 0x7f711c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.417+0000 7f7119d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f711c0735e0 0x7f711c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:57.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.417+0000 7f711a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c072cc0 0x7f711c19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:44942/0 (socket says 172.21.15.99:44942) 2023-08-25T08:57:57.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.417+0000 7f711a59c700 1 -- 172.21.15.99:0/1326160413 learned_addr learned my addr 172.21.15.99:0/1326160413 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:57:57.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.419+0000 7f711ad9d700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c111950 0x7f711c1a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:57.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.419+0000 7f7119d9b700 1 -- 172.21.15.99:0/1326160413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c072cc0 msgr2=0x7f711c19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:57.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.419+0000 7f7119d9b700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c072cc0 0x7f711c19d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:57.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.419+0000 7f7119d9b700 1 -- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c111950 msgr2=0x7f711c1a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:57.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.419+0000 7f7119d9b700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c111950 0x7f711c1a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:57.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.419+0000 7f7119d9b700 1 -- 172.21.15.99:0/1326160413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c0095f0 con 0x7f711c0735e0 2023-08-25T08:57:57.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.419+0000 7f711ad9d700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c111950 0x7f711c1a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:57:57.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.421+0000 7f7119d9b700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f711c0735e0 0x7f711c19d7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f710c000c00 tx=0x7f710c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.422+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c01d070 con 0x7f711c0735e0 2023-08-25T08:57:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.422+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f710c005580 con 0x7f711c0735e0 2023-08-25T08:57:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.422+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c00fdf0 con 0x7f711c0735e0 2023-08-25T08:57:57.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.424+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c1a27d0 con 0x7f711c0735e0 2023-08-25T08:57:57.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.425+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c1a9ec0 con 0x7f711c0735e0 2023-08-25T08:57:57.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.428+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f711c117320 con 0x7f711c0735e0 2023-08-25T08:57:57.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.433+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f710c04d020 con 0x7f711c0735e0 2023-08-25T08:57:57.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.435+0000 7f710b7fe700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f71040a01e0 0x7f71040a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.436+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f710c0ccfd0 con 0x7f711c0735e0 2023-08-25T08:57:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.436+0000 7f710b7fe700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] conn(0x7f71040a9e20 0x7f71040ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:57:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.436+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 --> [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71040ac990 con 0x7f71040a9e20 2023-08-25T08:57:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.436+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f710c0cd380 con 0x7f711c0735e0 2023-08-25T08:57:57.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.437+0000 7f711ad9d700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] conn(0x7f71040a9e20 0x7f71040ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:57.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.438+0000 7f711a59c700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f71040a01e0 0x7f71040a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:57:57.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.439+0000 7f711ad9d700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] conn(0x7f71040a9e20 0x7f71040ac2c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.441+0000 7f711a59c700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f71040a01e0 0x7f71040a26a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7110009ad0 tx=0x7f7110009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:57:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.441+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 <== osd.9 v2:172.21.15.99:6826/797626209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f71040ac990 con 0x7f71040a9e20 2023-08-25T08:57:57.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.472+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 --> [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f711c04e620 con 0x7f71040a9e20 2023-08-25T08:57:57.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.474+0000 7f710b7fe700 1 -- 172.21.15.99:0/1326160413 <== osd.9 v2:172.21.15.99:6826/797626209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f711c04e620 con 0x7f71040a9e20 2023-08-25T08:57:57.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.475+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] conn(0x7f71040a9e20 msgr2=0x7f71040ac2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:57.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.475+0000 7f7120914700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] conn(0x7f71040a9e20 0x7f71040ac2c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:57:57.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.475+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f71040a01e0 msgr2=0x7f71040a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:57.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.475+0000 7f7120914700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f71040a01e0 0x7f71040a26a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7110009ad0 tx=0x7f7110009210 comp rx=0 tx=0).stop 2023-08-25T08:57:57.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.475+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f711c0735e0 msgr2=0x7f711c19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:57:57.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.475+0000 7f7120914700 1 --2- 172.21.15.99:0/1326160413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f711c0735e0 0x7f711c19d7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f710c000c00 tx=0x7f710c003b90 comp rx=0 tx=0).stop 2023-08-25T08:57:57.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.480+0000 7f711ad9d700 1 -- 172.21.15.99:0/1326160413 reap_dead start 2023-08-25T08:57:57.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.486+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 shutdown_connections 2023-08-25T08:57:57.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.486+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 >> 172.21.15.99:0/1326160413 conn(0x7f711c0fc410 msgr2=0x7f711c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:57:57.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:57 smithi181 ceph-mon[131296]: osdmap e85: 12 total, 12 up, 12 in 2023-08-25T08:57:57.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:57 smithi181 ceph-mon[131296]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-08-25T08:57:57.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.494+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 shutdown_connections 2023-08-25T08:57:57.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:57:57.494+0000 7f7120914700 1 -- 172.21.15.99:0/1326160413 wait complete. 2023-08-25T08:57:58.163 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:58.221 INFO:teuthology.orchestra.run.smithi099.stdout:283467841563 2023-08-25T08:57:58.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-08-25T08:57:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:58.385 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:58.386 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:58.862 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:59.004 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:59.011 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:59.477 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:59.480 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:57:59.726 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:57:59 smithi099 ceph-mon[97641]: pgmap v287: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T08:57:59.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:57:59 smithi157 ceph-mon[130879]: pgmap v287: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T08:57:59.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:57:59 smithi181 ceph-mon[131296]: pgmap v287: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T08:58:00.502 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:00.865 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:01.770 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:01 smithi099 ceph-mon[97641]: pgmap v288: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:01.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:01 smithi157 ceph-mon[130879]: pgmap v288: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:01.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:01 smithi181 ceph-mon[131296]: pgmap v288: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:02.730 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:03.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 -- 172.21.15.99:0/1181106865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c072d50 msgr2=0x7f3f7c073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:03.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 --2- 172.21.15.99:0/1181106865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c072d50 0x7f3f7c073130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 -- 172.21.15.99:0/1181106865 shutdown_connections 2023-08-25T08:58:03.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 --2- 172.21.15.99:0/1181106865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f7c10d180 0x7f3f7c10f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 --2- 172.21.15.99:0/1181106865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c072d50 0x7f3f7c073130 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 --2- 172.21.15.99:0/1181106865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c073670 0x7f3f7c10cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 -- 172.21.15.99:0/1181106865 >> 172.21.15.99:0/1181106865 conn(0x7f3f7c0fc410 msgr2=0x7f3f7c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:03.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 -- 172.21.15.99:0/1181106865 shutdown_connections 2023-08-25T08:58:03.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.338+0000 7f3f8302d700 1 -- 172.21.15.99:0/1181106865 wait complete. 2023-08-25T08:58:03.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.339+0000 7f3f8302d700 1 Processor -- start 2023-08-25T08:58:03.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.339+0000 7f3f8302d700 1 -- start start 2023-08-25T08:58:03.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.339+0000 7f3f8302d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c072d50 0x7f3f7c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:03.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f8302d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f7c073670 0x7f3f7c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:03.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f8302d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c10d180 0x7f3f7c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:03.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f8302d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c199150 con 0x7f3f7c10d180 2023-08-25T08:58:03.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f8302d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c1992c0 con 0x7f3f7c073670 2023-08-25T08:58:03.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f8302d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c199480 con 0x7f3f7c072d50 2023-08-25T08:58:03.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c10d180 0x7f3f7c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:03.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f80dc9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c072d50 0x7f3f7c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:03.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c10d180 0x7f3f7c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46788/0 (socket says 172.21.15.99:46788) 2023-08-25T08:58:03.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 -- 172.21.15.99:0/2068954621 learned_addr learned my addr 172.21.15.99:0/2068954621 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:03.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f7bfff700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f7c073670 0x7f3f7c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:03.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 -- 172.21.15.99:0/2068954621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c072d50 msgr2=0x7f3f7c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:03.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c072d50 0x7f3f7c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 -- 172.21.15.99:0/2068954621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f7c073670 msgr2=0x7f3f7c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:03.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f7c073670 0x7f3f7c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 -- 172.21.15.99:0/2068954621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700095f0 con 0x7f3f7c10d180 2023-08-25T08:58:03.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.340+0000 7f3f815ca700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c10d180 0x7f3f7c198ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3f74007ee0 tx=0x7f3f7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:03.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.341+0000 7f3f79ffb700 1 -- 172.21.15.99:0/2068954621 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74017070 con 0x7f3f7c10d180 2023-08-25T08:58:03.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.341+0000 7f3f79ffb700 1 -- 172.21.15.99:0/2068954621 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7400f040 con 0x7f3f7c10d180 2023-08-25T08:58:03.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.341+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c199760 con 0x7f3f7c10d180 2023-08-25T08:58:03.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.341+0000 7f3f79ffb700 1 -- 172.21.15.99:0/2068954621 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74005580 con 0x7f3f7c10d180 2023-08-25T08:58:03.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.341+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c1a5a50 con 0x7f3f7c10d180 2023-08-25T08:58:03.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.342+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c04e620 con 0x7f3f7c10d180 2023-08-25T08:58:03.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.344+0000 7f3f79ffb700 1 -- 172.21.15.99:0/2068954621 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f3f7400db20 con 0x7f3f7c10d180 2023-08-25T08:58:03.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.345+0000 7f3f79ffb700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f6c0a0060 0x7f3f6c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:03.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.345+0000 7f3f79ffb700 1 -- 172.21.15.99:0/2068954621 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f740cb160 con 0x7f3f7c10d180 2023-08-25T08:58:03.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.345+0000 7f3f80dc9700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f6c0a0060 0x7f3f6c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:03.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.346+0000 7f3f80dc9700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f6c0a0060 0x7f3f6c0a2520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000b5c0 tx=0x7f3f70005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:03.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.348+0000 7f3f79ffb700 1 -- 172.21.15.99:0/2068954621 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3f74094d10 con 0x7f3f7c10d180 2023-08-25T08:58:03.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.557+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f3f7c199ff0 con 0x7f3f7c10d180 2023-08-25T08:58:03.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.558+0000 7f3f79ffb700 1 -- 172.21.15.99:0/2068954621 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f3f74098330 con 0x7f3f7c10d180 2023-08-25T08:58:03.147 INFO:teuthology.orchestra.run.smithi099.stdout:335007449110 2023-08-25T08:58:03.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f6c0a0060 msgr2=0x7f3f6c0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:03.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f6c0a0060 0x7f3f6c0a2520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000b5c0 tx=0x7f3f70005fd0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c10d180 msgr2=0x7f3f7c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:03.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c10d180 0x7f3f7c198ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3f74007ee0 tx=0x7f3f7400d3b0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 shutdown_connections 2023-08-25T08:58:03.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c10d180 0x7f3f7c198ae0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c072d50 0x7f3f7c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3f6c0a0060 0x7f3f6c0a2520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 --2- 172.21.15.99:0/2068954621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f7c073670 0x7f3f7c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 >> 172.21.15.99:0/2068954621 conn(0x7f3f7c0fc410 msgr2=0x7f3f7c10f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:03.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.560+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 shutdown_connections 2023-08-25T08:58:03.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:02.561+0000 7f3f8302d700 1 -- 172.21.15.99:0/2068954621 wait complete. 2023-08-25T08:58:03.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:03 smithi099 ceph-mon[97641]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:03.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:03 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2068954621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-25T08:58:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:03 smithi157 ceph-mon[130879]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:03 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2068954621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-25T08:58:03.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 -- 172.21.15.99:0/2658627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec106940 msgr2=0x7fa3ec106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:03.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/2658627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec106940 0x7fa3ec106d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009990 tx=0x7fa3dc009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 -- 172.21.15.99:0/2658627 shutdown_connections 2023-08-25T08:58:03.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/2658627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ec101300 0x7fa3ec10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/2658627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec106940 0x7fa3ec106d20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/2658627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ec100940 0x7fa3ec100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 -- 172.21.15.99:0/2658627 >> 172.21.15.99:0/2658627 conn(0x7fa3ec0fc410 msgr2=0x7fa3ec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:03.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 -- 172.21.15.99:0/2658627 shutdown_connections 2023-08-25T08:58:03.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.851+0000 7fa3f20c9700 1 -- 172.21.15.99:0/2658627 wait complete. 2023-08-25T08:58:03.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.852+0000 7fa3f20c9700 1 Processor -- start 2023-08-25T08:58:03.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.852+0000 7fa3f20c9700 1 -- start start 2023-08-25T08:58:03.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3f20c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec100940 0x7fa3ec19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:03.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3f20c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ec101300 0x7fa3ec19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:03.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3f20c9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ec106940 0x7fa3ec198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:03.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3f20c9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec199110 con 0x7fa3ec100940 2023-08-25T08:58:03.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3f20c9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec199280 con 0x7fa3ec106940 2023-08-25T08:58:03.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3f20c9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec199440 con 0x7fa3ec101300 2023-08-25T08:58:03.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3eb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec100940 0x7fa3ec19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:03.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3eb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec100940 0x7fa3ec19ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46806/0 (socket says 172.21.15.99:46806) 2023-08-25T08:58:03.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3eb7fe700 1 -- 172.21.15.99:0/380747746 learned_addr learned my addr 172.21.15.99:0/380747746 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:03.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3eaffd700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ec101300 0x7fa3ec19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:03.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.853+0000 7fa3ebfff700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ec106940 0x7fa3ec198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:03.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3eb7fe700 1 -- 172.21.15.99:0/380747746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ec101300 msgr2=0x7fa3ec19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:03.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3eb7fe700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ec101300 0x7fa3ec19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3eb7fe700 1 -- 172.21.15.99:0/380747746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ec106940 msgr2=0x7fa3ec198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:03.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3eb7fe700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ec106940 0x7fa3ec198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:03.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3eb7fe700 1 -- 172.21.15.99:0/380747746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0095f0 con 0x7fa3ec100940 2023-08-25T08:58:03.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3eb7fe700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec100940 0x7fa3ec19ea20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005910 tx=0x7fa3dc004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:03.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3e8ff9700 1 -- 172.21.15.99:0/380747746 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc01d070 con 0x7fa3ec100940 2023-08-25T08:58:03.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3e8ff9700 1 -- 172.21.15.99:0/380747746 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3dc022470 con 0x7fa3ec100940 2023-08-25T08:58:03.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3e8ff9700 1 -- 172.21.15.99:0/380747746 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc017650 con 0x7fa3ec100940 2023-08-25T08:58:03.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec1996c0 con 0x7fa3ec100940 2023-08-25T08:58:03.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.854+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec1a5a10 con 0x7fa3ec100940 2023-08-25T08:58:03.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.855+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec04e620 con 0x7fa3ec100940 2023-08-25T08:58:03.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.857+0000 7fa3e8ff9700 1 -- 172.21.15.99:0/380747746 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa3dc0225e0 con 0x7fa3ec100940 2023-08-25T08:58:03.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.859+0000 7fa3e8ff9700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa3d40a0060 0x7fa3d40a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:03.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.859+0000 7fa3e8ff9700 1 -- 172.21.15.99:0/380747746 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa3dc0cc5d0 con 0x7fa3ec100940 2023-08-25T08:58:03.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.859+0000 7fa3eaffd700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa3d40a0060 0x7fa3d40a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:03.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.859+0000 7fa3eaffd700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa3d40a0060 0x7fa3d40a2520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0fed50 tx=0x7fa3e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:03.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:03.863+0000 7fa3e8ff9700 1 -- 172.21.15.99:0/380747746 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa3dc05ce00 con 0x7fa3ec100940 2023-08-25T08:58:03.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:03 smithi181 ceph-mon[131296]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:03.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:03 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2068954621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-25T08:58:04.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.080+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa3ec199fb0 con 0x7fa3ec100940 2023-08-25T08:58:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.081+0000 7fa3e8ff9700 1 -- 172.21.15.99:0/380747746 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3dc096210 con 0x7fa3ec100940 2023-08-25T08:58:04.082 INFO:teuthology.orchestra.run.smithi099.stdout:55834574913 2023-08-25T08:58:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa3d40a0060 msgr2=0x7fa3d40a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa3d40a0060 0x7fa3d40a2520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0fed50 tx=0x7fa3e0009260 comp rx=0 tx=0).stop 2023-08-25T08:58:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec100940 msgr2=0x7fa3ec19ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec100940 0x7fa3ec19ea20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005910 tx=0x7fa3dc004710 comp rx=0 tx=0).stop 2023-08-25T08:58:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 shutdown_connections 2023-08-25T08:58:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa3d40a0060 0x7fa3d40a2520 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3ec106940 0x7fa3ec198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3ec100940 0x7fa3ec19ea20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 --2- 172.21.15.99:0/380747746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ec101300 0x7fa3ec19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 >> 172.21.15.99:0/380747746 conn(0x7fa3ec0fc410 msgr2=0x7fa3ec0fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.083+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 shutdown_connections 2023-08-25T08:58:04.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:04.084+0000 7fa3f20c9700 1 -- 172.21.15.99:0/380747746 wait complete. 2023-08-25T08:58:05.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.152+0000 7f8e859e0700 1 -- 172.21.15.99:0/433904994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80103770 msgr2=0x7f8e801077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:05.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.152+0000 7f8e859e0700 1 --2- 172.21.15.99:0/433904994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80103770 0x7f8e801077c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009990 tx=0x7f8e74009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.152+0000 7f8e859e0700 1 -- 172.21.15.99:0/433904994 shutdown_connections 2023-08-25T08:58:05.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.152+0000 7f8e859e0700 1 --2- 172.21.15.99:0/433904994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e80107ea0 0x7f8e8010fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.152+0000 7f8e859e0700 1 --2- 172.21.15.99:0/433904994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e80102dc0 0x7f8e801031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.152+0000 7f8e859e0700 1 --2- 172.21.15.99:0/433904994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80103770 0x7f8e801077c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.152+0000 7f8e859e0700 1 -- 172.21.15.99:0/433904994 >> 172.21.15.99:0/433904994 conn(0x7f8e800fe630 msgr2=0x7f8e80100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:05.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.153+0000 7f8e859e0700 1 -- 172.21.15.99:0/433904994 shutdown_connections 2023-08-25T08:58:05.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.153+0000 7f8e859e0700 1 -- 172.21.15.99:0/433904994 wait complete. 2023-08-25T08:58:05.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.153+0000 7f8e859e0700 1 Processor -- start 2023-08-25T08:58:05.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.153+0000 7f8e859e0700 1 -- start start 2023-08-25T08:58:05.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e859e0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e80102dc0 0x7f8e8019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:05.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e859e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e80103770 0x7f8e8019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:05.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e859e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80107ea0 0x7f8e80198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:05.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e859e0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801991c0 con 0x7f8e80107ea0 2023-08-25T08:58:05.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e859e0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80199330 con 0x7f8e80102dc0 2023-08-25T08:58:05.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e859e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801994f0 con 0x7f8e80103770 2023-08-25T08:58:05.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80107ea0 0x7f8e80198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:05.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80107ea0 0x7f8e80198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46820/0 (socket says 172.21.15.99:46820) 2023-08-25T08:58:05.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7f7fe700 1 -- 172.21.15.99:0/2609041126 learned_addr learned my addr 172.21.15.99:0/2609041126 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:05.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7effd700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e80102dc0 0x7f8e8019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:05.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7e7fc700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e80103770 0x7f8e8019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:05.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7f7fe700 1 -- 172.21.15.99:0/2609041126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e80103770 msgr2=0x7f8e8019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:05.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7f7fe700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e80103770 0x7f8e8019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7f7fe700 1 -- 172.21.15.99:0/2609041126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e80102dc0 msgr2=0x7f8e8019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:05.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7f7fe700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e80102dc0 0x7f8e8019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.154+0000 7f8e7f7fe700 1 -- 172.21.15.99:0/2609041126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740095f0 con 0x7f8e80107ea0 2023-08-25T08:58:05.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.156+0000 7f8e7effd700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e80102dc0 0x7f8e8019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:05.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.156+0000 7f8e7f7fe700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80107ea0 0x7f8e80198b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800b780 tx=0x7f8e7800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:05.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.156+0000 7f8e849de700 1 -- 172.21.15.99:0/2609041126 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800f840 con 0x7f8e80107ea0 2023-08-25T08:58:05.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.156+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e801997d0 con 0x7f8e80107ea0 2023-08-25T08:58:05.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.156+0000 7f8e849de700 1 -- 172.21.15.99:0/2609041126 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7800fe60 con 0x7f8e80107ea0 2023-08-25T08:58:05.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.156+0000 7f8e849de700 1 -- 172.21.15.99:0/2609041126 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800d3d0 con 0x7f8e80107ea0 2023-08-25T08:58:05.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.156+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801a5ac0 con 0x7f8e80107ea0 2023-08-25T08:58:05.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.156+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8004e620 con 0x7f8e80107ea0 2023-08-25T08:58:05.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.159+0000 7f8e849de700 1 -- 172.21.15.99:0/2609041126 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f8e7800f9a0 con 0x7f8e80107ea0 2023-08-25T08:58:05.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.160+0000 7f8e849de700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6809ffe0 0x7f8e680a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:05.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.160+0000 7f8e849de700 1 -- 172.21.15.99:0/2609041126 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8e780cb270 con 0x7f8e80107ea0 2023-08-25T08:58:05.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.160+0000 7f8e7effd700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6809ffe0 0x7f8e680a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:05.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.160+0000 7f8e7effd700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6809ffe0 0x7f8e680a24a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000bc10 tx=0x7f8e7000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:05.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.165+0000 7f8e849de700 1 -- 172.21.15.99:0/2609041126 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8e78094e80 con 0x7f8e80107ea0 2023-08-25T08:58:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:05 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/380747746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-25T08:58:05.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:05 smithi099 ceph-mon[97641]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:05.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.379+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8e80112c80 con 0x7f8e80107ea0 2023-08-25T08:58:05.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.379+0000 7f8e849de700 1 -- 172.21.15.99:0/2609041126 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e780984a0 con 0x7f8e80107ea0 2023-08-25T08:58:05.379 INFO:teuthology.orchestra.run.smithi099.stdout:141733920821 2023-08-25T08:58:05.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:05 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/380747746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-25T08:58:05.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:05 smithi157 ceph-mon[130879]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6809ffe0 msgr2=0x7f8e680a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6809ffe0 0x7f8e680a24a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000bc10 tx=0x7f8e7000b5f0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80107ea0 msgr2=0x7f8e80198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80107ea0 0x7f8e80198b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800b780 tx=0x7f8e7800bb40 comp rx=0 tx=0).stop 2023-08-25T08:58:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 shutdown_connections 2023-08-25T08:58:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80107ea0 0x7f8e80198b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6809ffe0 0x7f8e680a24a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e80102dc0 0x7f8e8019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 --2- 172.21.15.99:0/2609041126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e80103770 0x7f8e8019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.381+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 >> 172.21.15.99:0/2609041126 conn(0x7f8e800fe630 msgr2=0x7f8e800ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.382+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 shutdown_connections 2023-08-25T08:58:05.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:05.382+0000 7f8e859e0700 1 -- 172.21.15.99:0/2609041126 wait complete. 2023-08-25T08:58:05.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:05 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/380747746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-25T08:58:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:05 smithi181 ceph-mon[131296]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-08-25T08:58:05.649 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449108 got 335007449110 for osd.11 2023-08-25T08:58:05.650 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:06.309 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:06 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2609041126' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-25T08:58:06.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:06 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2609041126' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-25T08:58:06.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:06 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2609041126' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-25T08:58:06.739 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920818 got 141733920821 for osd.4 2023-08-25T08:58:06.740 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 -- 172.21.15.99:0/2561266357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8101300 msgr2=0x7f7ce810d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 --2- 172.21.15.99:0/2561266357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8101300 0x7f7ce810d810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000b0a0 tx=0x7f7ce000b3b0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 -- 172.21.15.99:0/2561266357 shutdown_connections 2023-08-25T08:58:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 --2- 172.21.15.99:0/2561266357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8101300 0x7f7ce810d810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 --2- 172.21.15.99:0/2561266357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ce8106940 0x7f7ce8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 --2- 172.21.15.99:0/2561266357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce8100940 0x7f7ce8100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 -- 172.21.15.99:0/2561266357 >> 172.21.15.99:0/2561266357 conn(0x7f7ce80fc410 msgr2=0x7f7ce80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 -- 172.21.15.99:0/2561266357 shutdown_connections 2023-08-25T08:58:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.059+0000 7f7ced611700 1 -- 172.21.15.99:0/2561266357 wait complete. 2023-08-25T08:58:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.060+0000 7f7ced611700 1 Processor -- start 2023-08-25T08:58:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ced611700 1 -- start start 2023-08-25T08:58:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ced611700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8100940 0x7f7ce8072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ced611700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce8101300 0x7f7ce8073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ced611700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ce8106940 0x7f7ce80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ced611700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce81a4fc0 con 0x7f7ce8100940 2023-08-25T08:58:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ced611700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce81a5130 con 0x7f7ce8101300 2023-08-25T08:58:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ced611700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce81a52f0 con 0x7f7ce8106940 2023-08-25T08:58:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ce6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8100940 0x7f7ce8072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ce6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8100940 0x7f7ce8072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38700/0 (socket says 172.21.15.99:38700) 2023-08-25T08:58:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ce6ffd700 1 -- 172.21.15.99:0/3428727066 learned_addr learned my addr 172.21.15.99:0/3428727066 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.061+0000 7f7ce67fc700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce8101300 0x7f7ce8073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ce77fe700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ce8106940 0x7f7ce80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ce6ffd700 1 -- 172.21.15.99:0/3428727066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ce8106940 msgr2=0x7f7ce80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ce6ffd700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ce8106940 0x7f7ce80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ce6ffd700 1 -- 172.21.15.99:0/3428727066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce8101300 msgr2=0x7f7ce8073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ce6ffd700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce8101300 0x7f7ce8073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ce6ffd700 1 -- 172.21.15.99:0/3428727066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce0009d00 con 0x7f7ce8100940 2023-08-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ce67fc700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce8101300 0x7f7ce8073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:07.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ce6ffd700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8100940 0x7f7ce8072c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800e9a0 tx=0x7f7cd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:07.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ccffff700 1 -- 172.21.15.99:0/3428727066 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8009790 con 0x7f7ce8100940 2023-08-25T08:58:07.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ccffff700 1 -- 172.21.15.99:0/3428727066 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cd8010430 con 0x7f7ce8100940 2023-08-25T08:58:07.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ccffff700 1 -- 172.21.15.99:0/3428727066 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8018680 con 0x7f7ce8100940 2023-08-25T08:58:07.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.062+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce81a55d0 con 0x7f7ce8100940 2023-08-25T08:58:07.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.063+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce81a5a80 con 0x7f7ce8100940 2023-08-25T08:58:07.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.064+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce804e620 con 0x7f7ce8100940 2023-08-25T08:58:07.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.065+0000 7f7ccffff700 1 -- 172.21.15.99:0/3428727066 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f7cd80105a0 con 0x7f7ce8100940 2023-08-25T08:58:07.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.068+0000 7f7ccffff700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7cd009ffe0 0x7f7cd00a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.068+0000 7f7ccffff700 1 -- 172.21.15.99:0/3428727066 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7cd8014070 con 0x7f7ce8100940 2023-08-25T08:58:07.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.068+0000 7f7ce67fc700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7cd009ffe0 0x7f7cd00a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.069+0000 7f7ce67fc700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7cd009ffe0 0x7f7cd00a24a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8074410 tx=0x7f7cdc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:07.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.072+0000 7f7ccffff700 1 -- 172.21.15.99:0/3428727066 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7cd8051e00 con 0x7f7ce8100940 2023-08-25T08:58:07.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 -- 172.21.15.99:0/479709119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec100940 msgr2=0x7f7aec100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 --2- 172.21.15.99:0/479709119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec100940 0x7f7aec100dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009990 tx=0x7f7ad4009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 -- 172.21.15.99:0/479709119 shutdown_connections 2023-08-25T08:58:07.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 --2- 172.21.15.99:0/479709119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aec101300 0x7f7aec10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 --2- 172.21.15.99:0/479709119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec106940 0x7f7aec106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 --2- 172.21.15.99:0/479709119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec100940 0x7f7aec100dc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 -- 172.21.15.99:0/479709119 >> 172.21.15.99:0/479709119 conn(0x7f7aec0fc410 msgr2=0x7f7aec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:07.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 -- 172.21.15.99:0/479709119 shutdown_connections 2023-08-25T08:58:07.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.945+0000 7f7af23d3700 1 -- 172.21.15.99:0/479709119 wait complete. 2023-08-25T08:58:07.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.947+0000 7f7af23d3700 1 Processor -- start 2023-08-25T08:58:07.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.947+0000 7f7af23d3700 1 -- start start 2023-08-25T08:58:07.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.947+0000 7f7af23d3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec100940 0x7f7aec072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.947+0000 7f7af23d3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec101300 0x7f7aec073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.947+0000 7f7af23d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aec106940 0x7f7aec0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7af23d3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec1a4ec0 con 0x7f7aec101300 2023-08-25T08:58:07.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7af23d3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec1a5030 con 0x7f7aec100940 2023-08-25T08:58:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7af23d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec1a51f0 con 0x7f7aec106940 2023-08-25T08:58:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec101300 0x7f7aec073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec101300 0x7f7aec073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38674/0 (socket says 172.21.15.99:38674) 2023-08-25T08:58:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7aebfff700 1 -- 172.21.15.99:0/2480503763 learned_addr learned my addr 172.21.15.99:0/2480503763 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7ae3fff700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec100940 0x7f7aec072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7aebfff700 1 -- 172.21.15.99:0/2480503763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aec106940 msgr2=0x7f7aec0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7aebfff700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aec106940 0x7f7aec0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7aebfff700 1 -- 172.21.15.99:0/2480503763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec100940 msgr2=0x7f7aec072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7aebfff700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec100940 0x7f7aec072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.948+0000 7f7aebfff700 1 -- 172.21.15.99:0/2480503763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40095f0 con 0x7f7aec101300 2023-08-25T08:58:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.949+0000 7f7aebfff700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec101300 0x7f7aec073160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005910 tx=0x7f7ad4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.949+0000 7f7ae9ffb700 1 -- 172.21.15.99:0/2480503763 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad401d070 con 0x7f7aec101300 2023-08-25T08:58:07.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.949+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec1a5470 con 0x7f7aec101300 2023-08-25T08:58:07.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.949+0000 7f7ae9ffb700 1 -- 172.21.15.99:0/2480503763 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ad4022470 con 0x7f7aec101300 2023-08-25T08:58:07.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.949+0000 7f7ae9ffb700 1 -- 172.21.15.99:0/2480503763 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4017650 con 0x7f7aec101300 2023-08-25T08:58:07.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.949+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec1a58e0 con 0x7f7aec101300 2023-08-25T08:58:07.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.950+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec04e620 con 0x7f7aec101300 2023-08-25T08:58:07.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.954+0000 7f7ae9ffb700 1 -- 172.21.15.99:0/2480503763 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f7ad40225e0 con 0x7f7aec101300 2023-08-25T08:58:07.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.956+0000 7f7ae9ffb700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7ad809ffe0 0x7f7ad80a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.956+0000 7f7ae9ffb700 1 -- 172.21.15.99:0/2480503763 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7ad40cce10 con 0x7f7aec101300 2023-08-25T08:58:07.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.957+0000 7f7ae3fff700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7ad809ffe0 0x7f7ad80a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.958+0000 7f7ae3fff700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7ad809ffe0 0x7f7ad80a24a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009be0 tx=0x7f7adc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:07.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.960+0000 7f7ae9ffb700 1 -- 172.21.15.99:0/2480503763 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7ad4096ab0 con 0x7f7aec101300 2023-08-25T08:58:07.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.177+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f7aec110a20 con 0x7f7aec101300 2023-08-25T08:58:07.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.178+0000 7f7ae9ffb700 1 -- 172.21.15.99:0/2480503763 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ad409a0d0 con 0x7f7aec101300 2023-08-25T08:58:07.178 INFO:teuthology.orchestra.run.smithi099.stdout:249108103203 2023-08-25T08:58:07.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7ad809ffe0 msgr2=0x7f7ad80a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7ad809ffe0 0x7f7ad80a24a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009be0 tx=0x7f7adc009260 comp rx=0 tx=0).stop 2023-08-25T08:58:07.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec101300 msgr2=0x7f7aec073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec101300 0x7f7aec073160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005910 tx=0x7f7ad4005720 comp rx=0 tx=0).stop 2023-08-25T08:58:07.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 shutdown_connections 2023-08-25T08:58:07.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aec106940 0x7f7aec0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7ad809ffe0 0x7f7ad80a24a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec100940 0x7f7aec072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 --2- 172.21.15.99:0/2480503763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aec101300 0x7f7aec073160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 >> 172.21.15.99:0/2480503763 conn(0x7f7aec0fc410 msgr2=0x7f7aec0fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:07.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 shutdown_connections 2023-08-25T08:58:07.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.180+0000 7f7af23d3700 1 -- 172.21.15.99:0/2480503763 wait complete. 2023-08-25T08:58:07.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.288+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7ce8110a80 con 0x7f7ce8100940 2023-08-25T08:58:07.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.289+0000 7f7ccffff700 1 -- 172.21.15.99:0/3428727066 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7cd8095130 con 0x7f7ce8100940 2023-08-25T08:58:07.289 INFO:teuthology.orchestra.run.smithi099.stdout:219043332134 2023-08-25T08:58:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.291+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7cd009ffe0 msgr2=0x7f7cd00a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.291+0000 7f7ced611700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7cd009ffe0 0x7f7cd00a24a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8074410 tx=0x7f7cdc009260 comp rx=0 tx=0).stop 2023-08-25T08:58:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.291+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8100940 msgr2=0x7f7ce8072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.291+0000 7f7ced611700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8100940 0x7f7ce8072c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800e9a0 tx=0x7f7cd800ecb0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.292+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 shutdown_connections 2023-08-25T08:58:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.292+0000 7f7ced611700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ce8106940 0x7f7ce80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.292+0000 7f7ced611700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ce8100940 0x7f7ce8072c20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.292+0000 7f7ced611700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f7cd009ffe0 0x7f7cd00a24a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.292+0000 7f7ced611700 1 --2- 172.21.15.99:0/3428727066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce8101300 0x7f7ce8073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.292+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 >> 172.21.15.99:0/3428727066 conn(0x7f7ce80fc410 msgr2=0x7f7ce80fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:07.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.292+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 shutdown_connections 2023-08-25T08:58:07.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.292+0000 7f7ced611700 1 -- 172.21.15.99:0/3428727066 wait complete. 2023-08-25T08:58:07.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:07 smithi099 ceph-mon[97641]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:07.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:07 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1963580711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-25T08:58:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:07 smithi157 ceph-mon[130879]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:07.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:07 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1963580711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-25T08:58:07.496 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:07 smithi181 ceph-mon[131296]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:07.496 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:07 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1963580711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-25T08:58:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 -- 172.21.15.99:0/3722867482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106940 msgr2=0x7f968c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 --2- 172.21.15.99:0/3722867482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106940 0x7f968c106d20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9688009990 tx=0x7f9688009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 -- 172.21.15.99:0/3722867482 shutdown_connections 2023-08-25T08:58:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 --2- 172.21.15.99:0/3722867482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968c101300 0x7f968c10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 --2- 172.21.15.99:0/3722867482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106940 0x7f968c106d20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 --2- 172.21.15.99:0/3722867482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c100940 0x7f968c100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 -- 172.21.15.99:0/3722867482 >> 172.21.15.99:0/3722867482 conn(0x7f968c0fc410 msgr2=0x7f968c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 -- 172.21.15.99:0/3722867482 shutdown_connections 2023-08-25T08:58:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.548+0000 7f96940dc700 1 -- 172.21.15.99:0/3722867482 wait complete. 2023-08-25T08:58:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.549+0000 7f96940dc700 1 Processor -- start 2023-08-25T08:58:07.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.549+0000 7f96940dc700 1 -- start start 2023-08-25T08:58:07.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f96940dc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968c100940 0x7f968c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f96940dc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c101300 0x7f968c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f96940dc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c106940 0x7f968c1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f96940dc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c196f30 con 0x7f968c101300 2023-08-25T08:58:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f96940dc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1970a0 con 0x7f968c106940 2023-08-25T08:58:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f96940dc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c197260 con 0x7f968c100940 2023-08-25T08:58:07.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f9692679700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c106940 0x7f968c1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f9691677700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c101300 0x7f968c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f9691e78700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968c100940 0x7f968c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f9692679700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c106940 0x7f968c1968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:45006/0 (socket says 172.21.15.99:45006) 2023-08-25T08:58:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.550+0000 7f9692679700 1 -- 172.21.15.99:0/1963580711 learned_addr learned my addr 172.21.15.99:0/1963580711 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9692679700 1 -- 172.21.15.99:0/1963580711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968c100940 msgr2=0x7f968c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9692679700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968c100940 0x7f968c19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9692679700 1 -- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c101300 msgr2=0x7f968c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9692679700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c101300 0x7f968c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9692679700 1 -- 172.21.15.99:0/1963580711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880095f0 con 0x7f968c106940 2023-08-25T08:58:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9692679700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c106940 0x7f968c1968c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9684007ee0 tx=0x7f968400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:07.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9682ffd700 1 -- 172.21.15.99:0/1963580711 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684017070 con 0x7f968c106940 2023-08-25T08:58:07.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c197540 con 0x7f968c106940 2023-08-25T08:58:07.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9682ffd700 1 -- 172.21.15.99:0/1963580711 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f968400f040 con 0x7f968c106940 2023-08-25T08:58:07.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f9682ffd700 1 -- 172.21.15.99:0/1963580711 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684005580 con 0x7f968c106940 2023-08-25T08:58:07.661 INFO:teuthology.orchestra.run.smithi099.stdout:304942678040 2023-08-25T08:58:07.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.551+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c1a3830 con 0x7f968c106940 2023-08-25T08:58:07.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.552+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c04e620 con 0x7f968c106940 2023-08-25T08:58:07.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.554+0000 7f9682ffd700 1 -- 172.21.15.99:0/1963580711 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f968400db20 con 0x7f968c106940 2023-08-25T08:58:07.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.556+0000 7f9682ffd700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f967809ffe0 0x7f96780a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.556+0000 7f9682ffd700 1 -- 172.21.15.99:0/1963580711 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f96840cb1e0 con 0x7f968c106940 2023-08-25T08:58:07.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.556+0000 7f9691e78700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f967809ffe0 0x7f96780a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.556+0000 7f9691e78700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f967809ffe0 0x7f96780a24a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f968800b2e0 tx=0x7f9688005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:07.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.558+0000 7f9682ffd700 1 -- 172.21.15.99:0/1963580711 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9684094d90 con 0x7f968c106940 2023-08-25T08:58:07.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.819+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f968c110a20 con 0x7f968c106940 2023-08-25T08:58:07.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.820+0000 7f9682ffd700 1 -- 172.21.15.99:0/1963580711 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f96840983b0 con 0x7f968c106940 2023-08-25T08:58:07.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f967809ffe0 msgr2=0x7f96780a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f967809ffe0 0x7f96780a24a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f968800b2e0 tx=0x7f9688005fd0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c106940 msgr2=0x7f968c1968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c106940 0x7f968c1968c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9684007ee0 tx=0x7f968400d3b0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 shutdown_connections 2023-08-25T08:58:07.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f967809ffe0 0x7f96780a24a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c106940 0x7f968c1968c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968c100940 0x7f968c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 --2- 172.21.15.99:0/1963580711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c101300 0x7f968c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.823+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 >> 172.21.15.99:0/1963580711 conn(0x7f968c0fc410 msgr2=0x7f968c0fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:07.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.824+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 shutdown_connections 2023-08-25T08:58:07.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:06.824+0000 7f96940dc700 1 -- 172.21.15.99:0/1963580711 wait complete. 2023-08-25T08:58:07.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.418+0000 7ff57d937700 1 -- 172.21.15.99:0/1543606456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5781037a0 msgr2=0x7ff5781077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.418+0000 7ff57d937700 1 --2- 172.21.15.99:0/1543606456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5781037a0 0x7ff5781077f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0098f0 tx=0x7ff56c009c00 comp rx=0 tx=0).stop 2023-08-25T08:58:07.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.419+0000 7ff57d937700 1 -- 172.21.15.99:0/1543606456 shutdown_connections 2023-08-25T08:58:07.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.419+0000 7ff57d937700 1 --2- 172.21.15.99:0/1543606456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff578107ed0 0x7ff57810faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.419+0000 7ff57d937700 1 --2- 172.21.15.99:0/1543606456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff578102df0 0x7ff5781031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.419+0000 7ff57d937700 1 --2- 172.21.15.99:0/1543606456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5781037a0 0x7ff5781077f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.419+0000 7ff57d937700 1 -- 172.21.15.99:0/1543606456 >> 172.21.15.99:0/1543606456 conn(0x7ff5780fe660 msgr2=0x7ff578100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:07.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.419+0000 7ff57d937700 1 -- 172.21.15.99:0/1543606456 shutdown_connections 2023-08-25T08:58:07.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.419+0000 7ff57d937700 1 -- 172.21.15.99:0/1543606456 wait complete. 2023-08-25T08:58:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.420+0000 7ff57d937700 1 Processor -- start 2023-08-25T08:58:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.420+0000 7ff57d937700 1 -- start start 2023-08-25T08:58:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.420+0000 7ff57d937700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff578102df0 0x7ff578198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.420+0000 7ff57d937700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5781037a0 0x7ff5781992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.420+0000 7ff57d937700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578107ed0 0x7ff57819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.420+0000 7ff57d937700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57819dd20 con 0x7ff5781037a0 2023-08-25T08:58:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.420+0000 7ff57d937700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57819de90 con 0x7ff578107ed0 2023-08-25T08:58:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.420+0000 7ff57d937700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57819e050 con 0x7ff578102df0 2023-08-25T08:58:07.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff578102df0 0x7ff578198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff578102df0 0x7ff578198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:36774/0 (socket says 172.21.15.99:36774) 2023-08-25T08:58:07.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 -- 172.21.15.99:0/3710348130 learned_addr learned my addr 172.21.15.99:0/3710348130 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:07.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff5777fe700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578107ed0 0x7ff57819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 -- 172.21.15.99:0/3710348130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578107ed0 msgr2=0x7ff57819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578107ed0 0x7ff57819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 -- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5781037a0 msgr2=0x7ff5781992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5781037a0 0x7ff5781992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 -- 172.21.15.99:0/3710348130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c009520 con 0x7ff578102df0 2023-08-25T08:58:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.421+0000 7ff576ffd700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff578102df0 0x7ff578198d60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff56800d980 tx=0x7ff56800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.422+0000 7ff57c935700 1 -- 172.21.15.99:0/3710348130 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568009930 con 0x7ff578102df0 2023-08-25T08:58:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.422+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57819e330 con 0x7ff578102df0 2023-08-25T08:58:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.422+0000 7ff57c935700 1 -- 172.21.15.99:0/3710348130 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff568010460 con 0x7ff578102df0 2023-08-25T08:58:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.422+0000 7ff57c935700 1 -- 172.21.15.99:0/3710348130 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56800f620 con 0x7ff578102df0 2023-08-25T08:58:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.422+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5781a5b90 con 0x7ff578102df0 2023-08-25T08:58:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.423+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57804e620 con 0x7ff578102df0 2023-08-25T08:58:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.425+0000 7ff57c935700 1 -- 172.21.15.99:0/3710348130 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ff5680105d0 con 0x7ff578102df0 2023-08-25T08:58:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.427+0000 7ff57c935700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff56009ffe0 0x7ff5600a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.427+0000 7ff57c935700 1 -- 172.21.15.99:0/3710348130 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff5680cb2a0 con 0x7ff578102df0 2023-08-25T08:58:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.427+0000 7ff5767fc700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff56009ffe0 0x7ff5600a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.428+0000 7ff5767fc700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff56009ffe0 0x7ff5600a24a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009810 tx=0x7ff56c005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:07.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.428+0000 7ff57c935700 1 -- 172.21.15.99:0/3710348130 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff568094e50 con 0x7ff578102df0 2023-08-25T08:58:07.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.648+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff578112c00 con 0x7ff578102df0 2023-08-25T08:58:07.744 INFO:teuthology.orchestra.run.smithi099.stdout:197568495660 2023-08-25T08:58:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.649+0000 7ff57c935700 1 -- 172.21.15.99:0/3710348130 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff568098470 con 0x7ff578102df0 2023-08-25T08:58:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff56009ffe0 msgr2=0x7ff5600a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff56009ffe0 0x7ff5600a24a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009810 tx=0x7ff56c005b40 comp rx=0 tx=0).stop 2023-08-25T08:58:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff578102df0 msgr2=0x7ff578198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:07.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff578102df0 0x7ff578198d60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff56800d980 tx=0x7ff56800dc90 comp rx=0 tx=0).stop 2023-08-25T08:58:07.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 shutdown_connections 2023-08-25T08:58:07.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff56009ffe0 0x7ff5600a24a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff578107ed0 0x7ff57819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff578102df0 0x7ff578198d60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 --2- 172.21.15.99:0/3710348130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5781037a0 0x7ff5781992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:07.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 >> 172.21.15.99:0/3710348130 conn(0x7ff5780fe660 msgr2=0x7ff5780ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:07.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 shutdown_connections 2023-08-25T08:58:07.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:07.652+0000 7ff57d937700 1 -- 172.21.15.99:0/3710348130 wait complete. 2023-08-25T08:58:07.814 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574912 got 55834574913 for osd.1 2023-08-25T08:58:07.815 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:07.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103201 got 249108103203 for osd.8 2023-08-25T08:58:07.861 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:08.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.016+0000 7ff91a617700 1 -- 172.21.15.99:0/4167264974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914106940 msgr2=0x7ff914106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.016+0000 7ff91a617700 1 --2- 172.21.15.99:0/4167264974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914106940 0x7ff914106d20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff904009990 tx=0x7ff904009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.016+0000 7ff91a617700 1 -- 172.21.15.99:0/4167264974 shutdown_connections 2023-08-25T08:58:08.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.016+0000 7ff91a617700 1 --2- 172.21.15.99:0/4167264974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914101300 0x7ff91410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.016+0000 7ff91a617700 1 --2- 172.21.15.99:0/4167264974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914106940 0x7ff914106d20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.016+0000 7ff91a617700 1 --2- 172.21.15.99:0/4167264974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff914100940 0x7ff914100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.016+0000 7ff91a617700 1 -- 172.21.15.99:0/4167264974 >> 172.21.15.99:0/4167264974 conn(0x7ff9140fc410 msgr2=0x7ff9140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:08.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.016+0000 7ff91a617700 1 -- 172.21.15.99:0/4167264974 shutdown_connections 2023-08-25T08:58:08.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.017+0000 7ff91a617700 1 -- 172.21.15.99:0/4167264974 wait complete. 2023-08-25T08:58:08.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.018+0000 7ff91a617700 1 Processor -- start 2023-08-25T08:58:08.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.018+0000 7ff91a617700 1 -- start start 2023-08-25T08:58:08.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff91a617700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100940 0x7ff91419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff91a617700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914101300 0x7ff91419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff91a617700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff914106940 0x7ff914198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff91a617700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141990a0 con 0x7ff914101300 2023-08-25T08:58:08.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff91a617700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff914199210 con 0x7ff914100940 2023-08-25T08:58:08.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff91a617700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141993d0 con 0x7ff914106940 2023-08-25T08:58:08.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff90bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100940 0x7ff91419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff913fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914101300 0x7ff91419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff90bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100940 0x7ff91419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:41552/0 (socket says 172.21.15.99:41552) 2023-08-25T08:58:08.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff913fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914101300 0x7ff91419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38720/0 (socket says 172.21.15.99:38720) 2023-08-25T08:58:08.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff90bfff700 1 -- 172.21.15.99:0/37080462 learned_addr learned my addr 172.21.15.99:0/37080462 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:08.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff918bb4700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff914106940 0x7ff914198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff913fff700 1 -- 172.21.15.99:0/37080462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff914106940 msgr2=0x7ff914198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff913fff700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff914106940 0x7ff914198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff913fff700 1 -- 172.21.15.99:0/37080462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100940 msgr2=0x7ff91419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff913fff700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100940 0x7ff91419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.019+0000 7ff913fff700 1 -- 172.21.15.99:0/37080462 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9040095f0 con 0x7ff914101300 2023-08-25T08:58:08.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.020+0000 7ff913fff700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914101300 0x7ff91419eef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00d950 tx=0x7ff8fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:08.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.020+0000 7ff911ffb700 1 -- 172.21.15.99:0/37080462 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc009930 con 0x7ff914101300 2023-08-25T08:58:08.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.020+0000 7ff911ffb700 1 -- 172.21.15.99:0/37080462 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8fc010460 con 0x7ff914101300 2023-08-25T08:58:08.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.020+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9141996b0 con 0x7ff914101300 2023-08-25T08:58:08.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.020+0000 7ff911ffb700 1 -- 172.21.15.99:0/37080462 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00b5d0 con 0x7ff914101300 2023-08-25T08:58:08.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.020+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141a59a0 con 0x7ff914101300 2023-08-25T08:58:08.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.021+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91404e620 con 0x7ff914101300 2023-08-25T08:58:08.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.023+0000 7ff911ffb700 1 -- 172.21.15.99:0/37080462 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ff8fc010820 con 0x7ff914101300 2023-08-25T08:58:08.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.024+0000 7ff911ffb700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9000a0060 0x7ff9000a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.024+0000 7ff911ffb700 1 -- 172.21.15.99:0/37080462 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff8fc0caf70 con 0x7ff914101300 2023-08-25T08:58:08.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.025+0000 7ff90bfff700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9000a0060 0x7ff9000a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.025+0000 7ff90bfff700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9000a0060 0x7ff9000a2520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff904006010 tx=0x7ff904005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:08.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.028+0000 7ff911ffb700 1 -- 172.21.15.99:0/37080462 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff8fc094b80 con 0x7ff914101300 2023-08-25T08:58:08.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.244+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff9140008d0 con 0x7ff914101300 2023-08-25T08:58:08.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.245+0000 7ff911ffb700 1 -- 172.21.15.99:0/37080462 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8fc017090 con 0x7ff914101300 2023-08-25T08:58:08.245 INFO:teuthology.orchestra.run.smithi099.stdout:163208757297 2023-08-25T08:58:08.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9000a0060 msgr2=0x7ff9000a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9000a0060 0x7ff9000a2520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff904006010 tx=0x7ff904005960 comp rx=0 tx=0).stop 2023-08-25T08:58:08.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914101300 msgr2=0x7ff91419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914101300 0x7ff91419eef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00d950 tx=0x7ff8fc00dc60 comp rx=0 tx=0).stop 2023-08-25T08:58:08.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 shutdown_connections 2023-08-25T08:58:08.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff914106940 0x7ff914198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff9000a0060 0x7ff9000a2520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914100940 0x7ff91419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 --2- 172.21.15.99:0/37080462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff914101300 0x7ff91419eef0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.247+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 >> 172.21.15.99:0/37080462 conn(0x7ff9140fc410 msgr2=0x7ff91410bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:08.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.248+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 shutdown_connections 2023-08-25T08:58:08.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.248+0000 7ff91a617700 1 -- 172.21.15.99:0/37080462 wait complete. 2023-08-25T08:58:08.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:08 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2480503763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-25T08:58:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:08 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3428727066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-25T08:58:08.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:08 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3710348130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-25T08:58:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:08 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2480503763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-25T08:58:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:08 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3428727066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-25T08:58:08.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:08 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3710348130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-25T08:58:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:08 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2480503763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-25T08:58:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:08 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3428727066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-25T08:58:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:08 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3710348130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-25T08:58:08.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.527+0000 7fc91cd33700 1 -- 172.21.15.99:0/640119036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918106940 msgr2=0x7fc918106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.527+0000 7fc91cd33700 1 --2- 172.21.15.99:0/640119036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918106940 0x7fc918106d20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc908009990 tx=0x7fc908009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.528+0000 7fc91cd33700 1 -- 172.21.15.99:0/640119036 shutdown_connections 2023-08-25T08:58:08.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.528+0000 7fc91cd33700 1 --2- 172.21.15.99:0/640119036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc918101300 0x7fc91810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.528+0000 7fc91cd33700 1 --2- 172.21.15.99:0/640119036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918106940 0x7fc918106d20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.528+0000 7fc91cd33700 1 --2- 172.21.15.99:0/640119036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc918100940 0x7fc918100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.528+0000 7fc91cd33700 1 -- 172.21.15.99:0/640119036 >> 172.21.15.99:0/640119036 conn(0x7fc9180fc410 msgr2=0x7fc9180fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:08.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.528+0000 7fc91cd33700 1 -- 172.21.15.99:0/640119036 shutdown_connections 2023-08-25T08:58:08.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.528+0000 7fc91cd33700 1 -- 172.21.15.99:0/640119036 wait complete. 2023-08-25T08:58:08.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.529+0000 7fc91cd33700 1 Processor -- start 2023-08-25T08:58:08.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.529+0000 7fc91cd33700 1 -- start start 2023-08-25T08:58:08.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc91cd33700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918100940 0x7fc918072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc91cd33700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc918101300 0x7fc918073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc91cd33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc918106940 0x7fc9180736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc91cd33700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9181a4e60 con 0x7fc918100940 2023-08-25T08:58:08.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc91cd33700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9181a4fd0 con 0x7fc918101300 2023-08-25T08:58:08.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc91cd33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9181a5190 con 0x7fc918106940 2023-08-25T08:58:08.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc90e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918100940 0x7fc918072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc90e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918100940 0x7fc918072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38738/0 (socket says 172.21.15.99:38738) 2023-08-25T08:58:08.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc90e59c700 1 -- 172.21.15.99:0/2422996204 learned_addr learned my addr 172.21.15.99:0/2422996204 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:08.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc91659c700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc918101300 0x7fc918073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.530+0000 7fc916d9d700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc918106940 0x7fc9180736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90e59c700 1 -- 172.21.15.99:0/2422996204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc918106940 msgr2=0x7fc9180736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90e59c700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc918106940 0x7fc9180736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90e59c700 1 -- 172.21.15.99:0/2422996204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc918101300 msgr2=0x7fc918073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90e59c700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc918101300 0x7fc918073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90e59c700 1 -- 172.21.15.99:0/2422996204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9080095f0 con 0x7fc918100940 2023-08-25T08:58:08.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc916d9d700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc918106940 0x7fc9180736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:08.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90e59c700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918100940 0x7fc918072c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc908000c00 tx=0x7fc9080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:08.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90ffff700 1 -- 172.21.15.99:0/2422996204 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90801d070 con 0x7fc918100940 2023-08-25T08:58:08.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90ffff700 1 -- 172.21.15.99:0/2422996204 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc90800fb80 con 0x7fc918100940 2023-08-25T08:58:08.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9181a5410 con 0x7fc918100940 2023-08-25T08:58:08.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc90ffff700 1 -- 172.21.15.99:0/2422996204 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9080173f0 con 0x7fc918100940 2023-08-25T08:58:08.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.531+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9181a58b0 con 0x7fc918100940 2023-08-25T08:58:08.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.532+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91804e620 con 0x7fc918100940 2023-08-25T08:58:08.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.534+0000 7fc90ffff700 1 -- 172.21.15.99:0/2422996204 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc90800fcf0 con 0x7fc918100940 2023-08-25T08:58:08.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.536+0000 7fc90ffff700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc90409ffe0 0x7fc9040a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.536+0000 7fc90ffff700 1 -- 172.21.15.99:0/2422996204 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc9080cc5b0 con 0x7fc918100940 2023-08-25T08:58:08.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.538+0000 7fc91659c700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc90409ffe0 0x7fc9040a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.539+0000 7fc91659c700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc90409ffe0 0x7fc9040a24a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc918074370 tx=0x7fc900009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.541+0000 7fc90ffff700 1 -- 172.21.15.99:0/2422996204 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc908096250 con 0x7fc918100940 2023-08-25T08:58:08.668 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678038 got 304942678040 for osd.10 2023-08-25T08:58:08.669 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.768+0000 7fa61c930700 1 -- 172.21.15.99:0/3420937774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa618072cf0 msgr2=0x7fa6180730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.768+0000 7fa61c930700 1 --2- 172.21.15.99:0/3420937774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa618072cf0 0x7fa6180730d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa608009990 tx=0x7fa608009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.769+0000 7fa61c930700 1 -- 172.21.15.99:0/3420937774 shutdown_connections 2023-08-25T08:58:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.769+0000 7fa61c930700 1 --2- 172.21.15.99:0/3420937774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61810d040 0x7fa61810f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.769+0000 7fa61c930700 1 --2- 172.21.15.99:0/3420937774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa618072cf0 0x7fa6180730d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.769+0000 7fa61c930700 1 --2- 172.21.15.99:0/3420937774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa618073610 0x7fa61810cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.769+0000 7fa61c930700 1 -- 172.21.15.99:0/3420937774 >> 172.21.15.99:0/3420937774 conn(0x7fa6180fc2b0 msgr2=0x7fa6180fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.770+0000 7fa61c930700 1 -- 172.21.15.99:0/3420937774 shutdown_connections 2023-08-25T08:58:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.770+0000 7fa61c930700 1 -- 172.21.15.99:0/3420937774 wait complete. 2023-08-25T08:58:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.770+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc918110a20 con 0x7fc918100940 2023-08-25T08:58:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.771+0000 7fc90ffff700 1 -- 172.21.15.99:0/2422996204 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc908099870 con 0x7fc918100940 2023-08-25T08:58:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.771+0000 7fa61c930700 1 Processor -- start 2023-08-25T08:58:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:34359738439 2023-08-25T08:58:08.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa61c930700 1 -- start start 2023-08-25T08:58:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa61c930700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa618072cf0 0x7fa618107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa61c930700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa618073610 0x7fa618108500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa61c930700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa61810d040 0x7fa618103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa61c930700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618103820 con 0x7fa618073610 2023-08-25T08:58:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa61c930700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618103990 con 0x7fa618072cf0 2023-08-25T08:58:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa61c930700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618103b50 con 0x7fa61810d040 2023-08-25T08:58:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa6177fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa618072cf0 0x7fa618107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa6177fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa618072cf0 0x7fa618107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:41568/0 (socket says 172.21.15.99:41568) 2023-08-25T08:58:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.772+0000 7fa6177fe700 1 -- 172.21.15.99:0/3528729048 learned_addr learned my addr 172.21.15.99:0/3528729048 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa616ffd700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa618073610 0x7fa618108500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa617fff700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa61810d040 0x7fa618103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa6177fe700 1 -- 172.21.15.99:0/3528729048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa61810d040 msgr2=0x7fa618103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa6177fe700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa61810d040 0x7fa618103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa6177fe700 1 -- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa618073610 msgr2=0x7fa618108500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa6177fe700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa618073610 0x7fa618108500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa6177fe700 1 -- 172.21.15.99:0/3528729048 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6080095f0 con 0x7fa618072cf0 2023-08-25T08:58:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa616ffd700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa618073610 0x7fa618108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:58:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa6177fe700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa618072cf0 0x7fa618107fa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa608005910 tx=0x7fa6080057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.774+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc90409ffe0 msgr2=0x7fc9040a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.774+0000 7fc91cd33700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc90409ffe0 0x7fc9040a24a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc918074370 tx=0x7fc900009260 comp rx=0 tx=0).stop 2023-08-25T08:58:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.774+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918100940 msgr2=0x7fc918072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.774+0000 7fc91cd33700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918100940 0x7fc918072c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc908000c00 tx=0x7fc9080047b0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.773+0000 7fa614ff9700 1 -- 172.21.15.99:0/3528729048 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60801d070 con 0x7fa618072cf0 2023-08-25T08:58:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.774+0000 7fa614ff9700 1 -- 172.21.15.99:0/3528729048 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa608022470 con 0x7fa618072cf0 2023-08-25T08:58:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.774+0000 7fa614ff9700 1 -- 172.21.15.99:0/3528729048 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608017650 con 0x7fa618072cf0 2023-08-25T08:58:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.774+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa618103dd0 con 0x7fa618072cf0 2023-08-25T08:58:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.774+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61810bef0 con 0x7fa618072cf0 2023-08-25T08:58:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 shutdown_connections 2023-08-25T08:58:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fc91cd33700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc918106940 0x7fc9180736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fc91cd33700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc918100940 0x7fc918072c20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fc91cd33700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc90409ffe0 0x7fc9040a24a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fc91cd33700 1 --2- 172.21.15.99:0/2422996204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc918101300 0x7fc918073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 >> 172.21.15.99:0/2422996204 conn(0x7fc9180fc410 msgr2=0x7fc9180fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 shutdown_connections 2023-08-25T08:58:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fc91cd33700 1 -- 172.21.15.99:0/2422996204 wait complete. 2023-08-25T08:58:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.775+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61804ee70 con 0x7fa618072cf0 2023-08-25T08:58:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.777+0000 7fa614ff9700 1 -- 172.21.15.99:0/3528729048 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa6080225e0 con 0x7fa618072cf0 2023-08-25T08:58:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.779+0000 7fa614ff9700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa6000a0060 0x7fa6000a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:08.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.779+0000 7fa614ff9700 1 -- 172.21.15.99:0/3528729048 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa6080cc9a0 con 0x7fa618072cf0 2023-08-25T08:58:08.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.779+0000 7fa616ffd700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa6000a0060 0x7fa6000a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:08.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.780+0000 7fa616ffd700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa6000a0060 0x7fa6000a2520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa618104840 tx=0x7fa60c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:08.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.781+0000 7fa614ff9700 1 -- 172.21.15.99:0/3528729048 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa608096640 con 0x7fa618072cf0 2023-08-25T08:58:08.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:08.999+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa618104580 con 0x7fa618072cf0 2023-08-25T08:58:09.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.000+0000 7fa614ff9700 1 -- 172.21.15.99:0/3528729048 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa608027020 con 0x7fa618072cf0 2023-08-25T08:58:09.000 INFO:teuthology.orchestra.run.smithi099.stdout:107374182456 2023-08-25T08:58:09.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.002+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa6000a0060 msgr2=0x7fa6000a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:09.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.002+0000 7fa61c930700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa6000a0060 0x7fa6000a2520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa618104840 tx=0x7fa60c009260 comp rx=0 tx=0).stop 2023-08-25T08:58:09.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.002+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa618072cf0 msgr2=0x7fa618107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:09.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.002+0000 7fa61c930700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa618072cf0 0x7fa618107fa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa608005910 tx=0x7fa6080057d0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.002+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 shutdown_connections 2023-08-25T08:58:09.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.002+0000 7fa61c930700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa61810d040 0x7fa618103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.002+0000 7fa61c930700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa6000a0060 0x7fa6000a2520 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.002+0000 7fa61c930700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa618072cf0 0x7fa618107fa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.003+0000 7fa61c930700 1 --2- 172.21.15.99:0/3528729048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa618073610 0x7fa618108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.003+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 >> 172.21.15.99:0/3528729048 conn(0x7fa6180fc2b0 msgr2=0x7fa61810f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:09.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.003+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 shutdown_connections 2023-08-25T08:58:09.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.003+0000 7fa61c930700 1 -- 172.21.15.99:0/3528729048 wait complete. 2023-08-25T08:58:09.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 -- 172.21.15.99:0/1029465013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 msgr2=0x7f1674106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:09.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 --2- 172.21.15.99:0/1029465013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 0x7f1674106d20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1670009990 tx=0x7f1670009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 -- 172.21.15.99:0/1029465013 shutdown_connections 2023-08-25T08:58:09.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 --2- 172.21.15.99:0/1029465013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1674101300 0x7f167410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 --2- 172.21.15.99:0/1029465013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 0x7f1674106d20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 --2- 172.21.15.99:0/1029465013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674100940 0x7f1674100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 -- 172.21.15.99:0/1029465013 >> 172.21.15.99:0/1029465013 conn(0x7f16740fc410 msgr2=0x7f16740fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:09.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 -- 172.21.15.99:0/1029465013 shutdown_connections 2023-08-25T08:58:09.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.176+0000 7f167badc700 1 -- 172.21.15.99:0/1029465013 wait complete. 2023-08-25T08:58:09.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.178+0000 7f167badc700 1 Processor -- start 2023-08-25T08:58:09.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.178+0000 7f167badc700 1 -- start start 2023-08-25T08:58:09.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.178+0000 7f167badc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1674100940 0x7f167419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:09.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.178+0000 7f167badc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674101300 0x7f167419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:09.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.178+0000 7f167badc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 0x7f1674198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:09.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167badc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16741990f0 con 0x7f1674106940 2023-08-25T08:58:09.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167badc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674199260 con 0x7f1674101300 2023-08-25T08:58:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167badc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674199420 con 0x7f1674100940 2023-08-25T08:58:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f1679878700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1674100940 0x7f167419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167a079700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 0x7f1674198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f1679878700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1674100940 0x7f167419ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:36834/0 (socket says 172.21.15.99:36834) 2023-08-25T08:58:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f1679878700 1 -- 172.21.15.99:0/1419101225 learned_addr learned my addr 172.21.15.99:0/1419101225 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f1679077700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674101300 0x7f167419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167a079700 1 -- 172.21.15.99:0/1419101225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1674100940 msgr2=0x7f167419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167a079700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1674100940 0x7f167419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167a079700 1 -- 172.21.15.99:0/1419101225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674101300 msgr2=0x7f167419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:09.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167a079700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674101300 0x7f167419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167a079700 1 -- 172.21.15.99:0/1419101225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700095f0 con 0x7f1674106940 2023-08-25T08:58:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.179+0000 7f167a079700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 0x7f1674198a80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f166c007f10 tx=0x7f166c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.180+0000 7f166affd700 1 -- 172.21.15.99:0/1419101225 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c004db0 con 0x7f1674106940 2023-08-25T08:58:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.180+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1674199700 con 0x7f1674106940 2023-08-25T08:58:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.180+0000 7f166affd700 1 -- 172.21.15.99:0/1419101225 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f166c00f040 con 0x7f1674106940 2023-08-25T08:58:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.180+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741a59f0 con 0x7f1674106940 2023-08-25T08:58:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.180+0000 7f166affd700 1 -- 172.21.15.99:0/1419101225 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c005990 con 0x7f1674106940 2023-08-25T08:58:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.181+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1658005320 con 0x7f1674106940 2023-08-25T08:58:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.183+0000 7f166affd700 1 -- 172.21.15.99:0/1419101225 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f166c005020 con 0x7f1674106940 2023-08-25T08:58:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.184+0000 7f166affd700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f16600a00e0 0x7f16600a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.184+0000 7f166affd700 1 -- 172.21.15.99:0/1419101225 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f166c0cab10 con 0x7f1674106940 2023-08-25T08:58:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.185+0000 7f1679878700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f16600a00e0 0x7f16600a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.185+0000 7f1679878700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f16600a00e0 0x7f16600a25a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1670006010 tx=0x7f1670005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:09.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.188+0000 7f166affd700 1 -- 172.21.15.99:0/1419101225 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f166c0946c0 con 0x7f1674106940 2023-08-25T08:58:09.283 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:09 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/37080462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-25T08:58:09.283 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:09 smithi099 ceph-mon[97641]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:09.284 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:09 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2422996204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-25T08:58:09.284 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:09 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3528729048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-25T08:58:09.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:09 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/37080462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-25T08:58:09.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:09 smithi157 ceph-mon[130879]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:09.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:09 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2422996204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-25T08:58:09.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:09 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3528729048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-25T08:58:09.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.398+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1658005190 con 0x7f1674106940 2023-08-25T08:58:09.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.399+0000 7f166affd700 1 -- 172.21.15.99:0/1419101225 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f166c097ce0 con 0x7f1674106940 2023-08-25T08:58:09.399 INFO:teuthology.orchestra.run.smithi099.stdout:77309411390 2023-08-25T08:58:09.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f16600a00e0 msgr2=0x7f16600a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:09.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f16600a00e0 0x7f16600a25a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1670006010 tx=0x7f1670005960 comp rx=0 tx=0).stop 2023-08-25T08:58:09.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 msgr2=0x7f1674198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:09.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 0x7f1674198a80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f166c007f10 tx=0x7f166c00d3b0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 shutdown_connections 2023-08-25T08:58:09.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1674106940 0x7f1674198a80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1674100940 0x7f167419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f16600a00e0 0x7f16600a25a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 --2- 172.21.15.99:0/1419101225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674101300 0x7f167419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:09.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 >> 172.21.15.99:0/1419101225 conn(0x7f16740fc410 msgr2=0x7f16740fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:09.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.401+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 shutdown_connections 2023-08-25T08:58:09.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:09.402+0000 7f167badc700 1 -- 172.21.15.99:0/1419101225 wait complete. 2023-08-25T08:58:09.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:09 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/37080462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-25T08:58:09.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:09 smithi181 ceph-mon[131296]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:09.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:09 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2422996204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-25T08:58:09.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:09 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3528729048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-25T08:58:09.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738437 got 34359738439 for osd.0 2023-08-25T08:58:09.687 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:09.740 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332131 got 219043332134 for osd.7 2023-08-25T08:58:09.740 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:10.146 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757295 got 163208757297 for osd.5 2023-08-25T08:58:10.146 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:10.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:10 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1419101225' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-25T08:58:10.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:10 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1419101225' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-25T08:58:10.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:10 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1419101225' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-25T08:58:10.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411386 got 77309411390 for osd.2 2023-08-25T08:58:10.505 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:10.854 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495657 got 197568495660 for osd.6 2023-08-25T08:58:10.854 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:11.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 -- 172.21.15.99:0/2813327934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e8072d50 msgr2=0x7f26e8073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:11.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 --2- 172.21.15.99:0/2813327934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e8072d50 0x7f26e8073130 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009990 tx=0x7f26d8009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 -- 172.21.15.99:0/2813327934 shutdown_connections 2023-08-25T08:58:11.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 --2- 172.21.15.99:0/2813327934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e810d180 0x7f26e810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 --2- 172.21.15.99:0/2813327934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e8072d50 0x7f26e8073130 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 --2- 172.21.15.99:0/2813327934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e8073670 0x7f26e810cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 -- 172.21.15.99:0/2813327934 >> 172.21.15.99:0/2813327934 conn(0x7f26e80fc410 msgr2=0x7f26e80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:11.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 -- 172.21.15.99:0/2813327934 shutdown_connections 2023-08-25T08:58:11.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.252+0000 7f26ed665700 1 -- 172.21.15.99:0/2813327934 wait complete. 2023-08-25T08:58:11.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.253+0000 7f26ed665700 1 Processor -- start 2023-08-25T08:58:11.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.253+0000 7f26ed665700 1 -- start start 2023-08-25T08:58:11.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26ed665700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e8072d50 0x7f26e8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:11.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26ed665700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8073670 0x7f26e81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:11.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26ed665700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e810d180 0x7f26e819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:11.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26ed665700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e819cba0 con 0x7f26e810d180 2023-08-25T08:58:11.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26ed665700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e819cd10 con 0x7f26e8073670 2023-08-25T08:58:11.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26ed665700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e819ced0 con 0x7f26e8072d50 2023-08-25T08:58:11.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e810d180 0x7f26e819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:11.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e810d180 0x7f26e819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38784/0 (socket says 172.21.15.99:38784) 2023-08-25T08:58:11.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e77fe700 1 -- 172.21.15.99:0/3599778185 learned_addr learned my addr 172.21.15.99:0/3599778185 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:11.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e67fc700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8073670 0x7f26e81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:11.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e6ffd700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e8072d50 0x7f26e8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:11.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e77fe700 1 -- 172.21.15.99:0/3599778185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e8072d50 msgr2=0x7f26e8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:11.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e77fe700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e8072d50 0x7f26e8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e77fe700 1 -- 172.21.15.99:0/3599778185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8073670 msgr2=0x7f26e81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:11.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e77fe700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8073670 0x7f26e81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e77fe700 1 -- 172.21.15.99:0/3599778185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d80095f0 con 0x7f26e810d180 2023-08-25T08:58:11.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.254+0000 7f26e67fc700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8073670 0x7f26e81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:11.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.255+0000 7f26e77fe700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e810d180 0x7f26e819c5a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26e000ef30 tx=0x7f26e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:11.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.255+0000 7f26cffff700 1 -- 172.21.15.99:0/3599778185 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0018070 con 0x7f26e810d180 2023-08-25T08:58:11.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.255+0000 7f26cffff700 1 -- 172.21.15.99:0/3599778185 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e000f040 con 0x7f26e810d180 2023-08-25T08:58:11.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.255+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e819d150 con 0x7f26e810d180 2023-08-25T08:58:11.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.255+0000 7f26cffff700 1 -- 172.21.15.99:0/3599778185 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e000cd00 con 0x7f26e810d180 2023-08-25T08:58:11.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.255+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e81a5b00 con 0x7f26e810d180 2023-08-25T08:58:11.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.256+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e804e620 con 0x7f26e810d180 2023-08-25T08:58:11.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.258+0000 7f26cffff700 1 -- 172.21.15.99:0/3599778185 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f26e0003a00 con 0x7f26e810d180 2023-08-25T08:58:11.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.260+0000 7f26cffff700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f26d009ffe0 0x7f26d00a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:11.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.260+0000 7f26cffff700 1 -- 172.21.15.99:0/3599778185 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f26e00caf30 con 0x7f26e810d180 2023-08-25T08:58:11.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.260+0000 7f26e6ffd700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f26d009ffe0 0x7f26d00a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:11.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182453 got 107374182456 for osd.3 2023-08-25T08:58:11.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.260+0000 7f26e6ffd700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f26d009ffe0 0x7f26d00a24a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b2e0 tx=0x7f26d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:11.263 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:11.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.264+0000 7f26cffff700 1 -- 172.21.15.99:0/3599778185 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f26e0094b70 con 0x7f26e810d180 2023-08-25T08:58:11.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:11 smithi099 ceph-mon[97641]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:11.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:11 smithi157 ceph-mon[130879]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:11.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.464+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f26e8112bf0 con 0x7f26e810d180 2023-08-25T08:58:11.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.464+0000 7f26cffff700 1 -- 172.21.15.99:0/3599778185 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f26e0098190 con 0x7f26e810d180 2023-08-25T08:58:11.465 INFO:teuthology.orchestra.run.smithi099.stdout:283467841566 2023-08-25T08:58:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.466+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f26d009ffe0 msgr2=0x7f26d00a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f26d009ffe0 0x7f26d00a24a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b2e0 tx=0x7f26d8005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e810d180 msgr2=0x7f26e819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e810d180 0x7f26e819c5a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26e000ef30 tx=0x7f26e000c3a0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 shutdown_connections 2023-08-25T08:58:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e810d180 0x7f26e819c5a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e8072d50 0x7f26e8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f26d009ffe0 0x7f26d00a24a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 --2- 172.21.15.99:0/3599778185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8073670 0x7f26e81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 >> 172.21.15.99:0/3599778185 conn(0x7f26e80fc410 msgr2=0x7f26e810f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 shutdown_connections 2023-08-25T08:58:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:11.467+0000 7f26ed665700 1 -- 172.21.15.99:0/3599778185 wait complete. 2023-08-25T08:58:11.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:11 smithi181 ceph-mon[131296]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-08-25T08:58:11.962 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841563 got 283467841566 for osd.9 2023-08-25T08:58:11.962 DEBUG:teuthology.parallel:result is None 2023-08-25T08:58:11.962 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-08-25T08:58:11.963 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph pg dump --format=json 2023-08-25T08:58:12.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:12 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3599778185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-25T08:58:12.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:12 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3599778185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-25T08:58:12.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:12 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3599778185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-25T08:58:12.726 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:13.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:13 smithi099 ceph-mon[97641]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:13.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:13 smithi157 ceph-mon[130879]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:13.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:13 smithi181 ceph-mon[131296]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:14.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.228+0000 7fb34b218700 1 -- 172.21.15.99:0/2569243240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344106940 msgr2=0x7fb344106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:14.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.228+0000 7fb34b218700 1 --2- 172.21.15.99:0/2569243240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344106940 0x7fb344106d20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb338009990 tx=0x7fb338009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.229+0000 7fb34b218700 1 -- 172.21.15.99:0/2569243240 shutdown_connections 2023-08-25T08:58:14.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.229+0000 7fb34b218700 1 --2- 172.21.15.99:0/2569243240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb344101300 0x7fb34410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.229+0000 7fb34b218700 1 --2- 172.21.15.99:0/2569243240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344106940 0x7fb344106d20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.229+0000 7fb34b218700 1 --2- 172.21.15.99:0/2569243240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb344100940 0x7fb344100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.229+0000 7fb34b218700 1 -- 172.21.15.99:0/2569243240 >> 172.21.15.99:0/2569243240 conn(0x7fb3440fc410 msgr2=0x7fb3440fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:14.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.229+0000 7fb34b218700 1 -- 172.21.15.99:0/2569243240 shutdown_connections 2023-08-25T08:58:14.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.229+0000 7fb34b218700 1 -- 172.21.15.99:0/2569243240 wait complete. 2023-08-25T08:58:14.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.230+0000 7fb34b218700 1 Processor -- start 2023-08-25T08:58:14.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.230+0000 7fb34b218700 1 -- start start 2023-08-25T08:58:14.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.230+0000 7fb34b218700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344100940 0x7fb34419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:14.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.230+0000 7fb34b218700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb344101300 0x7fb34419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:14.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.230+0000 7fb34b218700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb344106940 0x7fb344198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:14.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.230+0000 7fb34b218700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3441992c0 con 0x7fb344100940 2023-08-25T08:58:14.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.230+0000 7fb34b218700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344199430 con 0x7fb344101300 2023-08-25T08:58:14.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.230+0000 7fb34b218700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3441995f0 con 0x7fb344106940 2023-08-25T08:58:14.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344100940 0x7fb34419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:14.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344100940 0x7fb34419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38794/0 (socket says 172.21.15.99:38794) 2023-08-25T08:58:14.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 -- 172.21.15.99:0/760748777 learned_addr learned my addr 172.21.15.99:0/760748777 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:14.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb3497b5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb344106940 0x7fb344198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:14.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb343fff700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb344101300 0x7fb34419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:14.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 -- 172.21.15.99:0/760748777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb344106940 msgr2=0x7fb344198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:14.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb344106940 0x7fb344198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 -- 172.21.15.99:0/760748777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb344101300 msgr2=0x7fb34419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:14.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb344101300 0x7fb34419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 -- 172.21.15.99:0/760748777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380095f0 con 0x7fb344100940 2023-08-25T08:58:14.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.231+0000 7fb348fb4700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344100940 0x7fb34419eb20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb338005910 tx=0x7fb338004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:14.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.232+0000 7fb341ffb700 1 -- 172.21.15.99:0/760748777 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33801d070 con 0x7fb344100940 2023-08-25T08:58:14.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.232+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb344199870 con 0x7fb344100940 2023-08-25T08:58:14.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.232+0000 7fb341ffb700 1 -- 172.21.15.99:0/760748777 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb338022470 con 0x7fb344100940 2023-08-25T08:58:14.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.232+0000 7fb341ffb700 1 -- 172.21.15.99:0/760748777 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338017650 con 0x7fb344100940 2023-08-25T08:58:14.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.232+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3441a5aa0 con 0x7fb344100940 2023-08-25T08:58:14.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.233+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34404e620 con 0x7fb344100940 2023-08-25T08:58:14.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.235+0000 7fb341ffb700 1 -- 172.21.15.99:0/760748777 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb33800fad0 con 0x7fb344100940 2023-08-25T08:58:14.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.236+0000 7fb341ffb700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb33409ffe0 0x7fb3340a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:14.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.236+0000 7fb341ffb700 1 -- 172.21.15.99:0/760748777 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb3380cc8b0 con 0x7fb344100940 2023-08-25T08:58:14.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.236+0000 7fb343fff700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb33409ffe0 0x7fb3340a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:14.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.237+0000 7fb343fff700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb33409ffe0 0x7fb3340a24a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb34419a340 tx=0x7fb33000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:14.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.240+0000 7fb341ffb700 1 -- 172.21.15.99:0/760748777 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb338096550 con 0x7fb344100940 2023-08-25T08:58:14.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.434+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb344110a30 con 0x7fb33409ffe0 2023-08-25T08:58:14.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.445+0000 7fb341ffb700 1 -- 172.21.15.99:0/760748777 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340677 (secure 0 0 0) 0x7fb344110a30 con 0x7fb33409ffe0 2023-08-25T08:58:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:14.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb33409ffe0 msgr2=0x7fb3340a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:14.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb33409ffe0 0x7fb3340a24a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb34419a340 tx=0x7fb33000b410 comp rx=0 tx=0).stop 2023-08-25T08:58:14.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344100940 msgr2=0x7fb34419eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:14.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344100940 0x7fb34419eb20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb338005910 tx=0x7fb338004730 comp rx=0 tx=0).stop 2023-08-25T08:58:14.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 shutdown_connections 2023-08-25T08:58:14.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb344106940 0x7fb344198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb344100940 0x7fb34419eb20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb33409ffe0 0x7fb3340a24a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 --2- 172.21.15.99:0/760748777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb344101300 0x7fb34419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:14.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 >> 172.21.15.99:0/760748777 conn(0x7fb3440fc410 msgr2=0x7fb34410bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:14.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 shutdown_connections 2023-08-25T08:58:14.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:14.451+0000 7fb34b218700 1 -- 172.21.15.99:0/760748777 wait complete. 2023-08-25T08:58:14.456 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-08-25T08:58:14.998 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-08-25T08:58:12.448754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651608,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":439,"num_read_kb":334,"num_write":331,"num_write_kb":3041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335516,"kb_used_data":11260,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606308,"statfs":{"total":1151940427776,"available":1151596859392,"internally_reserved":0,"allocated":11530240,"data_stored":7274828,"data_compressed":178299,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003143"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052994+0000","last_change":"2023-08-25T08:57:19.519342+0000","last_active":"2023-08-25T08:57:56.052994+0000","last_peered":"2023-08-25T08:57:56.052994+0000","last_clean":"2023-08-25T08:57:56.052994+0000","last_became_active":"2023-08-25T08:57:19.519006+0000","last_became_peered":"2023-08-25T08:57:19.519006+0000","last_unstale":"2023-08-25T08:57:56.052994+0000","last_undegraded":"2023-08-25T08:57:56.052994+0000","last_fullsized":"2023-08-25T08:57:56.052994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:50.111134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369781+0000","last_change":"2023-08-25T08:57:19.521779+0000","last_active":"2023-08-25T08:57:56.369781+0000","last_peered":"2023-08-25T08:57:56.369781+0000","last_clean":"2023-08-25T08:57:56.369781+0000","last_became_active":"2023-08-25T08:57:19.521358+0000","last_became_peered":"2023-08-25T08:57:19.521358+0000","last_unstale":"2023-08-25T08:57:56.369781+0000","last_undegraded":"2023-08-25T08:57:56.369781+0000","last_fullsized":"2023-08-25T08:57:56.369781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:14:25.404973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370128+0000","last_change":"2023-08-25T08:57:19.531553+0000","last_active":"2023-08-25T08:57:56.370128+0000","last_peered":"2023-08-25T08:57:56.370128+0000","last_clean":"2023-08-25T08:57:56.370128+0000","last_became_active":"2023-08-25T08:57:19.531113+0000","last_became_peered":"2023-08-25T08:57:19.531113+0000","last_unstale":"2023-08-25T08:57:56.370128+0000","last_undegraded":"2023-08-25T08:57:56.370128+0000","last_fullsized":"2023-08-25T08:57:56.370128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:28:58.239330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.094847+0000","last_change":"2023-08-25T08:57:19.519696+0000","last_active":"2023-08-25T08:57:56.094847+0000","last_peered":"2023-08-25T08:57:56.094847+0000","last_clean":"2023-08-25T08:57:56.094847+0000","last_became_active":"2023-08-25T08:57:19.518918+0000","last_became_peered":"2023-08-25T08:57:19.518918+0000","last_unstale":"2023-08-25T08:57:56.094847+0000","last_undegraded":"2023-08-25T08:57:56.094847+0000","last_fullsized":"2023-08-25T08:57:56.094847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:39:25.972315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055842+0000","last_change":"2023-08-25T08:57:19.520706+0000","last_active":"2023-08-25T08:57:56.055842+0000","last_peered":"2023-08-25T08:57:56.055842+0000","last_clean":"2023-08-25T08:57:56.055842+0000","last_became_active":"2023-08-25T08:57:19.520333+0000","last_became_peered":"2023-08-25T08:57:19.520333+0000","last_unstale":"2023-08-25T08:57:56.055842+0000","last_undegraded":"2023-08-25T08:57:56.055842+0000","last_fullsized":"2023-08-25T08:57:56.055842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:54.763281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057288+0000","last_change":"2023-08-25T08:57:56.057288+0000","last_active":"2023-08-25T08:57:56.057288+0000","last_peered":"2023-08-25T08:57:56.057288+0000","last_clean":"2023-08-25T08:57:56.057288+0000","last_became_active":"2023-08-25T08:57:56.056839+0000","last_became_peered":"2023-08-25T08:57:56.056839+0000","last_unstale":"2023-08-25T08:57:56.057288+0000","last_undegraded":"2023-08-25T08:57:56.057288+0000","last_fullsized":"2023-08-25T08:57:56.057288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:10:06.945840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057789+0000","last_change":"2023-08-25T08:57:56.057789+0000","last_active":"2023-08-25T08:57:56.057789+0000","last_peered":"2023-08-25T08:57:56.057789+0000","last_clean":"2023-08-25T08:57:56.057789+0000","last_became_active":"2023-08-25T08:57:56.057382+0000","last_became_peered":"2023-08-25T08:57:56.057382+0000","last_unstale":"2023-08-25T08:57:56.057789+0000","last_undegraded":"2023-08-25T08:57:56.057789+0000","last_fullsized":"2023-08-25T08:57:56.057789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:28:35.484376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369844+0000","last_change":"2023-08-25T08:57:19.527429+0000","last_active":"2023-08-25T08:57:56.369844+0000","last_peered":"2023-08-25T08:57:56.369844+0000","last_clean":"2023-08-25T08:57:56.369844+0000","last_became_active":"2023-08-25T08:57:19.526370+0000","last_became_peered":"2023-08-25T08:57:19.526370+0000","last_unstale":"2023-08-25T08:57:56.369844+0000","last_undegraded":"2023-08-25T08:57:56.369844+0000","last_fullsized":"2023-08-25T08:57:56.369844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:23:42.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056522+0000","last_change":"2023-08-25T08:57:19.517136+0000","last_active":"2023-08-25T08:57:56.056522+0000","last_peered":"2023-08-25T08:57:56.056522+0000","last_clean":"2023-08-25T08:57:56.056522+0000","last_became_active":"2023-08-25T08:57:19.516852+0000","last_became_peered":"2023-08-25T08:57:19.516852+0000","last_unstale":"2023-08-25T08:57:56.056522+0000","last_undegraded":"2023-08-25T08:57:56.056522+0000","last_fullsized":"2023-08-25T08:57:56.056522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:43.370462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048485+0000","last_change":"2023-08-25T08:57:19.522122+0000","last_active":"2023-08-25T08:57:56.048485+0000","last_peered":"2023-08-25T08:57:56.048485+0000","last_clean":"2023-08-25T08:57:56.048485+0000","last_became_active":"2023-08-25T08:57:19.520984+0000","last_became_peered":"2023-08-25T08:57:19.520984+0000","last_unstale":"2023-08-25T08:57:56.048485+0000","last_undegraded":"2023-08-25T08:57:56.048485+0000","last_fullsized":"2023-08-25T08:57:56.048485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:23:36.989284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053380+0000","last_change":"2023-08-25T08:57:19.531356+0000","last_active":"2023-08-25T08:57:56.053380+0000","last_peered":"2023-08-25T08:57:56.053380+0000","last_clean":"2023-08-25T08:57:56.053380+0000","last_became_active":"2023-08-25T08:57:19.531050+0000","last_became_peered":"2023-08-25T08:57:19.531050+0000","last_unstale":"2023-08-25T08:57:56.053380+0000","last_undegraded":"2023-08-25T08:57:56.053380+0000","last_fullsized":"2023-08-25T08:57:56.053380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:11.201931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.094817+0000","last_change":"2023-08-25T08:57:19.520091+0000","last_active":"2023-08-25T08:57:56.094817+0000","last_peered":"2023-08-25T08:57:56.094817+0000","last_clean":"2023-08-25T08:57:56.094817+0000","last_became_active":"2023-08-25T08:57:19.519356+0000","last_became_peered":"2023-08-25T08:57:19.519356+0000","last_unstale":"2023-08-25T08:57:56.094817+0000","last_undegraded":"2023-08-25T08:57:56.094817+0000","last_fullsized":"2023-08-25T08:57:56.094817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:23.224068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049755+0000","last_change":"2023-08-25T08:57:19.515446+0000","last_active":"2023-08-25T08:57:56.049755+0000","last_peered":"2023-08-25T08:57:56.049755+0000","last_clean":"2023-08-25T08:57:56.049755+0000","last_became_active":"2023-08-25T08:57:19.514376+0000","last_became_peered":"2023-08-25T08:57:19.514376+0000","last_unstale":"2023-08-25T08:57:56.049755+0000","last_undegraded":"2023-08-25T08:57:56.049755+0000","last_fullsized":"2023-08-25T08:57:56.049755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:21:52.606389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369501+0000","last_change":"2023-08-25T08:57:19.521629+0000","last_active":"2023-08-25T08:57:56.369501+0000","last_peered":"2023-08-25T08:57:56.369501+0000","last_clean":"2023-08-25T08:57:56.369501+0000","last_became_active":"2023-08-25T08:57:19.521307+0000","last_became_peered":"2023-08-25T08:57:19.521307+0000","last_unstale":"2023-08-25T08:57:56.369501+0000","last_undegraded":"2023-08-25T08:57:56.369501+0000","last_fullsized":"2023-08-25T08:57:56.369501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:03.613694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369389+0000","last_change":"2023-08-25T08:57:19.521670+0000","last_active":"2023-08-25T08:57:56.369389+0000","last_peered":"2023-08-25T08:57:56.369389+0000","last_clean":"2023-08-25T08:57:56.369389+0000","last_became_active":"2023-08-25T08:57:19.521325+0000","last_became_peered":"2023-08-25T08:57:19.521325+0000","last_unstale":"2023-08-25T08:57:56.369389+0000","last_undegraded":"2023-08-25T08:57:56.369389+0000","last_fullsized":"2023-08-25T08:57:56.369389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:53.426927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055725+0000","last_change":"2023-08-25T08:57:19.526733+0000","last_active":"2023-08-25T08:57:56.055725+0000","last_peered":"2023-08-25T08:57:56.055725+0000","last_clean":"2023-08-25T08:57:56.055725+0000","last_became_active":"2023-08-25T08:57:19.525692+0000","last_became_peered":"2023-08-25T08:57:19.525692+0000","last_unstale":"2023-08-25T08:57:56.055725+0000","last_undegraded":"2023-08-25T08:57:56.055725+0000","last_fullsized":"2023-08-25T08:57:56.055725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:10:16.975270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053155+0000","last_change":"2023-08-25T08:57:19.519289+0000","last_active":"2023-08-25T08:57:56.053155+0000","last_peered":"2023-08-25T08:57:56.053155+0000","last_clean":"2023-08-25T08:57:56.053155+0000","last_became_active":"2023-08-25T08:57:19.518584+0000","last_became_peered":"2023-08-25T08:57:19.518584+0000","last_unstale":"2023-08-25T08:57:56.053155+0000","last_undegraded":"2023-08-25T08:57:56.053155+0000","last_fullsized":"2023-08-25T08:57:56.053155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:00:48.505383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052281+0000","last_change":"2023-08-25T08:57:19.525090+0000","last_active":"2023-08-25T08:57:56.052281+0000","last_peered":"2023-08-25T08:57:56.052281+0000","last_clean":"2023-08-25T08:57:56.052281+0000","last_became_active":"2023-08-25T08:57:19.524543+0000","last_became_peered":"2023-08-25T08:57:19.524543+0000","last_unstale":"2023-08-25T08:57:56.052281+0000","last_undegraded":"2023-08-25T08:57:56.052281+0000","last_fullsized":"2023-08-25T08:57:56.052281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:28:37.398394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049079+0000","last_change":"2023-08-25T08:57:19.526761+0000","last_active":"2023-08-25T08:57:56.049079+0000","last_peered":"2023-08-25T08:57:56.049079+0000","last_clean":"2023-08-25T08:57:56.049079+0000","last_became_active":"2023-08-25T08:57:19.526421+0000","last_became_peered":"2023-08-25T08:57:19.526421+0000","last_unstale":"2023-08-25T08:57:56.049079+0000","last_undegraded":"2023-08-25T08:57:56.049079+0000","last_fullsized":"2023-08-25T08:57:56.049079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:04.396487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052308+0000","last_change":"2023-08-25T08:57:18.717584+0000","last_active":"2023-08-25T08:57:56.052308+0000","last_peered":"2023-08-25T08:57:56.052308+0000","last_clean":"2023-08-25T08:57:56.052308+0000","last_became_active":"2023-08-25T08:57:18.717284+0000","last_became_peered":"2023-08-25T08:57:18.717284+0000","last_unstale":"2023-08-25T08:57:56.052308+0000","last_undegraded":"2023-08-25T08:57:56.052308+0000","last_fullsized":"2023-08-25T08:57:56.052308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:56:39.309635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370697+0000","last_change":"2023-08-25T08:57:19.527089+0000","last_active":"2023-08-25T08:57:56.370697+0000","last_peered":"2023-08-25T08:57:56.370697+0000","last_clean":"2023-08-25T08:57:56.370697+0000","last_became_active":"2023-08-25T08:57:19.526118+0000","last_became_peered":"2023-08-25T08:57:19.526118+0000","last_unstale":"2023-08-25T08:57:56.370697+0000","last_undegraded":"2023-08-25T08:57:56.370697+0000","last_fullsized":"2023-08-25T08:57:56.370697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:01.852226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370821+0000","last_change":"2023-08-25T08:57:18.720269+0000","last_active":"2023-08-25T08:57:56.370821+0000","last_peered":"2023-08-25T08:57:56.370821+0000","last_clean":"2023-08-25T08:57:56.370821+0000","last_became_active":"2023-08-25T08:57:18.719878+0000","last_became_peered":"2023-08-25T08:57:18.719878+0000","last_unstale":"2023-08-25T08:57:56.370821+0000","last_undegraded":"2023-08-25T08:57:56.370821+0000","last_fullsized":"2023-08-25T08:57:56.370821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:01:44.356022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370249+0000","last_change":"2023-08-25T08:57:19.531835+0000","last_active":"2023-08-25T08:57:56.370249+0000","last_peered":"2023-08-25T08:57:56.370249+0000","last_clean":"2023-08-25T08:57:56.370249+0000","last_became_active":"2023-08-25T08:57:19.531409+0000","last_became_peered":"2023-08-25T08:57:19.531409+0000","last_unstale":"2023-08-25T08:57:56.370249+0000","last_undegraded":"2023-08-25T08:57:56.370249+0000","last_fullsized":"2023-08-25T08:57:56.370249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:49:14.438548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.099661+0000","last_change":"2023-08-25T08:57:56.099661+0000","last_active":"2023-08-25T08:57:56.099661+0000","last_peered":"2023-08-25T08:57:56.099661+0000","last_clean":"2023-08-25T08:57:56.099661+0000","last_became_active":"2023-08-25T08:57:56.095753+0000","last_became_peered":"2023-08-25T08:57:56.095753+0000","last_unstale":"2023-08-25T08:57:56.099661+0000","last_undegraded":"2023-08-25T08:57:56.099661+0000","last_fullsized":"2023-08-25T08:57:56.099661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:38:49.238858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056179+0000","last_change":"2023-08-25T08:57:19.526495+0000","last_active":"2023-08-25T08:57:56.056179+0000","last_peered":"2023-08-25T08:57:56.056179+0000","last_clean":"2023-08-25T08:57:56.056179+0000","last_became_active":"2023-08-25T08:57:19.525204+0000","last_became_peered":"2023-08-25T08:57:19.525204+0000","last_unstale":"2023-08-25T08:57:56.056179+0000","last_undegraded":"2023-08-25T08:57:56.056179+0000","last_fullsized":"2023-08-25T08:57:56.056179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:27:16.103151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.062013+0000","last_change":"2023-08-25T08:57:56.061586+0000","last_active":"2023-08-25T08:57:56.062013+0000","last_peered":"2023-08-25T08:57:56.062013+0000","last_clean":"2023-08-25T08:57:56.062013+0000","last_became_active":"2023-08-25T08:57:56.057255+0000","last_became_peered":"2023-08-25T08:57:56.057255+0000","last_unstale":"2023-08-25T08:57:56.062013+0000","last_undegraded":"2023-08-25T08:57:56.062013+0000","last_fullsized":"2023-08-25T08:57:56.062013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:37:36.084255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057937+0000","last_change":"2023-08-25T08:57:56.057937+0000","last_active":"2023-08-25T08:57:56.057937+0000","last_peered":"2023-08-25T08:57:56.057937+0000","last_clean":"2023-08-25T08:57:56.057937+0000","last_became_active":"2023-08-25T08:57:56.057578+0000","last_became_peered":"2023-08-25T08:57:56.057578+0000","last_unstale":"2023-08-25T08:57:56.057937+0000","last_undegraded":"2023-08-25T08:57:56.057937+0000","last_fullsized":"2023-08-25T08:57:56.057937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:07:21.225532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:06.093109+0000","last_change":"2023-08-25T08:57:18.715563+0000","last_active":"2023-08-25T08:58:06.093109+0000","last_peered":"2023-08-25T08:58:06.093109+0000","last_clean":"2023-08-25T08:58:06.093109+0000","last_became_active":"2023-08-25T08:57:18.715269+0000","last_became_peered":"2023-08-25T08:57:18.715269+0000","last_unstale":"2023-08-25T08:58:06.093109+0000","last_undegraded":"2023-08-25T08:58:06.093109+0000","last_fullsized":"2023-08-25T08:58:06.093109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:24:34.134451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055868+0000","last_change":"2023-08-25T08:57:19.518705+0000","last_active":"2023-08-25T08:57:56.055868+0000","last_peered":"2023-08-25T08:57:56.055868+0000","last_clean":"2023-08-25T08:57:56.055868+0000","last_became_active":"2023-08-25T08:57:19.518422+0000","last_became_peered":"2023-08-25T08:57:19.518422+0000","last_unstale":"2023-08-25T08:57:56.055868+0000","last_undegraded":"2023-08-25T08:57:56.055868+0000","last_fullsized":"2023-08-25T08:57:56.055868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:16:21.057980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.047907+0000","last_change":"2023-08-25T08:57:18.714087+0000","last_active":"2023-08-25T08:57:56.047907+0000","last_peered":"2023-08-25T08:57:56.047907+0000","last_clean":"2023-08-25T08:57:56.047907+0000","last_became_active":"2023-08-25T08:57:18.713777+0000","last_became_peered":"2023-08-25T08:57:18.713777+0000","last_unstale":"2023-08-25T08:57:56.047907+0000","last_undegraded":"2023-08-25T08:57:56.047907+0000","last_fullsized":"2023-08-25T08:57:56.047907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:35.995427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051603+0000","last_change":"2023-08-25T08:57:19.518068+0000","last_active":"2023-08-25T08:57:56.051603+0000","last_peered":"2023-08-25T08:57:56.051603+0000","last_clean":"2023-08-25T08:57:56.051603+0000","last_became_active":"2023-08-25T08:57:19.517752+0000","last_became_peered":"2023-08-25T08:57:19.517752+0000","last_unstale":"2023-08-25T08:57:56.051603+0000","last_undegraded":"2023-08-25T08:57:56.051603+0000","last_fullsized":"2023-08-25T08:57:56.051603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:20.053896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049546+0000","last_change":"2023-08-25T08:57:18.713652+0000","last_active":"2023-08-25T08:57:56.049546+0000","last_peered":"2023-08-25T08:57:56.049546+0000","last_clean":"2023-08-25T08:57:56.049546+0000","last_became_active":"2023-08-25T08:57:18.713348+0000","last_became_peered":"2023-08-25T08:57:18.713348+0000","last_unstale":"2023-08-25T08:57:56.049546+0000","last_undegraded":"2023-08-25T08:57:56.049546+0000","last_fullsized":"2023-08-25T08:57:56.049546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:32.759661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057655+0000","last_change":"2023-08-25T08:57:56.057655+0000","last_active":"2023-08-25T08:57:56.057655+0000","last_peered":"2023-08-25T08:57:56.057655+0000","last_clean":"2023-08-25T08:57:56.057655+0000","last_became_active":"2023-08-25T08:57:56.057272+0000","last_became_peered":"2023-08-25T08:57:56.057272+0000","last_unstale":"2023-08-25T08:57:56.057655+0000","last_undegraded":"2023-08-25T08:57:56.057655+0000","last_fullsized":"2023-08-25T08:57:56.057655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:46.525543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051289+0000","last_change":"2023-08-25T08:57:18.713735+0000","last_active":"2023-08-25T08:57:56.051289+0000","last_peered":"2023-08-25T08:57:56.051289+0000","last_clean":"2023-08-25T08:57:56.051289+0000","last_became_active":"2023-08-25T08:57:18.713421+0000","last_became_peered":"2023-08-25T08:57:18.713421+0000","last_unstale":"2023-08-25T08:57:56.051289+0000","last_undegraded":"2023-08-25T08:57:56.051289+0000","last_fullsized":"2023-08-25T08:57:56.051289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:51.283967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052756+0000","last_change":"2023-08-25T08:57:19.513137+0000","last_active":"2023-08-25T08:57:56.052756+0000","last_peered":"2023-08-25T08:57:56.052756+0000","last_clean":"2023-08-25T08:57:56.052756+0000","last_became_active":"2023-08-25T08:57:19.512804+0000","last_became_peered":"2023-08-25T08:57:19.512804+0000","last_unstale":"2023-08-25T08:57:56.052756+0000","last_undegraded":"2023-08-25T08:57:56.052756+0000","last_fullsized":"2023-08-25T08:57:56.052756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:56.094424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:09.096746+0000","last_change":"2023-08-25T08:57:18.712855+0000","last_active":"2023-08-25T08:58:09.096746+0000","last_peered":"2023-08-25T08:58:09.096746+0000","last_clean":"2023-08-25T08:58:09.096746+0000","last_became_active":"2023-08-25T08:57:18.712327+0000","last_became_peered":"2023-08-25T08:57:18.712327+0000","last_unstale":"2023-08-25T08:58:09.096746+0000","last_undegraded":"2023-08-25T08:58:09.096746+0000","last_fullsized":"2023-08-25T08:58:09.096746+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:54.583482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056659+0000","last_change":"2023-08-25T08:57:19.526933+0000","last_active":"2023-08-25T08:57:56.056659+0000","last_peered":"2023-08-25T08:57:56.056659+0000","last_clean":"2023-08-25T08:57:56.056659+0000","last_became_active":"2023-08-25T08:57:19.524798+0000","last_became_peered":"2023-08-25T08:57:19.524798+0000","last_unstale":"2023-08-25T08:57:56.056659+0000","last_undegraded":"2023-08-25T08:57:56.056659+0000","last_fullsized":"2023-08-25T08:57:56.056659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:42:52.044387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053324+0000","last_change":"2023-08-25T08:57:18.714182+0000","last_active":"2023-08-25T08:57:56.053324+0000","last_peered":"2023-08-25T08:57:56.053324+0000","last_clean":"2023-08-25T08:57:56.053324+0000","last_became_active":"2023-08-25T08:57:18.713872+0000","last_became_peered":"2023-08-25T08:57:18.713872+0000","last_unstale":"2023-08-25T08:57:56.053324+0000","last_undegraded":"2023-08-25T08:57:56.053324+0000","last_fullsized":"2023-08-25T08:57:56.053324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:54.231840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370131+0000","last_change":"2023-08-25T08:57:19.530745+0000","last_active":"2023-08-25T08:57:56.370131+0000","last_peered":"2023-08-25T08:57:56.370131+0000","last_clean":"2023-08-25T08:57:56.370131+0000","last_became_active":"2023-08-25T08:57:19.530397+0000","last_became_peered":"2023-08-25T08:57:19.530397+0000","last_unstale":"2023-08-25T08:57:56.370131+0000","last_undegraded":"2023-08-25T08:57:56.370131+0000","last_fullsized":"2023-08-25T08:57:56.370131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:34:56.224834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052690+0000","last_change":"2023-08-25T08:57:18.716995+0000","last_active":"2023-08-25T08:57:56.052690+0000","last_peered":"2023-08-25T08:57:56.052690+0000","last_clean":"2023-08-25T08:57:56.052690+0000","last_became_active":"2023-08-25T08:57:18.716672+0000","last_became_peered":"2023-08-25T08:57:18.716672+0000","last_unstale":"2023-08-25T08:57:56.052690+0000","last_undegraded":"2023-08-25T08:57:56.052690+0000","last_fullsized":"2023-08-25T08:57:56.052690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:37:49.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056350+0000","last_change":"2023-08-25T08:57:19.526608+0000","last_active":"2023-08-25T08:57:56.056350+0000","last_peered":"2023-08-25T08:57:56.056350+0000","last_clean":"2023-08-25T08:57:56.056350+0000","last_became_active":"2023-08-25T08:57:19.525614+0000","last_became_peered":"2023-08-25T08:57:19.525614+0000","last_unstale":"2023-08-25T08:57:56.056350+0000","last_undegraded":"2023-08-25T08:57:56.056350+0000","last_fullsized":"2023-08-25T08:57:56.056350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:42.895003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:10.098313+0000","last_change":"2023-08-25T08:57:18.712588+0000","last_active":"2023-08-25T08:58:10.098313+0000","last_peered":"2023-08-25T08:58:10.098313+0000","last_clean":"2023-08-25T08:58:10.098313+0000","last_became_active":"2023-08-25T08:57:18.712259+0000","last_became_peered":"2023-08-25T08:57:18.712259+0000","last_unstale":"2023-08-25T08:58:10.098313+0000","last_undegraded":"2023-08-25T08:58:10.098313+0000","last_fullsized":"2023-08-25T08:58:10.098313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:24.881049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56994,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":101,"num_read_kb":105,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056324+0000","last_change":"2023-08-25T08:57:19.517907+0000","last_active":"2023-08-25T08:57:56.056324+0000","last_peered":"2023-08-25T08:57:56.056324+0000","last_clean":"2023-08-25T08:57:56.056324+0000","last_became_active":"2023-08-25T08:57:19.517596+0000","last_became_peered":"2023-08-25T08:57:19.517596+0000","last_unstale":"2023-08-25T08:57:56.056324+0000","last_undegraded":"2023-08-25T08:57:56.056324+0000","last_fullsized":"2023-08-25T08:57:56.056324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:20.224373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049213+0000","last_change":"2023-08-25T08:57:18.721564+0000","last_active":"2023-08-25T08:57:56.049213+0000","last_peered":"2023-08-25T08:57:56.049213+0000","last_clean":"2023-08-25T08:57:56.049213+0000","last_became_active":"2023-08-25T08:57:18.721235+0000","last_became_peered":"2023-08-25T08:57:18.721235+0000","last_unstale":"2023-08-25T08:57:56.049213+0000","last_undegraded":"2023-08-25T08:57:56.049213+0000","last_fullsized":"2023-08-25T08:57:56.049213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:57:00.047065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049661+0000","last_change":"2023-08-25T08:57:19.518399+0000","last_active":"2023-08-25T08:57:56.049661+0000","last_peered":"2023-08-25T08:57:56.049661+0000","last_clean":"2023-08-25T08:57:56.049661+0000","last_became_active":"2023-08-25T08:57:19.518061+0000","last_became_peered":"2023-08-25T08:57:19.518061+0000","last_unstale":"2023-08-25T08:57:56.049661+0000","last_undegraded":"2023-08-25T08:57:56.049661+0000","last_fullsized":"2023-08-25T08:57:56.049661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:11.217763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051741+0000","last_change":"2023-08-25T08:57:18.722020+0000","last_active":"2023-08-25T08:57:56.051741+0000","last_peered":"2023-08-25T08:57:56.051741+0000","last_clean":"2023-08-25T08:57:56.051741+0000","last_became_active":"2023-08-25T08:57:18.721707+0000","last_became_peered":"2023-08-25T08:57:18.721707+0000","last_unstale":"2023-08-25T08:57:56.051741+0000","last_undegraded":"2023-08-25T08:57:56.051741+0000","last_fullsized":"2023-08-25T08:57:56.051741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:47:40.076757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053276+0000","last_change":"2023-08-25T08:57:19.513954+0000","last_active":"2023-08-25T08:57:56.053276+0000","last_peered":"2023-08-25T08:57:56.053276+0000","last_clean":"2023-08-25T08:57:56.053276+0000","last_became_active":"2023-08-25T08:57:19.513645+0000","last_became_peered":"2023-08-25T08:57:19.513645+0000","last_unstale":"2023-08-25T08:57:56.053276+0000","last_undegraded":"2023-08-25T08:57:56.053276+0000","last_fullsized":"2023-08-25T08:57:56.053276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:24:16.072144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:11.117214+0000","last_change":"2023-08-25T08:57:18.711499+0000","last_active":"2023-08-25T08:58:11.117214+0000","last_peered":"2023-08-25T08:58:11.117214+0000","last_clean":"2023-08-25T08:58:11.117214+0000","last_became_active":"2023-08-25T08:57:18.711223+0000","last_became_peered":"2023-08-25T08:57:18.711223+0000","last_unstale":"2023-08-25T08:58:11.117214+0000","last_undegraded":"2023-08-25T08:58:11.117214+0000","last_fullsized":"2023-08-25T08:58:11.117214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:46.247137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052602+0000","last_change":"2023-08-25T08:57:19.518663+0000","last_active":"2023-08-25T08:57:56.052602+0000","last_peered":"2023-08-25T08:57:56.052602+0000","last_clean":"2023-08-25T08:57:56.052602+0000","last_became_active":"2023-08-25T08:57:19.518394+0000","last_became_peered":"2023-08-25T08:57:19.518394+0000","last_unstale":"2023-08-25T08:57:56.052602+0000","last_undegraded":"2023-08-25T08:57:56.052602+0000","last_fullsized":"2023-08-25T08:57:56.052602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:54:23.798982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055719+0000","last_change":"2023-08-25T08:57:18.718554+0000","last_active":"2023-08-25T08:57:56.055719+0000","last_peered":"2023-08-25T08:57:56.055719+0000","last_clean":"2023-08-25T08:57:56.055719+0000","last_became_active":"2023-08-25T08:57:18.718283+0000","last_became_peered":"2023-08-25T08:57:18.718283+0000","last_unstale":"2023-08-25T08:57:56.055719+0000","last_undegraded":"2023-08-25T08:57:56.055719+0000","last_fullsized":"2023-08-25T08:57:56.055719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:19:51.813157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051440+0000","last_change":"2023-08-25T08:57:19.520856+0000","last_active":"2023-08-25T08:57:56.051440+0000","last_peered":"2023-08-25T08:57:56.051440+0000","last_clean":"2023-08-25T08:57:56.051440+0000","last_became_active":"2023-08-25T08:57:19.520568+0000","last_became_peered":"2023-08-25T08:57:19.520568+0000","last_unstale":"2023-08-25T08:57:56.051440+0000","last_undegraded":"2023-08-25T08:57:56.051440+0000","last_fullsized":"2023-08-25T08:57:56.051440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:12.281170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055834+0000","last_change":"2023-08-25T08:57:18.715972+0000","last_active":"2023-08-25T08:57:56.055834+0000","last_peered":"2023-08-25T08:57:56.055834+0000","last_clean":"2023-08-25T08:57:56.055834+0000","last_became_active":"2023-08-25T08:57:18.715519+0000","last_became_peered":"2023-08-25T08:57:18.715519+0000","last_unstale":"2023-08-25T08:57:56.055834+0000","last_undegraded":"2023-08-25T08:57:56.055834+0000","last_fullsized":"2023-08-25T08:57:56.055834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:59.984267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057104+0000","last_change":"2023-08-25T08:57:56.057104+0000","last_active":"2023-08-25T08:57:56.057104+0000","last_peered":"2023-08-25T08:57:56.057104+0000","last_clean":"2023-08-25T08:57:56.057104+0000","last_became_active":"2023-08-25T08:57:56.056755+0000","last_became_peered":"2023-08-25T08:57:56.056755+0000","last_unstale":"2023-08-25T08:57:56.057104+0000","last_undegraded":"2023-08-25T08:57:56.057104+0000","last_fullsized":"2023-08-25T08:57:56.057104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:58:33.964262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053523+0000","last_change":"2023-08-25T08:57:18.713835+0000","last_active":"2023-08-25T08:57:56.053523+0000","last_peered":"2023-08-25T08:57:56.053523+0000","last_clean":"2023-08-25T08:57:56.053523+0000","last_became_active":"2023-08-25T08:57:18.713510+0000","last_became_peered":"2023-08-25T08:57:18.713510+0000","last_unstale":"2023-08-25T08:57:56.053523+0000","last_undegraded":"2023-08-25T08:57:56.053523+0000","last_fullsized":"2023-08-25T08:57:56.053523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:50.992010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048188+0000","last_change":"2023-08-25T08:57:19.526979+0000","last_active":"2023-08-25T08:57:56.048188+0000","last_peered":"2023-08-25T08:57:56.048188+0000","last_clean":"2023-08-25T08:57:56.048188+0000","last_became_active":"2023-08-25T08:57:19.526660+0000","last_became_peered":"2023-08-25T08:57:19.526660+0000","last_unstale":"2023-08-25T08:57:56.048188+0000","last_undegraded":"2023-08-25T08:57:56.048188+0000","last_fullsized":"2023-08-25T08:57:56.048188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:45:27.029396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053600+0000","last_change":"2023-08-25T08:57:18.717952+0000","last_active":"2023-08-25T08:57:56.053600+0000","last_peered":"2023-08-25T08:57:56.053600+0000","last_clean":"2023-08-25T08:57:56.053600+0000","last_became_active":"2023-08-25T08:57:18.717643+0000","last_became_peered":"2023-08-25T08:57:18.717643+0000","last_unstale":"2023-08-25T08:57:56.053600+0000","last_undegraded":"2023-08-25T08:57:56.053600+0000","last_fullsized":"2023-08-25T08:57:56.053600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:14:18.972908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048153+0000","last_change":"2023-08-25T08:57:19.521718+0000","last_active":"2023-08-25T08:57:56.048153+0000","last_peered":"2023-08-25T08:57:56.048153+0000","last_clean":"2023-08-25T08:57:56.048153+0000","last_became_active":"2023-08-25T08:57:19.521246+0000","last_became_peered":"2023-08-25T08:57:19.521246+0000","last_unstale":"2023-08-25T08:57:56.048153+0000","last_undegraded":"2023-08-25T08:57:56.048153+0000","last_fullsized":"2023-08-25T08:57:56.048153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:33.050413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055690+0000","last_change":"2023-08-25T08:57:56.055690+0000","last_active":"2023-08-25T08:57:56.055690+0000","last_peered":"2023-08-25T08:57:56.055690+0000","last_clean":"2023-08-25T08:57:56.055690+0000","last_became_active":"2023-08-25T08:57:56.055315+0000","last_became_peered":"2023-08-25T08:57:56.055315+0000","last_unstale":"2023-08-25T08:57:56.055690+0000","last_undegraded":"2023-08-25T08:57:56.055690+0000","last_fullsized":"2023-08-25T08:57:56.055690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:46:47.588215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:07.110864+0000","last_change":"2023-08-25T08:57:18.718657+0000","last_active":"2023-08-25T08:58:07.110864+0000","last_peered":"2023-08-25T08:58:07.110864+0000","last_clean":"2023-08-25T08:58:07.110864+0000","last_became_active":"2023-08-25T08:57:18.718348+0000","last_became_peered":"2023-08-25T08:57:18.718348+0000","last_unstale":"2023-08-25T08:58:07.110864+0000","last_undegraded":"2023-08-25T08:58:07.110864+0000","last_fullsized":"2023-08-25T08:58:07.110864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:18:40.166063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370089+0000","last_change":"2023-08-25T08:57:19.520997+0000","last_active":"2023-08-25T08:57:56.370089+0000","last_peered":"2023-08-25T08:57:56.370089+0000","last_clean":"2023-08-25T08:57:56.370089+0000","last_became_active":"2023-08-25T08:57:19.520695+0000","last_became_peered":"2023-08-25T08:57:19.520695+0000","last_unstale":"2023-08-25T08:57:56.370089+0000","last_undegraded":"2023-08-25T08:57:56.370089+0000","last_fullsized":"2023-08-25T08:57:56.370089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:01:18.093454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.094778+0000","last_change":"2023-08-25T08:57:18.714573+0000","last_active":"2023-08-25T08:57:56.094778+0000","last_peered":"2023-08-25T08:57:56.094778+0000","last_clean":"2023-08-25T08:57:56.094778+0000","last_became_active":"2023-08-25T08:57:18.714268+0000","last_became_peered":"2023-08-25T08:57:18.714268+0000","last_unstale":"2023-08-25T08:57:56.094778+0000","last_undegraded":"2023-08-25T08:57:56.094778+0000","last_fullsized":"2023-08-25T08:57:56.094778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:49.345077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053302+0000","last_change":"2023-08-25T08:56:14.362397+0000","last_active":"2023-08-25T08:57:56.053302+0000","last_peered":"2023-08-25T08:57:56.053302+0000","last_clean":"2023-08-25T08:57:56.053302+0000","last_became_active":"2023-08-25T08:56:14.361946+0000","last_became_peered":"2023-08-25T08:56:14.361946+0000","last_unstale":"2023-08-25T08:57:56.053302+0000","last_undegraded":"2023-08-25T08:57:56.053302+0000","last_fullsized":"2023-08-25T08:57:56.053302+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:53:09.022543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:53:09.022543+0000","last_clean_scrub_stamp":"2023-08-25T08:53:09.022543+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:07:59.847147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049410+0000","last_change":"2023-08-25T08:57:19.517547+0000","last_active":"2023-08-25T08:57:56.049410+0000","last_peered":"2023-08-25T08:57:56.049410+0000","last_clean":"2023-08-25T08:57:56.049410+0000","last_became_active":"2023-08-25T08:57:19.517221+0000","last_became_peered":"2023-08-25T08:57:19.517221+0000","last_unstale":"2023-08-25T08:57:56.049410+0000","last_undegraded":"2023-08-25T08:57:56.049410+0000","last_fullsized":"2023-08-25T08:57:56.049410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:34:19.765696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052875+0000","last_change":"2023-08-25T08:57:18.715769+0000","last_active":"2023-08-25T08:57:56.052875+0000","last_peered":"2023-08-25T08:57:56.052875+0000","last_clean":"2023-08-25T08:57:56.052875+0000","last_became_active":"2023-08-25T08:57:18.715399+0000","last_became_peered":"2023-08-25T08:57:18.715399+0000","last_unstale":"2023-08-25T08:57:56.052875+0000","last_undegraded":"2023-08-25T08:57:56.052875+0000","last_fullsized":"2023-08-25T08:57:56.052875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:52:20.906518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056347+0000","last_change":"2023-08-25T08:57:18.720658+0000","last_active":"2023-08-25T08:57:56.056347+0000","last_peered":"2023-08-25T08:57:56.056347+0000","last_clean":"2023-08-25T08:57:56.056347+0000","last_became_active":"2023-08-25T08:57:18.720419+0000","last_became_peered":"2023-08-25T08:57:18.720419+0000","last_unstale":"2023-08-25T08:57:56.056347+0000","last_undegraded":"2023-08-25T08:57:56.056347+0000","last_fullsized":"2023-08-25T08:57:56.056347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:00:29.486469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049985+0000","last_change":"2023-08-25T08:57:19.513257+0000","last_active":"2023-08-25T08:57:56.049985+0000","last_peered":"2023-08-25T08:57:56.049985+0000","last_clean":"2023-08-25T08:57:56.049985+0000","last_became_active":"2023-08-25T08:57:19.512852+0000","last_became_peered":"2023-08-25T08:57:19.512852+0000","last_unstale":"2023-08-25T08:57:56.049985+0000","last_undegraded":"2023-08-25T08:57:56.049985+0000","last_fullsized":"2023-08-25T08:57:56.049985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:15.512558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052981+0000","last_change":"2023-08-25T08:57:18.718740+0000","last_active":"2023-08-25T08:57:56.052981+0000","last_peered":"2023-08-25T08:57:56.052981+0000","last_clean":"2023-08-25T08:57:56.052981+0000","last_became_active":"2023-08-25T08:57:18.718433+0000","last_became_peered":"2023-08-25T08:57:18.718433+0000","last_unstale":"2023-08-25T08:57:56.052981+0000","last_undegraded":"2023-08-25T08:57:56.052981+0000","last_fullsized":"2023-08-25T08:57:56.052981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:18:39.835570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370989+0000","last_change":"2023-08-25T08:57:19.525867+0000","last_active":"2023-08-25T08:57:56.370989+0000","last_peered":"2023-08-25T08:57:56.370989+0000","last_clean":"2023-08-25T08:57:56.370989+0000","last_became_active":"2023-08-25T08:57:19.525492+0000","last_became_peered":"2023-08-25T08:57:19.525492+0000","last_unstale":"2023-08-25T08:57:56.370989+0000","last_undegraded":"2023-08-25T08:57:56.370989+0000","last_fullsized":"2023-08-25T08:57:56.370989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:30.289254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.095090+0000","last_change":"2023-08-25T08:57:18.711447+0000","last_active":"2023-08-25T08:57:56.095090+0000","last_peered":"2023-08-25T08:57:56.095090+0000","last_clean":"2023-08-25T08:57:56.095090+0000","last_became_active":"2023-08-25T08:57:18.711164+0000","last_became_peered":"2023-08-25T08:57:18.711164+0000","last_unstale":"2023-08-25T08:57:56.095090+0000","last_undegraded":"2023-08-25T08:57:56.095090+0000","last_fullsized":"2023-08-25T08:57:56.095090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:50.213859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369841+0000","last_change":"2023-08-25T08:57:19.528927+0000","last_active":"2023-08-25T08:57:56.369841+0000","last_peered":"2023-08-25T08:57:56.369841+0000","last_clean":"2023-08-25T08:57:56.369841+0000","last_became_active":"2023-08-25T08:57:19.528281+0000","last_became_peered":"2023-08-25T08:57:19.528281+0000","last_unstale":"2023-08-25T08:57:56.369841+0000","last_undegraded":"2023-08-25T08:57:56.369841+0000","last_fullsized":"2023-08-25T08:57:56.369841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:53:16.828292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055890+0000","last_change":"2023-08-25T08:57:18.713474+0000","last_active":"2023-08-25T08:57:56.055890+0000","last_peered":"2023-08-25T08:57:56.055890+0000","last_clean":"2023-08-25T08:57:56.055890+0000","last_became_active":"2023-08-25T08:57:18.713147+0000","last_became_peered":"2023-08-25T08:57:18.713147+0000","last_unstale":"2023-08-25T08:57:56.055890+0000","last_undegraded":"2023-08-25T08:57:56.055890+0000","last_fullsized":"2023-08-25T08:57:56.055890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:31:07.337490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056063+0000","last_change":"2023-08-25T08:57:19.515035+0000","last_active":"2023-08-25T08:57:56.056063+0000","last_peered":"2023-08-25T08:57:56.056063+0000","last_clean":"2023-08-25T08:57:56.056063+0000","last_became_active":"2023-08-25T08:57:19.514729+0000","last_became_peered":"2023-08-25T08:57:19.514729+0000","last_unstale":"2023-08-25T08:57:56.056063+0000","last_undegraded":"2023-08-25T08:57:56.056063+0000","last_fullsized":"2023-08-25T08:57:56.056063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:07:20.953665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369863+0000","last_change":"2023-08-25T08:57:18.721451+0000","last_active":"2023-08-25T08:57:56.369863+0000","last_peered":"2023-08-25T08:57:56.369863+0000","last_clean":"2023-08-25T08:57:56.369863+0000","last_became_active":"2023-08-25T08:57:18.721132+0000","last_became_peered":"2023-08-25T08:57:18.721132+0000","last_unstale":"2023-08-25T08:57:56.369863+0000","last_undegraded":"2023-08-25T08:57:56.369863+0000","last_fullsized":"2023-08-25T08:57:56.369863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:50.473107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052688+0000","last_change":"2023-08-25T08:57:19.519649+0000","last_active":"2023-08-25T08:57:56.052688+0000","last_peered":"2023-08-25T08:57:56.052688+0000","last_clean":"2023-08-25T08:57:56.052688+0000","last_became_active":"2023-08-25T08:57:19.518907+0000","last_became_peered":"2023-08-25T08:57:19.518907+0000","last_unstale":"2023-08-25T08:57:56.052688+0000","last_undegraded":"2023-08-25T08:57:56.052688+0000","last_fullsized":"2023-08-25T08:57:56.052688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:32.052200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048751+0000","last_change":"2023-08-25T08:57:18.722996+0000","last_active":"2023-08-25T08:57:56.048751+0000","last_peered":"2023-08-25T08:57:56.048751+0000","last_clean":"2023-08-25T08:57:56.048751+0000","last_became_active":"2023-08-25T08:57:18.722662+0000","last_became_peered":"2023-08-25T08:57:18.722662+0000","last_unstale":"2023-08-25T08:57:56.048751+0000","last_undegraded":"2023-08-25T08:57:56.048751+0000","last_fullsized":"2023-08-25T08:57:56.048751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:40:56.622549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370066+0000","last_change":"2023-08-25T08:57:19.529138+0000","last_active":"2023-08-25T08:57:56.370066+0000","last_peered":"2023-08-25T08:57:56.370066+0000","last_clean":"2023-08-25T08:57:56.370066+0000","last_became_active":"2023-08-25T08:57:19.527485+0000","last_became_peered":"2023-08-25T08:57:19.527485+0000","last_unstale":"2023-08-25T08:57:56.370066+0000","last_undegraded":"2023-08-25T08:57:56.370066+0000","last_fullsized":"2023-08-25T08:57:56.370066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:32:46.869523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056148+0000","last_change":"2023-08-25T08:57:18.717811+0000","last_active":"2023-08-25T08:57:56.056148+0000","last_peered":"2023-08-25T08:57:56.056148+0000","last_clean":"2023-08-25T08:57:56.056148+0000","last_became_active":"2023-08-25T08:57:18.717573+0000","last_became_peered":"2023-08-25T08:57:18.717573+0000","last_unstale":"2023-08-25T08:57:56.056148+0000","last_undegraded":"2023-08-25T08:57:56.056148+0000","last_fullsized":"2023-08-25T08:57:56.056148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:11.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053381+0000","last_change":"2023-08-25T08:57:19.528271+0000","last_active":"2023-08-25T08:57:56.053381+0000","last_peered":"2023-08-25T08:57:56.053381+0000","last_clean":"2023-08-25T08:57:56.053381+0000","last_became_active":"2023-08-25T08:57:19.528017+0000","last_became_peered":"2023-08-25T08:57:19.528017+0000","last_unstale":"2023-08-25T08:57:56.053381+0000","last_undegraded":"2023-08-25T08:57:56.053381+0000","last_fullsized":"2023-08-25T08:57:56.053381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:03:37.995896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049179+0000","last_change":"2023-08-25T08:57:18.715063+0000","last_active":"2023-08-25T08:57:56.049179+0000","last_peered":"2023-08-25T08:57:56.049179+0000","last_clean":"2023-08-25T08:57:56.049179+0000","last_became_active":"2023-08-25T08:57:18.714755+0000","last_became_peered":"2023-08-25T08:57:18.714755+0000","last_unstale":"2023-08-25T08:57:56.049179+0000","last_undegraded":"2023-08-25T08:57:56.049179+0000","last_fullsized":"2023-08-25T08:57:56.049179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:12:11.389492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053491+0000","last_change":"2023-08-25T08:57:19.528557+0000","last_active":"2023-08-25T08:57:56.053491+0000","last_peered":"2023-08-25T08:57:56.053491+0000","last_clean":"2023-08-25T08:57:56.053491+0000","last_became_active":"2023-08-25T08:57:19.528284+0000","last_became_peered":"2023-08-25T08:57:19.528284+0000","last_unstale":"2023-08-25T08:57:56.053491+0000","last_undegraded":"2023-08-25T08:57:56.053491+0000","last_fullsized":"2023-08-25T08:57:56.053491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:36:07.147532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052206+0000","last_change":"2023-08-25T08:57:18.722893+0000","last_active":"2023-08-25T08:57:56.052206+0000","last_peered":"2023-08-25T08:57:56.052206+0000","last_clean":"2023-08-25T08:57:56.052206+0000","last_became_active":"2023-08-25T08:57:18.722553+0000","last_became_peered":"2023-08-25T08:57:18.722553+0000","last_unstale":"2023-08-25T08:57:56.052206+0000","last_undegraded":"2023-08-25T08:57:56.052206+0000","last_fullsized":"2023-08-25T08:57:56.052206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:25:49.917208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052837+0000","last_change":"2023-08-25T08:57:19.517975+0000","last_active":"2023-08-25T08:57:56.052837+0000","last_peered":"2023-08-25T08:57:56.052837+0000","last_clean":"2023-08-25T08:57:56.052837+0000","last_became_active":"2023-08-25T08:57:19.517715+0000","last_became_peered":"2023-08-25T08:57:19.517715+0000","last_unstale":"2023-08-25T08:57:56.052837+0000","last_undegraded":"2023-08-25T08:57:56.052837+0000","last_fullsized":"2023-08-25T08:57:56.052837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:48.727902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049833+0000","last_change":"2023-08-25T08:57:18.717331+0000","last_active":"2023-08-25T08:57:56.049833+0000","last_peered":"2023-08-25T08:57:56.049833+0000","last_clean":"2023-08-25T08:57:56.049833+0000","last_became_active":"2023-08-25T08:57:18.716957+0000","last_became_peered":"2023-08-25T08:57:18.716957+0000","last_unstale":"2023-08-25T08:57:56.049833+0000","last_undegraded":"2023-08-25T08:57:56.049833+0000","last_fullsized":"2023-08-25T08:57:56.049833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:46:25.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052994+0000","last_change":"2023-08-25T08:57:19.515217+0000","last_active":"2023-08-25T08:57:56.052994+0000","last_peered":"2023-08-25T08:57:56.052994+0000","last_clean":"2023-08-25T08:57:56.052994+0000","last_became_active":"2023-08-25T08:57:19.514870+0000","last_became_peered":"2023-08-25T08:57:19.514870+0000","last_unstale":"2023-08-25T08:57:56.052994+0000","last_undegraded":"2023-08-25T08:57:56.052994+0000","last_fullsized":"2023-08-25T08:57:56.052994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:26:25.804692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056197+0000","last_change":"2023-08-25T08:57:56.056197+0000","last_active":"2023-08-25T08:57:56.056197+0000","last_peered":"2023-08-25T08:57:56.056197+0000","last_clean":"2023-08-25T08:57:56.056197+0000","last_became_active":"2023-08-25T08:57:56.055819+0000","last_became_peered":"2023-08-25T08:57:56.055819+0000","last_unstale":"2023-08-25T08:57:56.056197+0000","last_undegraded":"2023-08-25T08:57:56.056197+0000","last_fullsized":"2023-08-25T08:57:56.056197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:32.670928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048759+0000","last_change":"2023-08-25T08:57:19.521540+0000","last_active":"2023-08-25T08:57:56.048759+0000","last_peered":"2023-08-25T08:57:56.048759+0000","last_clean":"2023-08-25T08:57:56.048759+0000","last_became_active":"2023-08-25T08:57:19.521305+0000","last_became_peered":"2023-08-25T08:57:19.521305+0000","last_unstale":"2023-08-25T08:57:56.048759+0000","last_undegraded":"2023-08-25T08:57:56.048759+0000","last_fullsized":"2023-08-25T08:57:56.048759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:31:51.139757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048847+0000","last_change":"2023-08-25T08:57:19.526830+0000","last_active":"2023-08-25T08:57:56.048847+0000","last_peered":"2023-08-25T08:57:56.048847+0000","last_clean":"2023-08-25T08:57:56.048847+0000","last_became_active":"2023-08-25T08:57:19.526396+0000","last_became_peered":"2023-08-25T08:57:19.526396+0000","last_unstale":"2023-08-25T08:57:56.048847+0000","last_undegraded":"2023-08-25T08:57:56.048847+0000","last_fullsized":"2023-08-25T08:57:56.048847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:33:00.650463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369667+0000","last_change":"2023-08-25T08:57:19.522557+0000","last_active":"2023-08-25T08:57:56.369667+0000","last_peered":"2023-08-25T08:57:56.369667+0000","last_clean":"2023-08-25T08:57:56.369667+0000","last_became_active":"2023-08-25T08:57:19.522076+0000","last_became_peered":"2023-08-25T08:57:19.522076+0000","last_unstale":"2023-08-25T08:57:56.369667+0000","last_undegraded":"2023-08-25T08:57:56.369667+0000","last_fullsized":"2023-08-25T08:57:56.369667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:25:21.874038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049942+0000","last_change":"2023-08-25T08:57:19.518894+0000","last_active":"2023-08-25T08:57:56.049942+0000","last_peered":"2023-08-25T08:57:56.049942+0000","last_clean":"2023-08-25T08:57:56.049942+0000","last_became_active":"2023-08-25T08:57:19.518599+0000","last_became_peered":"2023-08-25T08:57:19.518599+0000","last_unstale":"2023-08-25T08:57:56.049942+0000","last_undegraded":"2023-08-25T08:57:56.049942+0000","last_fullsized":"2023-08-25T08:57:56.049942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:51:02.889569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051995+0000","last_change":"2023-08-25T08:57:19.513584+0000","last_active":"2023-08-25T08:57:56.051995+0000","last_peered":"2023-08-25T08:57:56.051995+0000","last_clean":"2023-08-25T08:57:56.051995+0000","last_became_active":"2023-08-25T08:57:19.513187+0000","last_became_peered":"2023-08-25T08:57:19.513187+0000","last_unstale":"2023-08-25T08:57:56.051995+0000","last_undegraded":"2023-08-25T08:57:56.051995+0000","last_fullsized":"2023-08-25T08:57:56.051995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:50:58.855527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049253+0000","last_change":"2023-08-25T08:57:19.528534+0000","last_active":"2023-08-25T08:57:56.049253+0000","last_peered":"2023-08-25T08:57:56.049253+0000","last_clean":"2023-08-25T08:57:56.049253+0000","last_became_active":"2023-08-25T08:57:19.528186+0000","last_became_peered":"2023-08-25T08:57:19.528186+0000","last_unstale":"2023-08-25T08:57:56.049253+0000","last_undegraded":"2023-08-25T08:57:56.049253+0000","last_fullsized":"2023-08-25T08:57:56.049253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:16:19.578944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370573+0000","last_change":"2023-08-25T08:57:19.529932+0000","last_active":"2023-08-25T08:57:56.370573+0000","last_peered":"2023-08-25T08:57:56.370573+0000","last_clean":"2023-08-25T08:57:56.370573+0000","last_became_active":"2023-08-25T08:57:19.529581+0000","last_became_peered":"2023-08-25T08:57:19.529581+0000","last_unstale":"2023-08-25T08:57:56.370573+0000","last_undegraded":"2023-08-25T08:57:56.370573+0000","last_fullsized":"2023-08-25T08:57:56.370573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:53.698651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051755+0000","last_change":"2023-08-25T08:57:19.527777+0000","last_active":"2023-08-25T08:57:56.051755+0000","last_peered":"2023-08-25T08:57:56.051755+0000","last_clean":"2023-08-25T08:57:56.051755+0000","last_became_active":"2023-08-25T08:57:19.527515+0000","last_became_peered":"2023-08-25T08:57:19.527515+0000","last_unstale":"2023-08-25T08:57:56.051755+0000","last_undegraded":"2023-08-25T08:57:56.051755+0000","last_fullsized":"2023-08-25T08:57:56.051755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:10.845020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369723+0000","last_change":"2023-08-25T08:57:19.522676+0000","last_active":"2023-08-25T08:57:56.369723+0000","last_peered":"2023-08-25T08:57:56.369723+0000","last_clean":"2023-08-25T08:57:56.369723+0000","last_became_active":"2023-08-25T08:57:19.522282+0000","last_became_peered":"2023-08-25T08:57:19.522282+0000","last_unstale":"2023-08-25T08:57:56.369723+0000","last_undegraded":"2023-08-25T08:57:56.369723+0000","last_fullsized":"2023-08-25T08:57:56.369723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:36:25.933969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.371553+0000","last_change":"2023-08-25T08:57:56.371553+0000","last_active":"2023-08-25T08:57:56.371553+0000","last_peered":"2023-08-25T08:57:56.371553+0000","last_clean":"2023-08-25T08:57:56.371553+0000","last_became_active":"2023-08-25T08:57:56.371059+0000","last_became_peered":"2023-08-25T08:57:56.371059+0000","last_unstale":"2023-08-25T08:57:56.371553+0000","last_undegraded":"2023-08-25T08:57:56.371553+0000","last_fullsized":"2023-08-25T08:57:56.371553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:23:14.732042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,3],"acting":[9,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049771+0000","last_change":"2023-08-25T08:57:19.518697+0000","last_active":"2023-08-25T08:57:56.049771+0000","last_peered":"2023-08-25T08:57:56.049771+0000","last_clean":"2023-08-25T08:57:56.049771+0000","last_became_active":"2023-08-25T08:57:19.518316+0000","last_became_peered":"2023-08-25T08:57:19.518316+0000","last_unstale":"2023-08-25T08:57:56.049771+0000","last_undegraded":"2023-08-25T08:57:56.049771+0000","last_fullsized":"2023-08-25T08:57:56.049771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:56:02.648409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053300+0000","last_change":"2023-08-25T08:57:19.515293+0000","last_active":"2023-08-25T08:57:56.053300+0000","last_peered":"2023-08-25T08:57:56.053300+0000","last_clean":"2023-08-25T08:57:56.053300+0000","last_became_active":"2023-08-25T08:57:19.514883+0000","last_became_peered":"2023-08-25T08:57:19.514883+0000","last_unstale":"2023-08-25T08:57:56.053300+0000","last_undegraded":"2023-08-25T08:57:56.053300+0000","last_fullsized":"2023-08-25T08:57:56.053300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:53:25.145029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250603,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60994,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":303,"num_read_kb":215,"num_write":148,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":182982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449112,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":443966,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.88}]},{"osd":7,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.746}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.91800000000000004}]},{"osd":10,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.93200000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1092447,"data_compressed":42060,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.495},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":2.419},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.373},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.36199999999999999}]},{"osd":8,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.55600000000000005}]},{"osd":9,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.57899999999999996}]},{"osd":11,"last update":"Fri Aug 25 08:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.61299999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841566,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444202,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.64900000000000002,"15min":0.58499999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":13.968,"5min":13.968,"15min":13.968},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.1339999999999999,"5min":0.82199999999999995,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":13.868,"5min":13.868,"15min":13.868},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":0.69199999999999995}]},{"osd":2,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.81399999999999995,"15min":0.83499999999999996},"min":{"1min":0.53900000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77500000000000002,"15min":0.78600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1200000000000001,"5min":1.155,"15min":1.155},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.85099999999999998,"15min":0.88300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1919999999999999,"5min":1.641,"15min":1.641},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86399999999999999,"15min":0.88},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.749}]},{"osd":6,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.878,"15min":0.89300000000000002},"min":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.274,"5min":1.0029999999999999,"15min":0.95799999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":13.922000000000001,"5min":13.922000000000001,"15min":13.922000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.90300000000000002,"15min":0.93000000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1579999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0069999999999999,"15min":0.98899999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":6.984,"5min":6.984,"15min":6.984},"last":0.78500000000000003}]},{"osd":8,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.94399999999999995,"15min":0.97399999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0129999999999999,"5min":1.208,"15min":1.208},"last":0.876},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.92600000000000005,"15min":0.94999999999999996},"min":{"1min":0.55400000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.258,"5min":1.778,"15min":1.778},"last":0.82699999999999996}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444122,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.82499999999999996}]},{"osd":1,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.55700000000000005,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.42199999999999999},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.34599999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.50800000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84799999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.499},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68100000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.625}]},{"osd":6,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68999999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.53300000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.878,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.70499999999999996}]},{"osd":9,"last update":"Fri Aug 25 08:57:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.47899999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0780000000000001,"5min":1.155,"15min":1.155},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":1036057,"data_compressed":42060,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"min":{"1min":0.32700000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60799999999999998,"5min":0.628,"15min":0.628},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.46200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.82599999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.496},"min":{"1min":0.38800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51100000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86199999999999999,"5min":0.879,"15min":0.879},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.95099999999999996}]},{"osd":6,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73899999999999999}]},{"osd":8,"last update":"Fri Aug 25 08:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85299999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70699999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92600000000000005}]},{"osd":9,"last update":"Fri Aug 25 08:57:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.59499999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91700000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90300000000000002}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.82699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":443597,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.443,"15min":0.433},"min":{"1min":0.30299999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.64100000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47599999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.66600000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77000000000000002,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.53700000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88900000000000001,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66000000000000003,"15min":0.63},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88500000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.86799999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.754}]},{"osd":4,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.747,"15min":0.76700000000000002},"min":{"1min":0.497,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0609999999999999,"5min":3.9500000000000002,"15min":3.9500000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.52300000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.82799999999999996,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.57399999999999995}]},{"osd":5,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95399999999999996,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.78500000000000003,"15min":0.72299999999999998},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.8650000000000002,"5min":5.8650000000000002,"15min":5.8650000000000002},"last":0.88200000000000001}]},{"osd":7,"last update":"Fri Aug 25 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73299999999999998,"15min":0.71899999999999997},"min":{"1min":0.60199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.51000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70599999999999996}]},{"osd":8,"last update":"Fri Aug 25 08:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.131,"15min":1.131},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.008,"5min":1.266,"15min":1.266},"last":0.92900000000000005}]},{"osd":9,"last update":"Fri Aug 25 08:57:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.85199999999999998,"15min":0.86699999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.85199999999999998,"15min":0.87},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Aug 25 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.83399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":500105,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65200000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.621,"15min":0.56399999999999995},"min":{"1min":0.34200000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91800000000000004,"5min":5.4359999999999999,"15min":5.4359999999999999},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Aug 25 08:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.441,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri Aug 25 08:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79300000000000004,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.496},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Aug 25 08:58:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44,"15min":0.40799999999999997},"min":{"1min":0.32700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.746,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.46600000000000003,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.73499999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42399999999999999}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Aug 25 08:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83599999999999997,"5min":1.107,"15min":1.107},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.071,"5min":1.153,"15min":1.153},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Aug 25 08:58:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81999999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63200000000000001}]},{"osd":8,"last update":"Fri Aug 25 08:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.44700000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.755},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92300000000000004,"5min":1.044,"15min":1.044},"last":0.79200000000000004}]},{"osd":9,"last update":"Fri Aug 25 08:57:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95099999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Aug 25 08:57:13 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":444862,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 08:57:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60099999999999998,"15min":0.54000000000000004},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.1259999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.62,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Aug 25 08:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.46100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60999999999999999,"15min":0.56399999999999995},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Aug 25 08:55:37 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.46300000000000002,"15min":0.45000000000000001},"min":{"1min":0.34999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Aug 25 08:58:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67300000000000004,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3109999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2250000000000001,"5min":1.306,"15min":1.306},"last":0.82199999999999995}]},{"osd":5,"last update":"Fri Aug 25 08:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.65000000000000002,"15min":0.626},"min":{"1min":0.497,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Aug 25 08:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64300000000000002,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69799999999999995,"15min":0.66500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Aug 25 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.77000000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68300000000000005}]},{"osd":8,"last update":"Fri Aug 25 08:57:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.78300000000000003,"15min":0.79600000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0700000000000001,"5min":1.302,"15min":1.302},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85999999999999999,"15min":0.86699999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.224,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1}]},{"osd":9,"last update":"Fri Aug 25 08:57:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.88600000000000001,"15min":0.90400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2929999999999999,"5min":1.337,"15min":1.337},"last":0.88400000000000001}]},{"osd":10,"last update":"Fri Aug 25 08:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.89700000000000002}]},{"osd":11,"last update":"Fri Aug 25 08:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95199999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":502118,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79700000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":1.409,"15min":1.409},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.41699999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Aug 25 08:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83899999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64500000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.503},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Aug 25 08:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42899999999999999,"15min":0.42099999999999999},"min":{"1min":0.27100000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.621},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":1.032,"15min":1.032},"last":0.627},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63400000000000001,"15min":0.61799999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":1.627,"15min":1.627},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91800000000000004,"5min":1.117,"15min":1.117},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":1.097,"15min":1.097},"last":0.64500000000000002}]},{"osd":8,"last update":"Fri Aug 25 08:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.316,"5min":0.308,"15min":0.308},"max":{"1min":0.84299999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0289999999999999,"5min":1.069,"15min":1.069},"last":0.95399999999999996}]},{"osd":9,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0669999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94299999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Aug 25 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.745},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443914,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.47099999999999997,"15min":0.42099999999999999},"min":{"1min":0.35199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.45700000000000002,"15min":0.41799999999999998},"min":{"1min":0.35399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.65200000000000002,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Aug 25 08:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63700000000000001,"15min":0.60599999999999998},"min":{"1min":0.54100000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Aug 25 08:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.67400000000000004,"15min":0.60999999999999999},"min":{"1min":0.53400000000000003,"5min":0.433,"15min":0.433},"max":{"1min":4.1050000000000004,"5min":4.1050000000000004,"15min":4.1050000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.64200000000000002,"15min":0.59799999999999998},"min":{"1min":0.52100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.497}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.52900000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.626},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Aug 25 08:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65300000000000002,"15min":0.629},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.68300000000000005,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.6180000000000003,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Aug 25 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.56799999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.97899999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98399999999999999,"5min":1.161,"15min":1.161},"last":0.65900000000000003}]},{"osd":8,"last update":"Fri Aug 25 08:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85199999999999998}]},{"osd":9,"last update":"Fri Aug 25 08:58:00 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.88100000000000001,"15min":0.85999999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.3730000000000002,"5min":4.3730000000000002,"15min":4.3730000000000002},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.83499999999999996,"15min":0.82099999999999995},"min":{"1min":0.66800000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":0.73699999999999999}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.752}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":444028,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.438}]},{"osd":1,"last update":"Fri Aug 25 08:58:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.40400000000000003},"min":{"1min":0.316,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57499999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.42199999999999999,"15min":0.40899999999999997},"min":{"1min":0.34300000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63600000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.442}]},{"osd":2,"last update":"Fri Aug 25 08:58:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.61799999999999999,"15min":0.58999999999999997},"min":{"1min":0.58499999999999996,"5min":0.376,"15min":0.376},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.754}]},{"osd":3,"last update":"Fri Aug 25 08:58:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.48499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60099999999999998,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Aug 25 08:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.65900000000000003},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.90700000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Aug 25 08:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59199999999999997,"15min":0.55200000000000005},"min":{"1min":0.496,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.91200000000000003,"5min":1.046,"15min":1.046},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58399999999999996,"15min":0.53800000000000003},"min":{"1min":0.5,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92200000000000004,"5min":1.03,"15min":1.03},"last":0.56699999999999995}]},{"osd":8,"last update":"Fri Aug 25 08:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.83799999999999997}]},{"osd":9,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":1.089,"15min":1.089},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66900000000000004}]},{"osd":10,"last update":"Fri Aug 25 08:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1035410,"data_compressed":42060,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45300000000000001,"15min":0.47099999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67900000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.46700000000000003,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61799999999999999,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.496,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":1.075,"15min":1.075},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.76800000000000002}]},{"osd":8,"last update":"Fri Aug 25 08:57:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68000000000000005}]},{"osd":9,"last update":"Fri Aug 25 08:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.53900000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Aug 25 08:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34629,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34629,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T08:58:15.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph pg dump --format=json 2023-08-25T08:58:15.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:15 smithi099 ceph-mon[97641]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:58:15.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:15 smithi099 ceph-mon[97641]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:15 smithi157 ceph-mon[130879]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:58:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:15 smithi157 ceph-mon[130879]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:15 smithi181 ceph-mon[131296]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:58:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:15 smithi181 ceph-mon[131296]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 -- 172.21.15.99:0/1979236411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18106940 msgr2=0x7f4c18106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/1979236411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18106940 0x7f4c18106d20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009990 tx=0x7f4c08009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 -- 172.21.15.99:0/1979236411 shutdown_connections 2023-08-25T08:58:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/1979236411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c18101300 0x7f4c1810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/1979236411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18106940 0x7f4c18106d20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/1979236411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c18100940 0x7f4c18100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 -- 172.21.15.99:0/1979236411 >> 172.21.15.99:0/1979236411 conn(0x7f4c180fc410 msgr2=0x7f4c180fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 -- 172.21.15.99:0/1979236411 shutdown_connections 2023-08-25T08:58:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.244+0000 7f4c1df9f700 1 -- 172.21.15.99:0/1979236411 wait complete. 2023-08-25T08:58:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.245+0000 7f4c1df9f700 1 Processor -- start 2023-08-25T08:58:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.245+0000 7f4c1df9f700 1 -- start start 2023-08-25T08:58:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c1df9f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18100940 0x7f4c1819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c1df9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c18101300 0x7f4c1819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c1df9f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c18106940 0x7f4c181968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c1df9f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c18196f60 con 0x7f4c18100940 2023-08-25T08:58:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c1df9f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c181970d0 con 0x7f4c18106940 2023-08-25T08:58:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c1df9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c18197290 con 0x7f4c18101300 2023-08-25T08:58:17.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18100940 0x7f4c1819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:17.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c16ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c18101300 0x7f4c1819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:17.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c16ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c18101300 0x7f4c1819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34058/0 (socket says 172.21.15.99:34058) 2023-08-25T08:58:17.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c16ffd700 1 -- 172.21.15.99:0/3746411822 learned_addr learned my addr 172.21.15.99:0/3746411822 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:17.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c17fff700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c18106940 0x7f4c181968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:17.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c177fe700 1 -- 172.21.15.99:0/3746411822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c18101300 msgr2=0x7f4c1819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:17.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c177fe700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c18101300 0x7f4c1819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c177fe700 1 -- 172.21.15.99:0/3746411822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c18106940 msgr2=0x7f4c181968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c177fe700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c18106940 0x7f4c181968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c177fe700 1 -- 172.21.15.99:0/3746411822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080095f0 con 0x7f4c18100940 2023-08-25T08:58:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.246+0000 7f4c16ffd700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c18101300 0x7f4c1819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.247+0000 7f4c177fe700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18100940 0x7f4c1819c870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c08000c00 tx=0x7f4c0801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.247+0000 7f4c14ff9700 1 -- 172.21.15.99:0/3746411822 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08021070 con 0x7f4c18100940 2023-08-25T08:58:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.247+0000 7f4c14ff9700 1 -- 172.21.15.99:0/3746411822 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0801ce40 con 0x7f4c18100940 2023-08-25T08:58:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.247+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c18197510 con 0x7f4c18100940 2023-08-25T08:58:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.247+0000 7f4c14ff9700 1 -- 172.21.15.99:0/3746411822 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08004de0 con 0x7f4c18100940 2023-08-25T08:58:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.247+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c18072d20 con 0x7f4c18100940 2023-08-25T08:58:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.248+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1804e620 con 0x7f4c18100940 2023-08-25T08:58:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.250+0000 7f4c14ff9700 1 -- 172.21.15.99:0/3746411822 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f4c08005050 con 0x7f4c18100940 2023-08-25T08:58:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.251+0000 7f4c14ff9700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c0009ffe0 0x7f4c000a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.252+0000 7f4c14ff9700 1 -- 172.21.15.99:0/3746411822 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c080d0a20 con 0x7f4c18100940 2023-08-25T08:58:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.252+0000 7f4c16ffd700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c0009ffe0 0x7f4c000a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.252+0000 7f4c16ffd700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c0009ffe0 0x7f4c000a24a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c180fed70 tx=0x7f4c0c007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.255+0000 7f4c14ff9700 1 -- 172.21.15.99:0/3746411822 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4c0809a660 con 0x7f4c18100940 2023-08-25T08:58:17.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.451+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c18110990 con 0x7f4c0009ffe0 2023-08-25T08:58:17.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.461+0000 7f4c14ff9700 1 -- 172.21.15.99:0/3746411822 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340530 (secure 0 0 0) 0x7f4c18110990 con 0x7f4c0009ffe0 2023-08-25T08:58:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:17.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c0009ffe0 msgr2=0x7f4c000a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:17.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c0009ffe0 0x7f4c000a24a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c180fed70 tx=0x7f4c0c007480 comp rx=0 tx=0).stop 2023-08-25T08:58:17.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18100940 msgr2=0x7f4c1819c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:17.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18100940 0x7f4c1819c870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c08000c00 tx=0x7f4c0801ca40 comp rx=0 tx=0).stop 2023-08-25T08:58:17.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 shutdown_connections 2023-08-25T08:58:17.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c0009ffe0 0x7f4c000a24a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c18106940 0x7f4c181968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c18100940 0x7f4c1819c870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 --2- 172.21.15.99:0/3746411822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c18101300 0x7f4c1819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:17.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 >> 172.21.15.99:0/3746411822 conn(0x7f4c180fc410 msgr2=0x7f4c180fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:17.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 shutdown_connections 2023-08-25T08:58:17.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:17.466+0000 7f4c1df9f700 1 -- 172.21.15.99:0/3746411822 wait complete. 2023-08-25T08:58:17.471 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-08-25T08:58:17.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:17 smithi099 ceph-mon[97641]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:17.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:17 smithi157 ceph-mon[130879]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:18.031 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:17 smithi181 ceph-mon[131296]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:18.032 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-08-25T08:58:16.449644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651608,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":464,"num_read_kb":349,"num_write":331,"num_write_kb":3041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335516,"kb_used_data":11260,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606308,"statfs":{"total":1151940427776,"available":1151596859392,"internally_reserved":0,"allocated":11530240,"data_stored":7274828,"data_compressed":178299,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002649"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052994+0000","last_change":"2023-08-25T08:57:19.519342+0000","last_active":"2023-08-25T08:57:56.052994+0000","last_peered":"2023-08-25T08:57:56.052994+0000","last_clean":"2023-08-25T08:57:56.052994+0000","last_became_active":"2023-08-25T08:57:19.519006+0000","last_became_peered":"2023-08-25T08:57:19.519006+0000","last_unstale":"2023-08-25T08:57:56.052994+0000","last_undegraded":"2023-08-25T08:57:56.052994+0000","last_fullsized":"2023-08-25T08:57:56.052994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:50.111134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369781+0000","last_change":"2023-08-25T08:57:19.521779+0000","last_active":"2023-08-25T08:57:56.369781+0000","last_peered":"2023-08-25T08:57:56.369781+0000","last_clean":"2023-08-25T08:57:56.369781+0000","last_became_active":"2023-08-25T08:57:19.521358+0000","last_became_peered":"2023-08-25T08:57:19.521358+0000","last_unstale":"2023-08-25T08:57:56.369781+0000","last_undegraded":"2023-08-25T08:57:56.369781+0000","last_fullsized":"2023-08-25T08:57:56.369781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:14:25.404973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370128+0000","last_change":"2023-08-25T08:57:19.531553+0000","last_active":"2023-08-25T08:57:56.370128+0000","last_peered":"2023-08-25T08:57:56.370128+0000","last_clean":"2023-08-25T08:57:56.370128+0000","last_became_active":"2023-08-25T08:57:19.531113+0000","last_became_peered":"2023-08-25T08:57:19.531113+0000","last_unstale":"2023-08-25T08:57:56.370128+0000","last_undegraded":"2023-08-25T08:57:56.370128+0000","last_fullsized":"2023-08-25T08:57:56.370128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:28:58.239330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.094847+0000","last_change":"2023-08-25T08:57:19.519696+0000","last_active":"2023-08-25T08:57:56.094847+0000","last_peered":"2023-08-25T08:57:56.094847+0000","last_clean":"2023-08-25T08:57:56.094847+0000","last_became_active":"2023-08-25T08:57:19.518918+0000","last_became_peered":"2023-08-25T08:57:19.518918+0000","last_unstale":"2023-08-25T08:57:56.094847+0000","last_undegraded":"2023-08-25T08:57:56.094847+0000","last_fullsized":"2023-08-25T08:57:56.094847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:39:25.972315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055842+0000","last_change":"2023-08-25T08:57:19.520706+0000","last_active":"2023-08-25T08:57:56.055842+0000","last_peered":"2023-08-25T08:57:56.055842+0000","last_clean":"2023-08-25T08:57:56.055842+0000","last_became_active":"2023-08-25T08:57:19.520333+0000","last_became_peered":"2023-08-25T08:57:19.520333+0000","last_unstale":"2023-08-25T08:57:56.055842+0000","last_undegraded":"2023-08-25T08:57:56.055842+0000","last_fullsized":"2023-08-25T08:57:56.055842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:54.763281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057288+0000","last_change":"2023-08-25T08:57:56.057288+0000","last_active":"2023-08-25T08:57:56.057288+0000","last_peered":"2023-08-25T08:57:56.057288+0000","last_clean":"2023-08-25T08:57:56.057288+0000","last_became_active":"2023-08-25T08:57:56.056839+0000","last_became_peered":"2023-08-25T08:57:56.056839+0000","last_unstale":"2023-08-25T08:57:56.057288+0000","last_undegraded":"2023-08-25T08:57:56.057288+0000","last_fullsized":"2023-08-25T08:57:56.057288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:10:06.945840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057789+0000","last_change":"2023-08-25T08:57:56.057789+0000","last_active":"2023-08-25T08:57:56.057789+0000","last_peered":"2023-08-25T08:57:56.057789+0000","last_clean":"2023-08-25T08:57:56.057789+0000","last_became_active":"2023-08-25T08:57:56.057382+0000","last_became_peered":"2023-08-25T08:57:56.057382+0000","last_unstale":"2023-08-25T08:57:56.057789+0000","last_undegraded":"2023-08-25T08:57:56.057789+0000","last_fullsized":"2023-08-25T08:57:56.057789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:28:35.484376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369844+0000","last_change":"2023-08-25T08:57:19.527429+0000","last_active":"2023-08-25T08:57:56.369844+0000","last_peered":"2023-08-25T08:57:56.369844+0000","last_clean":"2023-08-25T08:57:56.369844+0000","last_became_active":"2023-08-25T08:57:19.526370+0000","last_became_peered":"2023-08-25T08:57:19.526370+0000","last_unstale":"2023-08-25T08:57:56.369844+0000","last_undegraded":"2023-08-25T08:57:56.369844+0000","last_fullsized":"2023-08-25T08:57:56.369844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:23:42.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056522+0000","last_change":"2023-08-25T08:57:19.517136+0000","last_active":"2023-08-25T08:57:56.056522+0000","last_peered":"2023-08-25T08:57:56.056522+0000","last_clean":"2023-08-25T08:57:56.056522+0000","last_became_active":"2023-08-25T08:57:19.516852+0000","last_became_peered":"2023-08-25T08:57:19.516852+0000","last_unstale":"2023-08-25T08:57:56.056522+0000","last_undegraded":"2023-08-25T08:57:56.056522+0000","last_fullsized":"2023-08-25T08:57:56.056522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:43.370462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048485+0000","last_change":"2023-08-25T08:57:19.522122+0000","last_active":"2023-08-25T08:57:56.048485+0000","last_peered":"2023-08-25T08:57:56.048485+0000","last_clean":"2023-08-25T08:57:56.048485+0000","last_became_active":"2023-08-25T08:57:19.520984+0000","last_became_peered":"2023-08-25T08:57:19.520984+0000","last_unstale":"2023-08-25T08:57:56.048485+0000","last_undegraded":"2023-08-25T08:57:56.048485+0000","last_fullsized":"2023-08-25T08:57:56.048485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:23:36.989284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053380+0000","last_change":"2023-08-25T08:57:19.531356+0000","last_active":"2023-08-25T08:57:56.053380+0000","last_peered":"2023-08-25T08:57:56.053380+0000","last_clean":"2023-08-25T08:57:56.053380+0000","last_became_active":"2023-08-25T08:57:19.531050+0000","last_became_peered":"2023-08-25T08:57:19.531050+0000","last_unstale":"2023-08-25T08:57:56.053380+0000","last_undegraded":"2023-08-25T08:57:56.053380+0000","last_fullsized":"2023-08-25T08:57:56.053380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:11.201931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.094817+0000","last_change":"2023-08-25T08:57:19.520091+0000","last_active":"2023-08-25T08:57:56.094817+0000","last_peered":"2023-08-25T08:57:56.094817+0000","last_clean":"2023-08-25T08:57:56.094817+0000","last_became_active":"2023-08-25T08:57:19.519356+0000","last_became_peered":"2023-08-25T08:57:19.519356+0000","last_unstale":"2023-08-25T08:57:56.094817+0000","last_undegraded":"2023-08-25T08:57:56.094817+0000","last_fullsized":"2023-08-25T08:57:56.094817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:23.224068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049755+0000","last_change":"2023-08-25T08:57:19.515446+0000","last_active":"2023-08-25T08:57:56.049755+0000","last_peered":"2023-08-25T08:57:56.049755+0000","last_clean":"2023-08-25T08:57:56.049755+0000","last_became_active":"2023-08-25T08:57:19.514376+0000","last_became_peered":"2023-08-25T08:57:19.514376+0000","last_unstale":"2023-08-25T08:57:56.049755+0000","last_undegraded":"2023-08-25T08:57:56.049755+0000","last_fullsized":"2023-08-25T08:57:56.049755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:21:52.606389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369501+0000","last_change":"2023-08-25T08:57:19.521629+0000","last_active":"2023-08-25T08:57:56.369501+0000","last_peered":"2023-08-25T08:57:56.369501+0000","last_clean":"2023-08-25T08:57:56.369501+0000","last_became_active":"2023-08-25T08:57:19.521307+0000","last_became_peered":"2023-08-25T08:57:19.521307+0000","last_unstale":"2023-08-25T08:57:56.369501+0000","last_undegraded":"2023-08-25T08:57:56.369501+0000","last_fullsized":"2023-08-25T08:57:56.369501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:03.613694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369389+0000","last_change":"2023-08-25T08:57:19.521670+0000","last_active":"2023-08-25T08:57:56.369389+0000","last_peered":"2023-08-25T08:57:56.369389+0000","last_clean":"2023-08-25T08:57:56.369389+0000","last_became_active":"2023-08-25T08:57:19.521325+0000","last_became_peered":"2023-08-25T08:57:19.521325+0000","last_unstale":"2023-08-25T08:57:56.369389+0000","last_undegraded":"2023-08-25T08:57:56.369389+0000","last_fullsized":"2023-08-25T08:57:56.369389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:53.426927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055725+0000","last_change":"2023-08-25T08:57:19.526733+0000","last_active":"2023-08-25T08:57:56.055725+0000","last_peered":"2023-08-25T08:57:56.055725+0000","last_clean":"2023-08-25T08:57:56.055725+0000","last_became_active":"2023-08-25T08:57:19.525692+0000","last_became_peered":"2023-08-25T08:57:19.525692+0000","last_unstale":"2023-08-25T08:57:56.055725+0000","last_undegraded":"2023-08-25T08:57:56.055725+0000","last_fullsized":"2023-08-25T08:57:56.055725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:10:16.975270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053155+0000","last_change":"2023-08-25T08:57:19.519289+0000","last_active":"2023-08-25T08:57:56.053155+0000","last_peered":"2023-08-25T08:57:56.053155+0000","last_clean":"2023-08-25T08:57:56.053155+0000","last_became_active":"2023-08-25T08:57:19.518584+0000","last_became_peered":"2023-08-25T08:57:19.518584+0000","last_unstale":"2023-08-25T08:57:56.053155+0000","last_undegraded":"2023-08-25T08:57:56.053155+0000","last_fullsized":"2023-08-25T08:57:56.053155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:00:48.505383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052281+0000","last_change":"2023-08-25T08:57:19.525090+0000","last_active":"2023-08-25T08:57:56.052281+0000","last_peered":"2023-08-25T08:57:56.052281+0000","last_clean":"2023-08-25T08:57:56.052281+0000","last_became_active":"2023-08-25T08:57:19.524543+0000","last_became_peered":"2023-08-25T08:57:19.524543+0000","last_unstale":"2023-08-25T08:57:56.052281+0000","last_undegraded":"2023-08-25T08:57:56.052281+0000","last_fullsized":"2023-08-25T08:57:56.052281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:28:37.398394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049079+0000","last_change":"2023-08-25T08:57:19.526761+0000","last_active":"2023-08-25T08:57:56.049079+0000","last_peered":"2023-08-25T08:57:56.049079+0000","last_clean":"2023-08-25T08:57:56.049079+0000","last_became_active":"2023-08-25T08:57:19.526421+0000","last_became_peered":"2023-08-25T08:57:19.526421+0000","last_unstale":"2023-08-25T08:57:56.049079+0000","last_undegraded":"2023-08-25T08:57:56.049079+0000","last_fullsized":"2023-08-25T08:57:56.049079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:04.396487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052308+0000","last_change":"2023-08-25T08:57:18.717584+0000","last_active":"2023-08-25T08:57:56.052308+0000","last_peered":"2023-08-25T08:57:56.052308+0000","last_clean":"2023-08-25T08:57:56.052308+0000","last_became_active":"2023-08-25T08:57:18.717284+0000","last_became_peered":"2023-08-25T08:57:18.717284+0000","last_unstale":"2023-08-25T08:57:56.052308+0000","last_undegraded":"2023-08-25T08:57:56.052308+0000","last_fullsized":"2023-08-25T08:57:56.052308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:56:39.309635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370697+0000","last_change":"2023-08-25T08:57:19.527089+0000","last_active":"2023-08-25T08:57:56.370697+0000","last_peered":"2023-08-25T08:57:56.370697+0000","last_clean":"2023-08-25T08:57:56.370697+0000","last_became_active":"2023-08-25T08:57:19.526118+0000","last_became_peered":"2023-08-25T08:57:19.526118+0000","last_unstale":"2023-08-25T08:57:56.370697+0000","last_undegraded":"2023-08-25T08:57:56.370697+0000","last_fullsized":"2023-08-25T08:57:56.370697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:01.852226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370821+0000","last_change":"2023-08-25T08:57:18.720269+0000","last_active":"2023-08-25T08:57:56.370821+0000","last_peered":"2023-08-25T08:57:56.370821+0000","last_clean":"2023-08-25T08:57:56.370821+0000","last_became_active":"2023-08-25T08:57:18.719878+0000","last_became_peered":"2023-08-25T08:57:18.719878+0000","last_unstale":"2023-08-25T08:57:56.370821+0000","last_undegraded":"2023-08-25T08:57:56.370821+0000","last_fullsized":"2023-08-25T08:57:56.370821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:01:44.356022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370249+0000","last_change":"2023-08-25T08:57:19.531835+0000","last_active":"2023-08-25T08:57:56.370249+0000","last_peered":"2023-08-25T08:57:56.370249+0000","last_clean":"2023-08-25T08:57:56.370249+0000","last_became_active":"2023-08-25T08:57:19.531409+0000","last_became_peered":"2023-08-25T08:57:19.531409+0000","last_unstale":"2023-08-25T08:57:56.370249+0000","last_undegraded":"2023-08-25T08:57:56.370249+0000","last_fullsized":"2023-08-25T08:57:56.370249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:49:14.438548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.099661+0000","last_change":"2023-08-25T08:57:56.099661+0000","last_active":"2023-08-25T08:57:56.099661+0000","last_peered":"2023-08-25T08:57:56.099661+0000","last_clean":"2023-08-25T08:57:56.099661+0000","last_became_active":"2023-08-25T08:57:56.095753+0000","last_became_peered":"2023-08-25T08:57:56.095753+0000","last_unstale":"2023-08-25T08:57:56.099661+0000","last_undegraded":"2023-08-25T08:57:56.099661+0000","last_fullsized":"2023-08-25T08:57:56.099661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:38:49.238858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056179+0000","last_change":"2023-08-25T08:57:19.526495+0000","last_active":"2023-08-25T08:57:56.056179+0000","last_peered":"2023-08-25T08:57:56.056179+0000","last_clean":"2023-08-25T08:57:56.056179+0000","last_became_active":"2023-08-25T08:57:19.525204+0000","last_became_peered":"2023-08-25T08:57:19.525204+0000","last_unstale":"2023-08-25T08:57:56.056179+0000","last_undegraded":"2023-08-25T08:57:56.056179+0000","last_fullsized":"2023-08-25T08:57:56.056179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:27:16.103151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.062013+0000","last_change":"2023-08-25T08:57:56.061586+0000","last_active":"2023-08-25T08:57:56.062013+0000","last_peered":"2023-08-25T08:57:56.062013+0000","last_clean":"2023-08-25T08:57:56.062013+0000","last_became_active":"2023-08-25T08:57:56.057255+0000","last_became_peered":"2023-08-25T08:57:56.057255+0000","last_unstale":"2023-08-25T08:57:56.062013+0000","last_undegraded":"2023-08-25T08:57:56.062013+0000","last_fullsized":"2023-08-25T08:57:56.062013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:37:36.084255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057937+0000","last_change":"2023-08-25T08:57:56.057937+0000","last_active":"2023-08-25T08:57:56.057937+0000","last_peered":"2023-08-25T08:57:56.057937+0000","last_clean":"2023-08-25T08:57:56.057937+0000","last_became_active":"2023-08-25T08:57:56.057578+0000","last_became_peered":"2023-08-25T08:57:56.057578+0000","last_unstale":"2023-08-25T08:57:56.057937+0000","last_undegraded":"2023-08-25T08:57:56.057937+0000","last_fullsized":"2023-08-25T08:57:56.057937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:07:21.225532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:11.100474+0000","last_change":"2023-08-25T08:57:18.715563+0000","last_active":"2023-08-25T08:58:11.100474+0000","last_peered":"2023-08-25T08:58:11.100474+0000","last_clean":"2023-08-25T08:58:11.100474+0000","last_became_active":"2023-08-25T08:57:18.715269+0000","last_became_peered":"2023-08-25T08:57:18.715269+0000","last_unstale":"2023-08-25T08:58:11.100474+0000","last_undegraded":"2023-08-25T08:58:11.100474+0000","last_fullsized":"2023-08-25T08:58:11.100474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:24:34.134451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055868+0000","last_change":"2023-08-25T08:57:19.518705+0000","last_active":"2023-08-25T08:57:56.055868+0000","last_peered":"2023-08-25T08:57:56.055868+0000","last_clean":"2023-08-25T08:57:56.055868+0000","last_became_active":"2023-08-25T08:57:19.518422+0000","last_became_peered":"2023-08-25T08:57:19.518422+0000","last_unstale":"2023-08-25T08:57:56.055868+0000","last_undegraded":"2023-08-25T08:57:56.055868+0000","last_fullsized":"2023-08-25T08:57:56.055868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:16:21.057980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.047907+0000","last_change":"2023-08-25T08:57:18.714087+0000","last_active":"2023-08-25T08:57:56.047907+0000","last_peered":"2023-08-25T08:57:56.047907+0000","last_clean":"2023-08-25T08:57:56.047907+0000","last_became_active":"2023-08-25T08:57:18.713777+0000","last_became_peered":"2023-08-25T08:57:18.713777+0000","last_unstale":"2023-08-25T08:57:56.047907+0000","last_undegraded":"2023-08-25T08:57:56.047907+0000","last_fullsized":"2023-08-25T08:57:56.047907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:35.995427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051603+0000","last_change":"2023-08-25T08:57:19.518068+0000","last_active":"2023-08-25T08:57:56.051603+0000","last_peered":"2023-08-25T08:57:56.051603+0000","last_clean":"2023-08-25T08:57:56.051603+0000","last_became_active":"2023-08-25T08:57:19.517752+0000","last_became_peered":"2023-08-25T08:57:19.517752+0000","last_unstale":"2023-08-25T08:57:56.051603+0000","last_undegraded":"2023-08-25T08:57:56.051603+0000","last_fullsized":"2023-08-25T08:57:56.051603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:20.053896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049546+0000","last_change":"2023-08-25T08:57:18.713652+0000","last_active":"2023-08-25T08:57:56.049546+0000","last_peered":"2023-08-25T08:57:56.049546+0000","last_clean":"2023-08-25T08:57:56.049546+0000","last_became_active":"2023-08-25T08:57:18.713348+0000","last_became_peered":"2023-08-25T08:57:18.713348+0000","last_unstale":"2023-08-25T08:57:56.049546+0000","last_undegraded":"2023-08-25T08:57:56.049546+0000","last_fullsized":"2023-08-25T08:57:56.049546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:32.759661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057655+0000","last_change":"2023-08-25T08:57:56.057655+0000","last_active":"2023-08-25T08:57:56.057655+0000","last_peered":"2023-08-25T08:57:56.057655+0000","last_clean":"2023-08-25T08:57:56.057655+0000","last_became_active":"2023-08-25T08:57:56.057272+0000","last_became_peered":"2023-08-25T08:57:56.057272+0000","last_unstale":"2023-08-25T08:57:56.057655+0000","last_undegraded":"2023-08-25T08:57:56.057655+0000","last_fullsized":"2023-08-25T08:57:56.057655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:46.525543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051289+0000","last_change":"2023-08-25T08:57:18.713735+0000","last_active":"2023-08-25T08:57:56.051289+0000","last_peered":"2023-08-25T08:57:56.051289+0000","last_clean":"2023-08-25T08:57:56.051289+0000","last_became_active":"2023-08-25T08:57:18.713421+0000","last_became_peered":"2023-08-25T08:57:18.713421+0000","last_unstale":"2023-08-25T08:57:56.051289+0000","last_undegraded":"2023-08-25T08:57:56.051289+0000","last_fullsized":"2023-08-25T08:57:56.051289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:51.283967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052756+0000","last_change":"2023-08-25T08:57:19.513137+0000","last_active":"2023-08-25T08:57:56.052756+0000","last_peered":"2023-08-25T08:57:56.052756+0000","last_clean":"2023-08-25T08:57:56.052756+0000","last_became_active":"2023-08-25T08:57:19.512804+0000","last_became_peered":"2023-08-25T08:57:19.512804+0000","last_unstale":"2023-08-25T08:57:56.052756+0000","last_undegraded":"2023-08-25T08:57:56.052756+0000","last_fullsized":"2023-08-25T08:57:56.052756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:56.094424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:14.104389+0000","last_change":"2023-08-25T08:57:18.712855+0000","last_active":"2023-08-25T08:58:14.104389+0000","last_peered":"2023-08-25T08:58:14.104389+0000","last_clean":"2023-08-25T08:58:14.104389+0000","last_became_active":"2023-08-25T08:57:18.712327+0000","last_became_peered":"2023-08-25T08:57:18.712327+0000","last_unstale":"2023-08-25T08:58:14.104389+0000","last_undegraded":"2023-08-25T08:58:14.104389+0000","last_fullsized":"2023-08-25T08:58:14.104389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:54.583482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056659+0000","last_change":"2023-08-25T08:57:19.526933+0000","last_active":"2023-08-25T08:57:56.056659+0000","last_peered":"2023-08-25T08:57:56.056659+0000","last_clean":"2023-08-25T08:57:56.056659+0000","last_became_active":"2023-08-25T08:57:19.524798+0000","last_became_peered":"2023-08-25T08:57:19.524798+0000","last_unstale":"2023-08-25T08:57:56.056659+0000","last_undegraded":"2023-08-25T08:57:56.056659+0000","last_fullsized":"2023-08-25T08:57:56.056659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:42:52.044387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053324+0000","last_change":"2023-08-25T08:57:18.714182+0000","last_active":"2023-08-25T08:57:56.053324+0000","last_peered":"2023-08-25T08:57:56.053324+0000","last_clean":"2023-08-25T08:57:56.053324+0000","last_became_active":"2023-08-25T08:57:18.713872+0000","last_became_peered":"2023-08-25T08:57:18.713872+0000","last_unstale":"2023-08-25T08:57:56.053324+0000","last_undegraded":"2023-08-25T08:57:56.053324+0000","last_fullsized":"2023-08-25T08:57:56.053324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:54.231840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370131+0000","last_change":"2023-08-25T08:57:19.530745+0000","last_active":"2023-08-25T08:57:56.370131+0000","last_peered":"2023-08-25T08:57:56.370131+0000","last_clean":"2023-08-25T08:57:56.370131+0000","last_became_active":"2023-08-25T08:57:19.530397+0000","last_became_peered":"2023-08-25T08:57:19.530397+0000","last_unstale":"2023-08-25T08:57:56.370131+0000","last_undegraded":"2023-08-25T08:57:56.370131+0000","last_fullsized":"2023-08-25T08:57:56.370131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:34:56.224834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052690+0000","last_change":"2023-08-25T08:57:18.716995+0000","last_active":"2023-08-25T08:57:56.052690+0000","last_peered":"2023-08-25T08:57:56.052690+0000","last_clean":"2023-08-25T08:57:56.052690+0000","last_became_active":"2023-08-25T08:57:18.716672+0000","last_became_peered":"2023-08-25T08:57:18.716672+0000","last_unstale":"2023-08-25T08:57:56.052690+0000","last_undegraded":"2023-08-25T08:57:56.052690+0000","last_fullsized":"2023-08-25T08:57:56.052690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:37:49.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056350+0000","last_change":"2023-08-25T08:57:19.526608+0000","last_active":"2023-08-25T08:57:56.056350+0000","last_peered":"2023-08-25T08:57:56.056350+0000","last_clean":"2023-08-25T08:57:56.056350+0000","last_became_active":"2023-08-25T08:57:19.525614+0000","last_became_peered":"2023-08-25T08:57:19.525614+0000","last_unstale":"2023-08-25T08:57:56.056350+0000","last_undegraded":"2023-08-25T08:57:56.056350+0000","last_fullsized":"2023-08-25T08:57:56.056350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:42.895003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":155,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:15.105737+0000","last_change":"2023-08-25T08:57:18.712588+0000","last_active":"2023-08-25T08:58:15.105737+0000","last_peered":"2023-08-25T08:58:15.105737+0000","last_clean":"2023-08-25T08:58:15.105737+0000","last_became_active":"2023-08-25T08:57:18.712259+0000","last_became_peered":"2023-08-25T08:57:18.712259+0000","last_unstale":"2023-08-25T08:58:15.105737+0000","last_undegraded":"2023-08-25T08:58:15.105737+0000","last_fullsized":"2023-08-25T08:58:15.105737+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:24.881049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56994,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":110,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056324+0000","last_change":"2023-08-25T08:57:19.517907+0000","last_active":"2023-08-25T08:57:56.056324+0000","last_peered":"2023-08-25T08:57:56.056324+0000","last_clean":"2023-08-25T08:57:56.056324+0000","last_became_active":"2023-08-25T08:57:19.517596+0000","last_became_peered":"2023-08-25T08:57:19.517596+0000","last_unstale":"2023-08-25T08:57:56.056324+0000","last_undegraded":"2023-08-25T08:57:56.056324+0000","last_fullsized":"2023-08-25T08:57:56.056324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:20.224373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049213+0000","last_change":"2023-08-25T08:57:18.721564+0000","last_active":"2023-08-25T08:57:56.049213+0000","last_peered":"2023-08-25T08:57:56.049213+0000","last_clean":"2023-08-25T08:57:56.049213+0000","last_became_active":"2023-08-25T08:57:18.721235+0000","last_became_peered":"2023-08-25T08:57:18.721235+0000","last_unstale":"2023-08-25T08:57:56.049213+0000","last_undegraded":"2023-08-25T08:57:56.049213+0000","last_fullsized":"2023-08-25T08:57:56.049213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:57:00.047065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049661+0000","last_change":"2023-08-25T08:57:19.518399+0000","last_active":"2023-08-25T08:57:56.049661+0000","last_peered":"2023-08-25T08:57:56.049661+0000","last_clean":"2023-08-25T08:57:56.049661+0000","last_became_active":"2023-08-25T08:57:19.518061+0000","last_became_peered":"2023-08-25T08:57:19.518061+0000","last_unstale":"2023-08-25T08:57:56.049661+0000","last_undegraded":"2023-08-25T08:57:56.049661+0000","last_fullsized":"2023-08-25T08:57:56.049661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:11.217763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051741+0000","last_change":"2023-08-25T08:57:18.722020+0000","last_active":"2023-08-25T08:57:56.051741+0000","last_peered":"2023-08-25T08:57:56.051741+0000","last_clean":"2023-08-25T08:57:56.051741+0000","last_became_active":"2023-08-25T08:57:18.721707+0000","last_became_peered":"2023-08-25T08:57:18.721707+0000","last_unstale":"2023-08-25T08:57:56.051741+0000","last_undegraded":"2023-08-25T08:57:56.051741+0000","last_fullsized":"2023-08-25T08:57:56.051741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:47:40.076757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053276+0000","last_change":"2023-08-25T08:57:19.513954+0000","last_active":"2023-08-25T08:57:56.053276+0000","last_peered":"2023-08-25T08:57:56.053276+0000","last_clean":"2023-08-25T08:57:56.053276+0000","last_became_active":"2023-08-25T08:57:19.513645+0000","last_became_peered":"2023-08-25T08:57:19.513645+0000","last_unstale":"2023-08-25T08:57:56.053276+0000","last_undegraded":"2023-08-25T08:57:56.053276+0000","last_fullsized":"2023-08-25T08:57:56.053276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:24:16.072144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:11.117214+0000","last_change":"2023-08-25T08:57:18.711499+0000","last_active":"2023-08-25T08:58:11.117214+0000","last_peered":"2023-08-25T08:58:11.117214+0000","last_clean":"2023-08-25T08:58:11.117214+0000","last_became_active":"2023-08-25T08:57:18.711223+0000","last_became_peered":"2023-08-25T08:57:18.711223+0000","last_unstale":"2023-08-25T08:58:11.117214+0000","last_undegraded":"2023-08-25T08:58:11.117214+0000","last_fullsized":"2023-08-25T08:58:11.117214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:46.247137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052602+0000","last_change":"2023-08-25T08:57:19.518663+0000","last_active":"2023-08-25T08:57:56.052602+0000","last_peered":"2023-08-25T08:57:56.052602+0000","last_clean":"2023-08-25T08:57:56.052602+0000","last_became_active":"2023-08-25T08:57:19.518394+0000","last_became_peered":"2023-08-25T08:57:19.518394+0000","last_unstale":"2023-08-25T08:57:56.052602+0000","last_undegraded":"2023-08-25T08:57:56.052602+0000","last_fullsized":"2023-08-25T08:57:56.052602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:54:23.798982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055719+0000","last_change":"2023-08-25T08:57:18.718554+0000","last_active":"2023-08-25T08:57:56.055719+0000","last_peered":"2023-08-25T08:57:56.055719+0000","last_clean":"2023-08-25T08:57:56.055719+0000","last_became_active":"2023-08-25T08:57:18.718283+0000","last_became_peered":"2023-08-25T08:57:18.718283+0000","last_unstale":"2023-08-25T08:57:56.055719+0000","last_undegraded":"2023-08-25T08:57:56.055719+0000","last_fullsized":"2023-08-25T08:57:56.055719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:19:51.813157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051440+0000","last_change":"2023-08-25T08:57:19.520856+0000","last_active":"2023-08-25T08:57:56.051440+0000","last_peered":"2023-08-25T08:57:56.051440+0000","last_clean":"2023-08-25T08:57:56.051440+0000","last_became_active":"2023-08-25T08:57:19.520568+0000","last_became_peered":"2023-08-25T08:57:19.520568+0000","last_unstale":"2023-08-25T08:57:56.051440+0000","last_undegraded":"2023-08-25T08:57:56.051440+0000","last_fullsized":"2023-08-25T08:57:56.051440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:12.281170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055834+0000","last_change":"2023-08-25T08:57:18.715972+0000","last_active":"2023-08-25T08:57:56.055834+0000","last_peered":"2023-08-25T08:57:56.055834+0000","last_clean":"2023-08-25T08:57:56.055834+0000","last_became_active":"2023-08-25T08:57:18.715519+0000","last_became_peered":"2023-08-25T08:57:18.715519+0000","last_unstale":"2023-08-25T08:57:56.055834+0000","last_undegraded":"2023-08-25T08:57:56.055834+0000","last_fullsized":"2023-08-25T08:57:56.055834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:59.984267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.057104+0000","last_change":"2023-08-25T08:57:56.057104+0000","last_active":"2023-08-25T08:57:56.057104+0000","last_peered":"2023-08-25T08:57:56.057104+0000","last_clean":"2023-08-25T08:57:56.057104+0000","last_became_active":"2023-08-25T08:57:56.056755+0000","last_became_peered":"2023-08-25T08:57:56.056755+0000","last_unstale":"2023-08-25T08:57:56.057104+0000","last_undegraded":"2023-08-25T08:57:56.057104+0000","last_fullsized":"2023-08-25T08:57:56.057104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:58:33.964262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053523+0000","last_change":"2023-08-25T08:57:18.713835+0000","last_active":"2023-08-25T08:57:56.053523+0000","last_peered":"2023-08-25T08:57:56.053523+0000","last_clean":"2023-08-25T08:57:56.053523+0000","last_became_active":"2023-08-25T08:57:18.713510+0000","last_became_peered":"2023-08-25T08:57:18.713510+0000","last_unstale":"2023-08-25T08:57:56.053523+0000","last_undegraded":"2023-08-25T08:57:56.053523+0000","last_fullsized":"2023-08-25T08:57:56.053523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:50.992010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048188+0000","last_change":"2023-08-25T08:57:19.526979+0000","last_active":"2023-08-25T08:57:56.048188+0000","last_peered":"2023-08-25T08:57:56.048188+0000","last_clean":"2023-08-25T08:57:56.048188+0000","last_became_active":"2023-08-25T08:57:19.526660+0000","last_became_peered":"2023-08-25T08:57:19.526660+0000","last_unstale":"2023-08-25T08:57:56.048188+0000","last_undegraded":"2023-08-25T08:57:56.048188+0000","last_fullsized":"2023-08-25T08:57:56.048188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:45:27.029396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053600+0000","last_change":"2023-08-25T08:57:18.717952+0000","last_active":"2023-08-25T08:57:56.053600+0000","last_peered":"2023-08-25T08:57:56.053600+0000","last_clean":"2023-08-25T08:57:56.053600+0000","last_became_active":"2023-08-25T08:57:18.717643+0000","last_became_peered":"2023-08-25T08:57:18.717643+0000","last_unstale":"2023-08-25T08:57:56.053600+0000","last_undegraded":"2023-08-25T08:57:56.053600+0000","last_fullsized":"2023-08-25T08:57:56.053600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:14:18.972908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048153+0000","last_change":"2023-08-25T08:57:19.521718+0000","last_active":"2023-08-25T08:57:56.048153+0000","last_peered":"2023-08-25T08:57:56.048153+0000","last_clean":"2023-08-25T08:57:56.048153+0000","last_became_active":"2023-08-25T08:57:19.521246+0000","last_became_peered":"2023-08-25T08:57:19.521246+0000","last_unstale":"2023-08-25T08:57:56.048153+0000","last_undegraded":"2023-08-25T08:57:56.048153+0000","last_fullsized":"2023-08-25T08:57:56.048153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:33.050413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055690+0000","last_change":"2023-08-25T08:57:56.055690+0000","last_active":"2023-08-25T08:57:56.055690+0000","last_peered":"2023-08-25T08:57:56.055690+0000","last_clean":"2023-08-25T08:57:56.055690+0000","last_became_active":"2023-08-25T08:57:56.055315+0000","last_became_peered":"2023-08-25T08:57:56.055315+0000","last_unstale":"2023-08-25T08:57:56.055690+0000","last_undegraded":"2023-08-25T08:57:56.055690+0000","last_fullsized":"2023-08-25T08:57:56.055690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:46:47.588215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:58:12.118002+0000","last_change":"2023-08-25T08:57:18.718657+0000","last_active":"2023-08-25T08:58:12.118002+0000","last_peered":"2023-08-25T08:58:12.118002+0000","last_clean":"2023-08-25T08:58:12.118002+0000","last_became_active":"2023-08-25T08:57:18.718348+0000","last_became_peered":"2023-08-25T08:57:18.718348+0000","last_unstale":"2023-08-25T08:58:12.118002+0000","last_undegraded":"2023-08-25T08:58:12.118002+0000","last_fullsized":"2023-08-25T08:58:12.118002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:18:40.166063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370089+0000","last_change":"2023-08-25T08:57:19.520997+0000","last_active":"2023-08-25T08:57:56.370089+0000","last_peered":"2023-08-25T08:57:56.370089+0000","last_clean":"2023-08-25T08:57:56.370089+0000","last_became_active":"2023-08-25T08:57:19.520695+0000","last_became_peered":"2023-08-25T08:57:19.520695+0000","last_unstale":"2023-08-25T08:57:56.370089+0000","last_undegraded":"2023-08-25T08:57:56.370089+0000","last_fullsized":"2023-08-25T08:57:56.370089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:01:18.093454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.094778+0000","last_change":"2023-08-25T08:57:18.714573+0000","last_active":"2023-08-25T08:57:56.094778+0000","last_peered":"2023-08-25T08:57:56.094778+0000","last_clean":"2023-08-25T08:57:56.094778+0000","last_became_active":"2023-08-25T08:57:18.714268+0000","last_became_peered":"2023-08-25T08:57:18.714268+0000","last_unstale":"2023-08-25T08:57:56.094778+0000","last_undegraded":"2023-08-25T08:57:56.094778+0000","last_fullsized":"2023-08-25T08:57:56.094778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:49.345077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053302+0000","last_change":"2023-08-25T08:56:14.362397+0000","last_active":"2023-08-25T08:57:56.053302+0000","last_peered":"2023-08-25T08:57:56.053302+0000","last_clean":"2023-08-25T08:57:56.053302+0000","last_became_active":"2023-08-25T08:56:14.361946+0000","last_became_peered":"2023-08-25T08:56:14.361946+0000","last_unstale":"2023-08-25T08:57:56.053302+0000","last_undegraded":"2023-08-25T08:57:56.053302+0000","last_fullsized":"2023-08-25T08:57:56.053302+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:53:09.022543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:53:09.022543+0000","last_clean_scrub_stamp":"2023-08-25T08:53:09.022543+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:07:59.847147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049410+0000","last_change":"2023-08-25T08:57:19.517547+0000","last_active":"2023-08-25T08:57:56.049410+0000","last_peered":"2023-08-25T08:57:56.049410+0000","last_clean":"2023-08-25T08:57:56.049410+0000","last_became_active":"2023-08-25T08:57:19.517221+0000","last_became_peered":"2023-08-25T08:57:19.517221+0000","last_unstale":"2023-08-25T08:57:56.049410+0000","last_undegraded":"2023-08-25T08:57:56.049410+0000","last_fullsized":"2023-08-25T08:57:56.049410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:34:19.765696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052875+0000","last_change":"2023-08-25T08:57:18.715769+0000","last_active":"2023-08-25T08:57:56.052875+0000","last_peered":"2023-08-25T08:57:56.052875+0000","last_clean":"2023-08-25T08:57:56.052875+0000","last_became_active":"2023-08-25T08:57:18.715399+0000","last_became_peered":"2023-08-25T08:57:18.715399+0000","last_unstale":"2023-08-25T08:57:56.052875+0000","last_undegraded":"2023-08-25T08:57:56.052875+0000","last_fullsized":"2023-08-25T08:57:56.052875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:52:20.906518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056347+0000","last_change":"2023-08-25T08:57:18.720658+0000","last_active":"2023-08-25T08:57:56.056347+0000","last_peered":"2023-08-25T08:57:56.056347+0000","last_clean":"2023-08-25T08:57:56.056347+0000","last_became_active":"2023-08-25T08:57:18.720419+0000","last_became_peered":"2023-08-25T08:57:18.720419+0000","last_unstale":"2023-08-25T08:57:56.056347+0000","last_undegraded":"2023-08-25T08:57:56.056347+0000","last_fullsized":"2023-08-25T08:57:56.056347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:00:29.486469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049985+0000","last_change":"2023-08-25T08:57:19.513257+0000","last_active":"2023-08-25T08:57:56.049985+0000","last_peered":"2023-08-25T08:57:56.049985+0000","last_clean":"2023-08-25T08:57:56.049985+0000","last_became_active":"2023-08-25T08:57:19.512852+0000","last_became_peered":"2023-08-25T08:57:19.512852+0000","last_unstale":"2023-08-25T08:57:56.049985+0000","last_undegraded":"2023-08-25T08:57:56.049985+0000","last_fullsized":"2023-08-25T08:57:56.049985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:15.512558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052981+0000","last_change":"2023-08-25T08:57:18.718740+0000","last_active":"2023-08-25T08:57:56.052981+0000","last_peered":"2023-08-25T08:57:56.052981+0000","last_clean":"2023-08-25T08:57:56.052981+0000","last_became_active":"2023-08-25T08:57:18.718433+0000","last_became_peered":"2023-08-25T08:57:18.718433+0000","last_unstale":"2023-08-25T08:57:56.052981+0000","last_undegraded":"2023-08-25T08:57:56.052981+0000","last_fullsized":"2023-08-25T08:57:56.052981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:18:39.835570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370989+0000","last_change":"2023-08-25T08:57:19.525867+0000","last_active":"2023-08-25T08:57:56.370989+0000","last_peered":"2023-08-25T08:57:56.370989+0000","last_clean":"2023-08-25T08:57:56.370989+0000","last_became_active":"2023-08-25T08:57:19.525492+0000","last_became_peered":"2023-08-25T08:57:19.525492+0000","last_unstale":"2023-08-25T08:57:56.370989+0000","last_undegraded":"2023-08-25T08:57:56.370989+0000","last_fullsized":"2023-08-25T08:57:56.370989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:30.289254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.095090+0000","last_change":"2023-08-25T08:57:18.711447+0000","last_active":"2023-08-25T08:57:56.095090+0000","last_peered":"2023-08-25T08:57:56.095090+0000","last_clean":"2023-08-25T08:57:56.095090+0000","last_became_active":"2023-08-25T08:57:18.711164+0000","last_became_peered":"2023-08-25T08:57:18.711164+0000","last_unstale":"2023-08-25T08:57:56.095090+0000","last_undegraded":"2023-08-25T08:57:56.095090+0000","last_fullsized":"2023-08-25T08:57:56.095090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:50.213859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369841+0000","last_change":"2023-08-25T08:57:19.528927+0000","last_active":"2023-08-25T08:57:56.369841+0000","last_peered":"2023-08-25T08:57:56.369841+0000","last_clean":"2023-08-25T08:57:56.369841+0000","last_became_active":"2023-08-25T08:57:19.528281+0000","last_became_peered":"2023-08-25T08:57:19.528281+0000","last_unstale":"2023-08-25T08:57:56.369841+0000","last_undegraded":"2023-08-25T08:57:56.369841+0000","last_fullsized":"2023-08-25T08:57:56.369841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:53:16.828292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.055890+0000","last_change":"2023-08-25T08:57:18.713474+0000","last_active":"2023-08-25T08:57:56.055890+0000","last_peered":"2023-08-25T08:57:56.055890+0000","last_clean":"2023-08-25T08:57:56.055890+0000","last_became_active":"2023-08-25T08:57:18.713147+0000","last_became_peered":"2023-08-25T08:57:18.713147+0000","last_unstale":"2023-08-25T08:57:56.055890+0000","last_undegraded":"2023-08-25T08:57:56.055890+0000","last_fullsized":"2023-08-25T08:57:56.055890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:31:07.337490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056063+0000","last_change":"2023-08-25T08:57:19.515035+0000","last_active":"2023-08-25T08:57:56.056063+0000","last_peered":"2023-08-25T08:57:56.056063+0000","last_clean":"2023-08-25T08:57:56.056063+0000","last_became_active":"2023-08-25T08:57:19.514729+0000","last_became_peered":"2023-08-25T08:57:19.514729+0000","last_unstale":"2023-08-25T08:57:56.056063+0000","last_undegraded":"2023-08-25T08:57:56.056063+0000","last_fullsized":"2023-08-25T08:57:56.056063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:07:20.953665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369863+0000","last_change":"2023-08-25T08:57:18.721451+0000","last_active":"2023-08-25T08:57:56.369863+0000","last_peered":"2023-08-25T08:57:56.369863+0000","last_clean":"2023-08-25T08:57:56.369863+0000","last_became_active":"2023-08-25T08:57:18.721132+0000","last_became_peered":"2023-08-25T08:57:18.721132+0000","last_unstale":"2023-08-25T08:57:56.369863+0000","last_undegraded":"2023-08-25T08:57:56.369863+0000","last_fullsized":"2023-08-25T08:57:56.369863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:50.473107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052688+0000","last_change":"2023-08-25T08:57:19.519649+0000","last_active":"2023-08-25T08:57:56.052688+0000","last_peered":"2023-08-25T08:57:56.052688+0000","last_clean":"2023-08-25T08:57:56.052688+0000","last_became_active":"2023-08-25T08:57:19.518907+0000","last_became_peered":"2023-08-25T08:57:19.518907+0000","last_unstale":"2023-08-25T08:57:56.052688+0000","last_undegraded":"2023-08-25T08:57:56.052688+0000","last_fullsized":"2023-08-25T08:57:56.052688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:32.052200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048751+0000","last_change":"2023-08-25T08:57:18.722996+0000","last_active":"2023-08-25T08:57:56.048751+0000","last_peered":"2023-08-25T08:57:56.048751+0000","last_clean":"2023-08-25T08:57:56.048751+0000","last_became_active":"2023-08-25T08:57:18.722662+0000","last_became_peered":"2023-08-25T08:57:18.722662+0000","last_unstale":"2023-08-25T08:57:56.048751+0000","last_undegraded":"2023-08-25T08:57:56.048751+0000","last_fullsized":"2023-08-25T08:57:56.048751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:40:56.622549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370066+0000","last_change":"2023-08-25T08:57:19.529138+0000","last_active":"2023-08-25T08:57:56.370066+0000","last_peered":"2023-08-25T08:57:56.370066+0000","last_clean":"2023-08-25T08:57:56.370066+0000","last_became_active":"2023-08-25T08:57:19.527485+0000","last_became_peered":"2023-08-25T08:57:19.527485+0000","last_unstale":"2023-08-25T08:57:56.370066+0000","last_undegraded":"2023-08-25T08:57:56.370066+0000","last_fullsized":"2023-08-25T08:57:56.370066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:32:46.869523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056148+0000","last_change":"2023-08-25T08:57:18.717811+0000","last_active":"2023-08-25T08:57:56.056148+0000","last_peered":"2023-08-25T08:57:56.056148+0000","last_clean":"2023-08-25T08:57:56.056148+0000","last_became_active":"2023-08-25T08:57:18.717573+0000","last_became_peered":"2023-08-25T08:57:18.717573+0000","last_unstale":"2023-08-25T08:57:56.056148+0000","last_undegraded":"2023-08-25T08:57:56.056148+0000","last_fullsized":"2023-08-25T08:57:56.056148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:11.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053381+0000","last_change":"2023-08-25T08:57:19.528271+0000","last_active":"2023-08-25T08:57:56.053381+0000","last_peered":"2023-08-25T08:57:56.053381+0000","last_clean":"2023-08-25T08:57:56.053381+0000","last_became_active":"2023-08-25T08:57:19.528017+0000","last_became_peered":"2023-08-25T08:57:19.528017+0000","last_unstale":"2023-08-25T08:57:56.053381+0000","last_undegraded":"2023-08-25T08:57:56.053381+0000","last_fullsized":"2023-08-25T08:57:56.053381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:03:37.995896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049179+0000","last_change":"2023-08-25T08:57:18.715063+0000","last_active":"2023-08-25T08:57:56.049179+0000","last_peered":"2023-08-25T08:57:56.049179+0000","last_clean":"2023-08-25T08:57:56.049179+0000","last_became_active":"2023-08-25T08:57:18.714755+0000","last_became_peered":"2023-08-25T08:57:18.714755+0000","last_unstale":"2023-08-25T08:57:56.049179+0000","last_undegraded":"2023-08-25T08:57:56.049179+0000","last_fullsized":"2023-08-25T08:57:56.049179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:12:11.389492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053491+0000","last_change":"2023-08-25T08:57:19.528557+0000","last_active":"2023-08-25T08:57:56.053491+0000","last_peered":"2023-08-25T08:57:56.053491+0000","last_clean":"2023-08-25T08:57:56.053491+0000","last_became_active":"2023-08-25T08:57:19.528284+0000","last_became_peered":"2023-08-25T08:57:19.528284+0000","last_unstale":"2023-08-25T08:57:56.053491+0000","last_undegraded":"2023-08-25T08:57:56.053491+0000","last_fullsized":"2023-08-25T08:57:56.053491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:36:07.147532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052206+0000","last_change":"2023-08-25T08:57:18.722893+0000","last_active":"2023-08-25T08:57:56.052206+0000","last_peered":"2023-08-25T08:57:56.052206+0000","last_clean":"2023-08-25T08:57:56.052206+0000","last_became_active":"2023-08-25T08:57:18.722553+0000","last_became_peered":"2023-08-25T08:57:18.722553+0000","last_unstale":"2023-08-25T08:57:56.052206+0000","last_undegraded":"2023-08-25T08:57:56.052206+0000","last_fullsized":"2023-08-25T08:57:56.052206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:25:49.917208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052837+0000","last_change":"2023-08-25T08:57:19.517975+0000","last_active":"2023-08-25T08:57:56.052837+0000","last_peered":"2023-08-25T08:57:56.052837+0000","last_clean":"2023-08-25T08:57:56.052837+0000","last_became_active":"2023-08-25T08:57:19.517715+0000","last_became_peered":"2023-08-25T08:57:19.517715+0000","last_unstale":"2023-08-25T08:57:56.052837+0000","last_undegraded":"2023-08-25T08:57:56.052837+0000","last_fullsized":"2023-08-25T08:57:56.052837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:48.727902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049833+0000","last_change":"2023-08-25T08:57:18.717331+0000","last_active":"2023-08-25T08:57:56.049833+0000","last_peered":"2023-08-25T08:57:56.049833+0000","last_clean":"2023-08-25T08:57:56.049833+0000","last_became_active":"2023-08-25T08:57:18.716957+0000","last_became_peered":"2023-08-25T08:57:18.716957+0000","last_unstale":"2023-08-25T08:57:56.049833+0000","last_undegraded":"2023-08-25T08:57:56.049833+0000","last_fullsized":"2023-08-25T08:57:56.049833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:17.691850+0000","last_clean_scrub_stamp":"2023-08-25T08:57:17.691850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:46:25.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.052994+0000","last_change":"2023-08-25T08:57:19.515217+0000","last_active":"2023-08-25T08:57:56.052994+0000","last_peered":"2023-08-25T08:57:56.052994+0000","last_clean":"2023-08-25T08:57:56.052994+0000","last_became_active":"2023-08-25T08:57:19.514870+0000","last_became_peered":"2023-08-25T08:57:19.514870+0000","last_unstale":"2023-08-25T08:57:56.052994+0000","last_undegraded":"2023-08-25T08:57:56.052994+0000","last_fullsized":"2023-08-25T08:57:56.052994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:26:25.804692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.056197+0000","last_change":"2023-08-25T08:57:56.056197+0000","last_active":"2023-08-25T08:57:56.056197+0000","last_peered":"2023-08-25T08:57:56.056197+0000","last_clean":"2023-08-25T08:57:56.056197+0000","last_became_active":"2023-08-25T08:57:56.055819+0000","last_became_peered":"2023-08-25T08:57:56.055819+0000","last_unstale":"2023-08-25T08:57:56.056197+0000","last_undegraded":"2023-08-25T08:57:56.056197+0000","last_fullsized":"2023-08-25T08:57:56.056197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:32.670928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048759+0000","last_change":"2023-08-25T08:57:19.521540+0000","last_active":"2023-08-25T08:57:56.048759+0000","last_peered":"2023-08-25T08:57:56.048759+0000","last_clean":"2023-08-25T08:57:56.048759+0000","last_became_active":"2023-08-25T08:57:19.521305+0000","last_became_peered":"2023-08-25T08:57:19.521305+0000","last_unstale":"2023-08-25T08:57:56.048759+0000","last_undegraded":"2023-08-25T08:57:56.048759+0000","last_fullsized":"2023-08-25T08:57:56.048759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:31:51.139757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.048847+0000","last_change":"2023-08-25T08:57:19.526830+0000","last_active":"2023-08-25T08:57:56.048847+0000","last_peered":"2023-08-25T08:57:56.048847+0000","last_clean":"2023-08-25T08:57:56.048847+0000","last_became_active":"2023-08-25T08:57:19.526396+0000","last_became_peered":"2023-08-25T08:57:19.526396+0000","last_unstale":"2023-08-25T08:57:56.048847+0000","last_undegraded":"2023-08-25T08:57:56.048847+0000","last_fullsized":"2023-08-25T08:57:56.048847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:33:00.650463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369667+0000","last_change":"2023-08-25T08:57:19.522557+0000","last_active":"2023-08-25T08:57:56.369667+0000","last_peered":"2023-08-25T08:57:56.369667+0000","last_clean":"2023-08-25T08:57:56.369667+0000","last_became_active":"2023-08-25T08:57:19.522076+0000","last_became_peered":"2023-08-25T08:57:19.522076+0000","last_unstale":"2023-08-25T08:57:56.369667+0000","last_undegraded":"2023-08-25T08:57:56.369667+0000","last_fullsized":"2023-08-25T08:57:56.369667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:25:21.874038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049942+0000","last_change":"2023-08-25T08:57:19.518894+0000","last_active":"2023-08-25T08:57:56.049942+0000","last_peered":"2023-08-25T08:57:56.049942+0000","last_clean":"2023-08-25T08:57:56.049942+0000","last_became_active":"2023-08-25T08:57:19.518599+0000","last_became_peered":"2023-08-25T08:57:19.518599+0000","last_unstale":"2023-08-25T08:57:56.049942+0000","last_undegraded":"2023-08-25T08:57:56.049942+0000","last_fullsized":"2023-08-25T08:57:56.049942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:51:02.889569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051995+0000","last_change":"2023-08-25T08:57:19.513584+0000","last_active":"2023-08-25T08:57:56.051995+0000","last_peered":"2023-08-25T08:57:56.051995+0000","last_clean":"2023-08-25T08:57:56.051995+0000","last_became_active":"2023-08-25T08:57:19.513187+0000","last_became_peered":"2023-08-25T08:57:19.513187+0000","last_unstale":"2023-08-25T08:57:56.051995+0000","last_undegraded":"2023-08-25T08:57:56.051995+0000","last_fullsized":"2023-08-25T08:57:56.051995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:50:58.855527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049253+0000","last_change":"2023-08-25T08:57:19.528534+0000","last_active":"2023-08-25T08:57:56.049253+0000","last_peered":"2023-08-25T08:57:56.049253+0000","last_clean":"2023-08-25T08:57:56.049253+0000","last_became_active":"2023-08-25T08:57:19.528186+0000","last_became_peered":"2023-08-25T08:57:19.528186+0000","last_unstale":"2023-08-25T08:57:56.049253+0000","last_undegraded":"2023-08-25T08:57:56.049253+0000","last_fullsized":"2023-08-25T08:57:56.049253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:16:19.578944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.370573+0000","last_change":"2023-08-25T08:57:19.529932+0000","last_active":"2023-08-25T08:57:56.370573+0000","last_peered":"2023-08-25T08:57:56.370573+0000","last_clean":"2023-08-25T08:57:56.370573+0000","last_became_active":"2023-08-25T08:57:19.529581+0000","last_became_peered":"2023-08-25T08:57:19.529581+0000","last_unstale":"2023-08-25T08:57:56.370573+0000","last_undegraded":"2023-08-25T08:57:56.370573+0000","last_fullsized":"2023-08-25T08:57:56.370573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:53.698651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.051755+0000","last_change":"2023-08-25T08:57:19.527777+0000","last_active":"2023-08-25T08:57:56.051755+0000","last_peered":"2023-08-25T08:57:56.051755+0000","last_clean":"2023-08-25T08:57:56.051755+0000","last_became_active":"2023-08-25T08:57:19.527515+0000","last_became_peered":"2023-08-25T08:57:19.527515+0000","last_unstale":"2023-08-25T08:57:56.051755+0000","last_undegraded":"2023-08-25T08:57:56.051755+0000","last_fullsized":"2023-08-25T08:57:56.051755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:10.845020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.369723+0000","last_change":"2023-08-25T08:57:19.522676+0000","last_active":"2023-08-25T08:57:56.369723+0000","last_peered":"2023-08-25T08:57:56.369723+0000","last_clean":"2023-08-25T08:57:56.369723+0000","last_became_active":"2023-08-25T08:57:19.522282+0000","last_became_peered":"2023-08-25T08:57:19.522282+0000","last_unstale":"2023-08-25T08:57:56.369723+0000","last_undegraded":"2023-08-25T08:57:56.369723+0000","last_fullsized":"2023-08-25T08:57:56.369723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:36:25.933969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.371553+0000","last_change":"2023-08-25T08:57:56.371553+0000","last_active":"2023-08-25T08:57:56.371553+0000","last_peered":"2023-08-25T08:57:56.371553+0000","last_clean":"2023-08-25T08:57:56.371553+0000","last_became_active":"2023-08-25T08:57:56.371059+0000","last_became_peered":"2023-08-25T08:57:56.371059+0000","last_unstale":"2023-08-25T08:57:56.371553+0000","last_undegraded":"2023-08-25T08:57:56.371553+0000","last_fullsized":"2023-08-25T08:57:56.371553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:23:14.732042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,3],"acting":[9,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.049771+0000","last_change":"2023-08-25T08:57:19.518697+0000","last_active":"2023-08-25T08:57:56.049771+0000","last_peered":"2023-08-25T08:57:56.049771+0000","last_clean":"2023-08-25T08:57:56.049771+0000","last_became_active":"2023-08-25T08:57:19.518316+0000","last_became_peered":"2023-08-25T08:57:19.518316+0000","last_unstale":"2023-08-25T08:57:56.049771+0000","last_undegraded":"2023-08-25T08:57:56.049771+0000","last_fullsized":"2023-08-25T08:57:56.049771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:56:02.648409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-08-25T08:57:56.053300+0000","last_change":"2023-08-25T08:57:19.515293+0000","last_active":"2023-08-25T08:57:56.053300+0000","last_peered":"2023-08-25T08:57:56.053300+0000","last_clean":"2023-08-25T08:57:56.053300+0000","last_became_active":"2023-08-25T08:57:19.514883+0000","last_became_peered":"2023-08-25T08:57:19.514883+0000","last_unstale":"2023-08-25T08:57:56.053300+0000","last_undegraded":"2023-08-25T08:57:56.053300+0000","last_fullsized":"2023-08-25T08:57:56.053300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T08:57:18.695735+0000","last_clean_scrub_stamp":"2023-08-25T08:57:18.695735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:53:25.145029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250603,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60994,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":328,"num_read_kb":230,"num_write":148,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":182982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449112,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":443966,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.88}]},{"osd":7,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.746}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.91800000000000004}]},{"osd":10,"last update":"Fri Aug 25 08:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.93200000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678043,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1092447,"data_compressed":42060,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Aug 25 08:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69699999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48699999999999999,"15min":0.48999999999999999},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.84499999999999997,"5min":0.874,"15min":0.874},"last":0.32500000000000001}]},{"osd":2,"last update":"Fri Aug 25 08:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64800000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81699999999999995,"5min":1.075,"15min":1.075},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Aug 25 08:58:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.745,"15min":0.69199999999999995},"min":{"1min":0.44,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.9920000000000009,"5min":9.9920000000000009,"15min":9.9920000000000009},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Aug 25 08:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.82899999999999996,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Aug 25 08:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91900000000000004,"5min":1.016,"15min":1.016},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92900000000000005,"5min":1.127,"15min":1.127},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Aug 25 08:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.85899999999999999,"15min":0.83699999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":4.6950000000000003,"5min":4.6950000000000003,"15min":4.6950000000000003},"last":0.499},{"interface":"front","average":{"1min":1.262,"5min":0.876,"15min":0.81200000000000006},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":10.523999999999999,"5min":10.523999999999999,"15min":10.523999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Aug 25 08:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.373},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.36199999999999999}]},{"osd":8,"last update":"Fri Aug 25 08:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98599999999999999,"5min":1.169,"15min":1.169},"last":0.62},{"interface":"front","average":{"1min":0.752,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.349,"15min":1.349},"last":0.73099999999999998}]},{"osd":9,"last update":"Fri Aug 25 08:58:15 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":0.92400000000000004,"15min":0.86899999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":10.157,"5min":10.157,"15min":10.157},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.2370000000000001,"5min":0.93400000000000005,"15min":0.88400000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":10.263,"5min":10.263,"15min":10.263},"last":0.67100000000000004}]},{"osd":11,"last update":"Fri Aug 25 08:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.70499999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444202,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.64900000000000002,"15min":0.58499999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":13.968,"5min":13.968,"15min":13.968},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":0.82199999999999995,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":13.868,"5min":13.868,"15min":13.868},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.81399999999999995,"15min":0.83499999999999996},"min":{"1min":0.53900000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77500000000000002,"15min":0.78600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1200000000000001,"5min":1.155,"15min":1.155},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.85099999999999998,"15min":0.88300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1919999999999999,"5min":1.641,"15min":1.641},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86399999999999999,"15min":0.88},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.878,"15min":0.89300000000000002},"min":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.274,"5min":1.0029999999999999,"15min":0.95799999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":13.922000000000001,"5min":13.922000000000001,"15min":13.922000000000001},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.90300000000000002,"15min":0.93000000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1579999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0069999999999999,"15min":0.98899999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":6.984,"5min":6.984,"15min":6.984},"last":0.746}]},{"osd":8,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.94399999999999995,"15min":0.97399999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0129999999999999,"5min":1.208,"15min":1.208},"last":0.754},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.92600000000000005,"15min":0.94999999999999996},"min":{"1min":0.55400000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.258,"5min":1.778,"15min":1.778},"last":0.87}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103205,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444122,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.74399999999999999}]},{"osd":1,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.55700000000000005,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.42199999999999999},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.34599999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.875},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.50800000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.77100000000000002}]},{"osd":4,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84799999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68100000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68999999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.89400000000000002}]},{"osd":7,"last update":"Fri Aug 25 08:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.53300000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.878,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.92900000000000005}]},{"osd":9,"last update":"Fri Aug 25 08:57:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.47899999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0780000000000001,"5min":1.155,"15min":1.155},"last":0.95599999999999996}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.02}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332136,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":1036057,"data_compressed":42060,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41499999999999998,"15min":0.40799999999999997},"min":{"1min":0.30399999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.51800000000000002,"5min":0.628,"15min":0.628},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.41699999999999998},"min":{"1min":0.30199999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64700000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.499}]},{"osd":2,"last update":"Fri Aug 25 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58899999999999997,"15min":0.55700000000000005},"min":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86499999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78800000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Aug 25 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75800000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.77200000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Aug 25 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Aug 25 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.51400000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.48499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Aug 25 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.78500000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.84699999999999998}]},{"osd":8,"last update":"Fri Aug 25 08:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85299999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70699999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Aug 25 08:57:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.59499999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91700000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.86499999999999999}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495662,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":443597,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.443,"15min":0.433},"min":{"1min":0.30299999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.64100000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47599999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.66600000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.438}]},{"osd":1,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77000000000000002,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.53700000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88900000000000001,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66000000000000003,"15min":0.63},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88500000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.86799999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.747,"15min":0.76700000000000002},"min":{"1min":0.497,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0609999999999999,"5min":3.9500000000000002,"15min":3.9500000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.52300000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.82799999999999996,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95399999999999996,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.78500000000000003,"15min":0.72299999999999998},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.8650000000000002,"5min":5.8650000000000002,"15min":5.8650000000000002},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Aug 25 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.749,"15min":0.72499999999999998},"min":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1879999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.73699999999999999,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri Aug 25 08:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.131,"15min":1.131},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.008,"5min":1.266,"15min":1.266},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Aug 25 08:57:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.85199999999999998,"15min":0.86699999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.85199999999999998,"15min":0.87},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.85799999999999998}]},{"osd":10,"last update":"Fri Aug 25 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.89400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":500105,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65200000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.621,"15min":0.56399999999999995},"min":{"1min":0.34200000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91800000000000004,"5min":5.4359999999999999,"15min":5.4359999999999999},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Aug 25 08:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.441,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Aug 25 08:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79300000000000004,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Aug 25 08:58:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44,"15min":0.40799999999999997},"min":{"1min":0.32700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.746,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.433},{"interface":"front","average":{"1min":0.504,"5min":0.46600000000000003,"15min":0.42799999999999999},"min":{"1min":0.26900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.73499999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Aug 25 08:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83599999999999997,"5min":1.107,"15min":1.107},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.071,"5min":1.153,"15min":1.153},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 25 08:58:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81999999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68799999999999994}]},{"osd":8,"last update":"Fri Aug 25 08:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.44700000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92300000000000004,"5min":1.044,"15min":1.044},"last":0.85499999999999998}]},{"osd":9,"last update":"Fri Aug 25 08:57:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95099999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.74299999999999999}]},{"osd":10,"last update":"Fri Aug 25 08:57:13 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":444862,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 08:57:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60099999999999998,"15min":0.54000000000000004},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.1259999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.62,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Aug 25 08:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.46100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60999999999999999,"15min":0.56399999999999995},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Aug 25 08:55:37 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.46300000000000002,"15min":0.45000000000000001},"min":{"1min":0.34999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Aug 25 08:58:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67300000000000004,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3109999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2250000000000001,"5min":1.306,"15min":1.306},"last":0.82199999999999995}]},{"osd":5,"last update":"Fri Aug 25 08:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.65000000000000002,"15min":0.626},"min":{"1min":0.497,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Aug 25 08:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64300000000000002,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69799999999999995,"15min":0.66500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Aug 25 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.77000000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68300000000000005}]},{"osd":8,"last update":"Fri Aug 25 08:57:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.78300000000000003,"15min":0.79600000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0700000000000001,"5min":1.302,"15min":1.302},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85999999999999999,"15min":0.86699999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.224,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1}]},{"osd":9,"last update":"Fri Aug 25 08:57:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.88600000000000001,"15min":0.90400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2929999999999999,"5min":1.337,"15min":1.337},"last":0.88400000000000001}]},{"osd":10,"last update":"Fri Aug 25 08:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.89700000000000002}]},{"osd":11,"last update":"Fri Aug 25 08:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95199999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":502118,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79700000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":1.409,"15min":1.409},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.41699999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Aug 25 08:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83899999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64500000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Aug 25 08:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42899999999999999,"15min":0.42099999999999999},"min":{"1min":0.27100000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.371},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Aug 25 08:57:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.621},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":1.032,"15min":1.032},"last":0.5},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63400000000000001,"15min":0.61799999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":1.627,"15min":1.627},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Aug 25 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91800000000000004,"5min":1.117,"15min":1.117},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":1.097,"15min":1.097},"last":0.64500000000000002}]},{"osd":8,"last update":"Fri Aug 25 08:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.316,"5min":0.308,"15min":0.308},"max":{"1min":0.84299999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0289999999999999,"5min":1.069,"15min":1.069},"last":0.438}]},{"osd":9,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0669999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94299999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73999999999999999}]},{"osd":10,"last update":"Fri Aug 25 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443914,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.47099999999999997,"15min":0.42099999999999999},"min":{"1min":0.35199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.45700000000000002,"15min":0.41799999999999998},"min":{"1min":0.35399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Aug 25 08:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.65200000000000002,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.67600000000000005}]},{"osd":2,"last update":"Fri Aug 25 08:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63700000000000001,"15min":0.60599999999999998},"min":{"1min":0.54100000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Aug 25 08:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.67400000000000004,"15min":0.60999999999999999},"min":{"1min":0.53400000000000003,"5min":0.433,"15min":0.433},"max":{"1min":4.1050000000000004,"5min":4.1050000000000004,"15min":4.1050000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.64200000000000002,"15min":0.59799999999999998},"min":{"1min":0.52100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.745}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.52900000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Aug 25 08:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65300000000000002,"15min":0.629},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.68300000000000005,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.6180000000000003,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Aug 25 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71199999999999997,"15min":0.69899999999999995},"min":{"1min":0.50900000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.77900000000000003,"15min":0.73899999999999999},"min":{"1min":0.58499999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.78700000000000003}]},{"osd":8,"last update":"Fri Aug 25 08:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85199999999999998}]},{"osd":9,"last update":"Fri Aug 25 08:58:00 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.88100000000000001,"15min":0.85999999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.3730000000000002,"5min":4.3730000000000002,"15min":4.3730000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.83499999999999996,"15min":0.82099999999999995},"min":{"1min":0.66800000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":0.50600000000000001}]},{"osd":10,"last update":"Fri Aug 25 08:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.76300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":444028,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.438}]},{"osd":1,"last update":"Fri Aug 25 08:58:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.40400000000000003},"min":{"1min":0.316,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57499999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.42199999999999999,"15min":0.40899999999999997},"min":{"1min":0.34300000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63600000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.442}]},{"osd":2,"last update":"Fri Aug 25 08:58:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.61799999999999999,"15min":0.58999999999999997},"min":{"1min":0.58499999999999996,"5min":0.376,"15min":0.376},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.754}]},{"osd":3,"last update":"Fri Aug 25 08:58:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.48499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Aug 25 08:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60099999999999998,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Aug 25 08:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.65900000000000003},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.90700000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Aug 25 08:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59199999999999997,"15min":0.55200000000000005},"min":{"1min":0.496,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.91200000000000003,"5min":1.046,"15min":1.046},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58399999999999996,"15min":0.53800000000000003},"min":{"1min":0.5,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92200000000000004,"5min":1.03,"15min":1.03},"last":0.56699999999999995}]},{"osd":8,"last update":"Fri Aug 25 08:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.83799999999999997}]},{"osd":9,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":1.089,"15min":1.089},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66900000000000004}]},{"osd":10,"last update":"Fri Aug 25 08:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1035410,"data_compressed":42060,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45300000000000001,"15min":0.47099999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67900000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.46700000000000003,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.76600000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Aug 25 08:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61799999999999999,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Aug 25 08:57:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.496,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":1.075,"15min":1.075},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Aug 25 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.627},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0189999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.72099999999999997}]},{"osd":8,"last update":"Fri Aug 25 08:57:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57299999999999995}]},{"osd":9,"last update":"Fri Aug 25 08:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.53900000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.78100000000000003}]},{"osd":10,"last update":"Fri Aug 25 08:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34629,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34629,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36269,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T08:58:18.041 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-08-25T08:58:18.044 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-25T08:58:18.044 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-08-25T08:58:18.045 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph health --format=json 2023-08-25T08:58:18.771 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:18.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:18 smithi099 ceph-mon[97641]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:58:18.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:18 smithi157 ceph-mon[130879]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:58:18.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:18 smithi181 ceph-mon[131296]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T08:58:19.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:19 smithi099 ceph-mon[97641]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:19.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:19 smithi157 ceph-mon[130879]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:19.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:19 smithi181 ceph-mon[131296]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:20.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 -- 172.21.15.99:0/1856980292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa001037a0 msgr2=0x7faa001077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:20.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 --2- 172.21.15.99:0/1856980292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa001037a0 0x7faa001077f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80098f0 tx=0x7fa9e8009c00 comp rx=0 tx=0).stop 2023-08-25T08:58:20.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 -- 172.21.15.99:0/1856980292 shutdown_connections 2023-08-25T08:58:20.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 --2- 172.21.15.99:0/1856980292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa00107ed0 0x7faa0010faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 --2- 172.21.15.99:0/1856980292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00102df0 0x7faa001031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 --2- 172.21.15.99:0/1856980292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa001037a0 0x7faa001077f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 -- 172.21.15.99:0/1856980292 >> 172.21.15.99:0/1856980292 conn(0x7faa000fe660 msgr2=0x7faa00100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:20.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 -- 172.21.15.99:0/1856980292 shutdown_connections 2023-08-25T08:58:20.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.186+0000 7faa04b3f700 1 -- 172.21.15.99:0/1856980292 wait complete. 2023-08-25T08:58:20.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.187+0000 7faa04b3f700 1 Processor -- start 2023-08-25T08:58:20.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7faa04b3f700 1 -- start start 2023-08-25T08:58:20.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7faa04b3f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa00102df0 0x7faa00198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:20.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7faa04b3f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001037a0 0x7faa00199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:20.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7faa04b3f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa00107ed0 0x7faa0019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:20.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7faa04b3f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0019ddc0 con 0x7faa00107ed0 2023-08-25T08:58:20.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7faa04b3f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0019df30 con 0x7faa00102df0 2023-08-25T08:58:20.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7faa04b3f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0019e0f0 con 0x7faa001037a0 2023-08-25T08:58:20.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7fa9fe59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa00102df0 0x7faa00198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:20.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7fa9fed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa00107ed0 0x7faa0019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:20.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7fa9fe59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa00102df0 0x7faa00198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47426/0 (socket says 172.21.15.99:47426) 2023-08-25T08:58:20.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7fa9fe59c700 1 -- 172.21.15.99:0/2198202267 learned_addr learned my addr 172.21.15.99:0/2198202267 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:20.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.188+0000 7fa9f7fff700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001037a0 0x7faa00199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:20.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9fe59c700 1 -- 172.21.15.99:0/2198202267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001037a0 msgr2=0x7faa00199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:20.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9fe59c700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001037a0 0x7faa00199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9fe59c700 1 -- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa00107ed0 msgr2=0x7faa0019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:20.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9fe59c700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa00107ed0 0x7faa0019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9fe59c700 1 -- 172.21.15.99:0/2198202267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f8009d00 con 0x7faa00102df0 2023-08-25T08:58:20.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9fed9d700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa00107ed0 0x7faa0019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:20.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9fe59c700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa00102df0 0x7faa00198e00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000e9a0 tx=0x7fa9f000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:20.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9f77fe700 1 -- 172.21.15.99:0/2198202267 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0009790 con 0x7faa00102df0 2023-08-25T08:58:20.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e8009520 con 0x7faa00102df0 2023-08-25T08:58:20.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9f77fe700 1 -- 172.21.15.99:0/2198202267 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f0010430 con 0x7faa00102df0 2023-08-25T08:58:20.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7fa9f77fe700 1 -- 172.21.15.99:0/2198202267 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0018680 con 0x7faa00102df0 2023-08-25T08:58:20.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.189+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa001a5a60 con 0x7faa00102df0 2023-08-25T08:58:20.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.190+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0004e620 con 0x7faa00102df0 2023-08-25T08:58:20.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.192+0000 7fa9f77fe700 1 -- 172.21.15.99:0/2198202267 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa9f00105a0 con 0x7faa00102df0 2023-08-25T08:58:20.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.194+0000 7fa9f77fe700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9ec09ffe0 0x7fa9ec0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.194+0000 7fa9f77fe700 1 -- 172.21.15.99:0/2198202267 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa9f0014070 con 0x7faa00102df0 2023-08-25T08:58:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.195+0000 7fa9f7fff700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9ec09ffe0 0x7fa9ec0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.195+0000 7fa9f7fff700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9ec09ffe0 0x7fa9ec0a24a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009fd0 tx=0x7fa9e8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:20.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.196+0000 7fa9f77fe700 1 -- 172.21.15.99:0/2198202267 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa9f00953c0 con 0x7faa00102df0 2023-08-25T08:58:20.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.448+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faa0019a160 con 0x7faa00102df0 2023-08-25T08:58:20.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.448+0000 7fa9f77fe700 1 -- 172.21.15.99:0/2198202267 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa9f00989e0 con 0x7faa00102df0 2023-08-25T08:58:20.449 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:20.450 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-25T08:58:20.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9ec09ffe0 msgr2=0x7fa9ec0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:20.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9ec09ffe0 0x7fa9ec0a24a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009fd0 tx=0x7fa9e8019040 comp rx=0 tx=0).stop 2023-08-25T08:58:20.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa00102df0 msgr2=0x7faa00198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:20.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa00102df0 0x7faa00198e00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000e9a0 tx=0x7fa9f000ecb0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 shutdown_connections 2023-08-25T08:58:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa00107ed0 0x7faa0019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa9ec09ffe0 0x7fa9ec0a24a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa00102df0 0x7faa00198e00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 --2- 172.21.15.99:0/2198202267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001037a0 0x7faa00199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 >> 172.21.15.99:0/2198202267 conn(0x7faa000fe660 msgr2=0x7faa000ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 shutdown_connections 2023-08-25T08:58:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:20.451+0000 7faa04b3f700 1 -- 172.21.15.99:0/2198202267 wait complete. 2023-08-25T08:58:20.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:20 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2198202267' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-25T08:58:20.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:20 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2198202267' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-25T08:58:20.945 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-08-25T08:58:20.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:58:20.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:20 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2198202267' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-25T08:58:21.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.153+0000 7f4983fff700 1 -- 172.21.15.99:0/1340578226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 msgr2=0x7f497c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.153+0000 7f4983fff700 1 --2- 172.21.15.99:0/1340578226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 0x7f497c101f80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f496c0099c0 tx=0x7f496c009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.153+0000 7f4983fff700 1 -- 172.21.15.99:0/1340578226 shutdown_connections 2023-08-25T08:58:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.153+0000 7f4983fff700 1 --2- 172.21.15.99:0/1340578226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f497c1024c0 0x7f497c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.154+0000 7f4983fff700 1 --2- 172.21.15.99:0/1340578226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c100930 0x7f497c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.154+0000 7f4983fff700 1 --2- 172.21.15.99:0/1340578226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 0x7f497c101f80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.154+0000 7f4983fff700 1 -- 172.21.15.99:0/1340578226 >> 172.21.15.99:0/1340578226 conn(0x7f497c0fbe60 msgr2=0x7f497c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.154+0000 7f4983fff700 1 -- 172.21.15.99:0/1340578226 shutdown_connections 2023-08-25T08:58:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.154+0000 7f4983fff700 1 -- 172.21.15.99:0/1340578226 wait complete. 2023-08-25T08:58:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.155+0000 7f4983fff700 1 Processor -- start 2023-08-25T08:58:21.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.155+0000 7f4983fff700 1 -- start start 2023-08-25T08:58:21.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.155+0000 7f4983fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f497c100930 0x7f497c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:21.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.155+0000 7f4983fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 0x7f497c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:21.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.155+0000 7f4983fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c1024c0 0x7f497c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:21.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.155+0000 7f4983fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c19e150 con 0x7f497c101b20 2023-08-25T08:58:21.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.155+0000 7f4983fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c19e290 con 0x7f497c1024c0 2023-08-25T08:58:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.155+0000 7f4983fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c19e420 con 0x7f497c100930 2023-08-25T08:58:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 0x7f497c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 0x7f497c196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38114/0 (socket says 172.21.15.99:38114) 2023-08-25T08:58:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 -- 172.21.15.99:0/3878144069 learned_addr learned my addr 172.21.15.99:0/3878144069 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f4981d9b700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f497c100930 0x7f497c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498259c700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c1024c0 0x7f497c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 -- 172.21.15.99:0/3878144069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f497c100930 msgr2=0x7f497c1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f497c100930 0x7f497c1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 -- 172.21.15.99:0/3878144069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c1024c0 msgr2=0x7f497c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c1024c0 0x7f497c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 -- 172.21.15.99:0/3878144069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c0095f0 con 0x7f497c101b20 2023-08-25T08:58:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f4981d9b700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f497c100930 0x7f497c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498259c700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c1024c0 0x7f497c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f498159a700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 0x7f497c196b10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f496c009fd0 tx=0x7f496c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f4972ffd700 1 -- 172.21.15.99:0/3878144069 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c01d070 con 0x7f497c101b20 2023-08-25T08:58:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f4972ffd700 1 -- 172.21.15.99:0/3878144069 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f496c005510 con 0x7f497c101b20 2023-08-25T08:58:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.156+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c19e6a0 con 0x7f497c101b20 2023-08-25T08:58:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.157+0000 7f4972ffd700 1 -- 172.21.15.99:0/3878144069 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c004ae0 con 0x7f497c101b20 2023-08-25T08:58:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.157+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c19eb20 con 0x7f497c101b20 2023-08-25T08:58:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.158+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c10c190 con 0x7f497c101b20 2023-08-25T08:58:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.159+0000 7f4972ffd700 1 -- 172.21.15.99:0/3878144069 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f496c004cc0 con 0x7f497c101b20 2023-08-25T08:58:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.161+0000 7f4972ffd700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f49680a00a0 0x7f49680a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.161+0000 7f4972ffd700 1 -- 172.21.15.99:0/3878144069 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f496c0cc240 con 0x7f497c101b20 2023-08-25T08:58:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.161+0000 7f4981d9b700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f49680a00a0 0x7f49680a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.161+0000 7f4981d9b700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f49680a00a0 0x7f49680a2560 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f497c101980 tx=0x7f4978009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:21.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.165+0000 7f4972ffd700 1 -- 172.21.15.99:0/3878144069 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f496c095e80 con 0x7f497c101b20 2023-08-25T08:58:21.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.404+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f497c1077c0 con 0x7f497c101b20 2023-08-25T08:58:21.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.405+0000 7f4972ffd700 1 -- 172.21.15.99:0/3878144069 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22646 (secure 0 0 0) 0x7f496c0994a0 con 0x7f497c101b20 2023-08-25T08:58:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:21.408 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:56.041917+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:58:21.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f49680a00a0 msgr2=0x7f49680a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f49680a00a0 0x7f49680a2560 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f497c101980 tx=0x7f4978009260 comp rx=0 tx=0).stop 2023-08-25T08:58:21.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 msgr2=0x7f497c196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 0x7f497c196b10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f496c009fd0 tx=0x7f496c00f690 comp rx=0 tx=0).stop 2023-08-25T08:58:21.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 shutdown_connections 2023-08-25T08:58:21.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f49680a00a0 0x7f49680a2560 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c1024c0 0x7f497c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f497c100930 0x7f497c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 --2- 172.21.15.99:0/3878144069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f497c101b20 0x7f497c196b10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 >> 172.21.15.99:0/3878144069 conn(0x7f497c0fbe60 msgr2=0x7f497c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:21.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 shutdown_connections 2023-08-25T08:58:21.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.408+0000 7f4983fff700 1 -- 172.21.15.99:0/3878144069 wait complete. 2023-08-25T08:58:21.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T08:57:17.389400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T08:57:18.695578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T08:58:21.429 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T08:58:21.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.639+0000 7f60f77ff700 1 -- 172.21.15.99:0/3736533816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 msgr2=0x7f60f0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.639+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3736533816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 0x7f60f0100d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0099c0 tx=0x7f60ec009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.639+0000 7f60f77ff700 1 -- 172.21.15.99:0/3736533816 shutdown_connections 2023-08-25T08:58:21.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.639+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3736533816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f01024c0 0x7f60f0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.639+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3736533816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 0x7f60f0100d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.639+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3736533816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0101b20 0x7f60f0101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.639+0000 7f60f77ff700 1 -- 172.21.15.99:0/3736533816 >> 172.21.15.99:0/3736533816 conn(0x7f60f00fbea0 msgr2=0x7f60f00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.639+0000 7f60f77ff700 1 -- 172.21.15.99:0/3736533816 shutdown_connections 2023-08-25T08:58:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.641+0000 7f60f77ff700 1 -- 172.21.15.99:0/3736533816 wait complete. 2023-08-25T08:58:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.642+0000 7f60f77ff700 1 Processor -- start 2023-08-25T08:58:21.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.642+0000 7f60f77ff700 1 -- start start 2023-08-25T08:58:21.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.642+0000 7f60f77ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 0x7f60f0196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:21.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.642+0000 7f60f77ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f0101b20 0x7f60f0196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:21.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.642+0000 7f60f77ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f01024c0 0x7f60f019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:21.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.642+0000 7f60f77ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f019e200 con 0x7f60f01024c0 2023-08-25T08:58:21.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.642+0000 7f60f77ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f019e340 con 0x7f60f0100930 2023-08-25T08:58:21.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.642+0000 7f60f77ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f019e4d0 con 0x7f60f0101b20 2023-08-25T08:58:21.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 0x7f60f0196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:21.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f4d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f0101b20 0x7f60f0196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:21.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 0x7f60f0196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47456/0 (socket says 172.21.15.99:47456) 2023-08-25T08:58:21.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 -- 172.21.15.99:0/3526339416 learned_addr learned my addr 172.21.15.99:0/3526339416 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:21.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f5d9c700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f01024c0 0x7f60f019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:21.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 -- 172.21.15.99:0/3526339416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f0101b20 msgr2=0x7f60f0196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f0101b20 0x7f60f0196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 -- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f01024c0 msgr2=0x7f60f019dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f01024c0 0x7f60f019dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 -- 172.21.15.99:0/3526339416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e0009520 con 0x7f60f0100930 2023-08-25T08:58:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f4d9a700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f0101b20 0x7f60f0196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.643+0000 7f60f559b700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 0x7f60f0196680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009990 tx=0x7f60ec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.644+0000 7f60e67fc700 1 -- 172.21.15.99:0/3526339416 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec01d070 con 0x7f60f0100930 2023-08-25T08:58:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.644+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e8009c40 con 0x7f60f0100930 2023-08-25T08:58:21.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.644+0000 7f60e67fc700 1 -- 172.21.15.99:0/3526339416 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60ec005030 con 0x7f60f0100930 2023-08-25T08:58:21.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.644+0000 7f60e67fc700 1 -- 172.21.15.99:0/3526339416 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec004b30 con 0x7f60f0100930 2023-08-25T08:58:21.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.644+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec0095f0 con 0x7f60f0100930 2023-08-25T08:58:21.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.645+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f01078b0 con 0x7f60f0100930 2023-08-25T08:58:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.647+0000 7f60e67fc700 1 -- 172.21.15.99:0/3526339416 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f60ec004c90 con 0x7f60f0100930 2023-08-25T08:58:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.648+0000 7f60e67fc700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60dc0a0020 0x7f60dc0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.649+0000 7f60e67fc700 1 -- 172.21.15.99:0/3526339416 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f60ec0cbec0 con 0x7f60f0100930 2023-08-25T08:58:21.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.649+0000 7f60f4d9a700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60dc0a0020 0x7f60dc0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:21.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.649+0000 7f60f4d9a700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60dc0a0020 0x7f60dc0a24e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009fd0 tx=0x7f60e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:21.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.652+0000 7f60e67fc700 1 -- 172.21.15.99:0/3526339416 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f60ec095b00 con 0x7f60f0100930 2023-08-25T08:58:21.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:21 smithi099 ceph-mon[97641]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:21.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:21 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:58:21.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:21 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3878144069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.871+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60f010c400 con 0x7f60f0100930 2023-08-25T08:58:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.872+0000 7f60e67fc700 1 -- 172.21.15.99:0/3526339416 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60ec099120 con 0x7f60f0100930 2023-08-25T08:58:21.873 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T08:58:21.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60dc0a0020 msgr2=0x7f60dc0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60dc0a0020 0x7f60dc0a24e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009fd0 tx=0x7f60e0009260 comp rx=0 tx=0).stop 2023-08-25T08:58:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 msgr2=0x7f60f0196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 0x7f60f0196680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009990 tx=0x7f60ec00f690 comp rx=0 tx=0).stop 2023-08-25T08:58:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 shutdown_connections 2023-08-25T08:58:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f01024c0 0x7f60f019dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60dc0a0020 0x7f60dc0a24e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f0100930 0x7f60f0196680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 --2- 172.21.15.99:0/3526339416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60f0101b20 0x7f60f0196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 >> 172.21.15.99:0/3526339416 conn(0x7f60f00fbea0 msgr2=0x7f60f00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 shutdown_connections 2023-08-25T08:58:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:21.875+0000 7f60f77ff700 1 -- 172.21.15.99:0/3526339416 wait complete. 2023-08-25T08:58:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:21 smithi157 ceph-mon[130879]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:21 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:58:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:21 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3878144069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:21.895 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T08:58:21.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:21 smithi181 ceph-mon[131296]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:21.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:21 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:58:21.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:21 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3878144069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:22.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.102+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1563576524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8101b00 msgr2=0x7f9ef8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.102+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1563576524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8101b00 0x7f9ef8101f60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0099c0 tx=0x7f9eec009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.102+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1563576524 shutdown_connections 2023-08-25T08:58:22.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.102+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1563576524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef81024a0 0x7f9ef8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.102+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1563576524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef8100910 0x7f9ef8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.102+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1563576524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8101b00 0x7f9ef8101f60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.102+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1563576524 >> 172.21.15.99:0/1563576524 conn(0x7f9ef80fbe60 msgr2=0x7f9ef80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:22.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.102+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1563576524 shutdown_connections 2023-08-25T08:58:22.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.103+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1563576524 wait complete. 2023-08-25T08:58:22.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.103+0000 7f9efd5ff700 1 Processor -- start 2023-08-25T08:58:22.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9efd5ff700 1 -- start start 2023-08-25T08:58:22.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9efd5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef8100910 0x7f9ef8196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:22.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9efd5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef8101b00 0x7f9ef8196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:22.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9efd5ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef81024a0 0x7f9ef819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:22.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9efd5ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef819e190 con 0x7f9ef81024a0 2023-08-25T08:58:22.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9efd5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef819e2d0 con 0x7f9ef8101b00 2023-08-25T08:58:22.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9efd5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef819e460 con 0x7f9ef8100910 2023-08-25T08:58:22.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9ef67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef8101b00 0x7f9ef8196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:22.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9ef6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef8100910 0x7f9ef8196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:22.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9ef6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef8100910 0x7f9ef8196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34114/0 (socket says 172.21.15.99:34114) 2023-08-25T08:58:22.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9ef67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef8101b00 0x7f9ef8196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47478/0 (socket says 172.21.15.99:47478) 2023-08-25T08:58:22.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9ef6ffd700 1 -- 172.21.15.99:0/1465441689 learned_addr learned my addr 172.21.15.99:0/1465441689 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:22.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.104+0000 7f9ef77fe700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef81024a0 0x7f9ef819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:22.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9ef6ffd700 1 -- 172.21.15.99:0/1465441689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef8101b00 msgr2=0x7f9ef8196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9ef6ffd700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef8101b00 0x7f9ef8196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9ef6ffd700 1 -- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef81024a0 msgr2=0x7f9ef819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9ef6ffd700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef81024a0 0x7f9ef819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9ef6ffd700 1 -- 172.21.15.99:0/1465441689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec0095f0 con 0x7f9ef8100910 2023-08-25T08:58:22.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9ef6ffd700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef8100910 0x7f9ef8196610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800e970 tx=0x7f9ee800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:22.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9edffff700 1 -- 172.21.15.99:0/1465441689 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8009790 con 0x7f9ef8100910 2023-08-25T08:58:22.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9edffff700 1 -- 172.21.15.99:0/1465441689 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ee800c7e0 con 0x7f9ef8100910 2023-08-25T08:58:22.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef819e740 con 0x7f9ef8100910 2023-08-25T08:58:22.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9edffff700 1 -- 172.21.15.99:0/1465441689 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8003c70 con 0x7f9ef8100910 2023-08-25T08:58:22.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.105+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef819ec50 con 0x7f9ef8100910 2023-08-25T08:58:22.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.106+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef8107890 con 0x7f9ef8100910 2023-08-25T08:58:22.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.109+0000 7f9edffff700 1 -- 172.21.15.99:0/1465441689 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f9ee8004220 con 0x7f9ef8100910 2023-08-25T08:58:22.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.110+0000 7f9edffff700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9ee009ffe0 0x7f9ee00a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:22.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.110+0000 7f9edffff700 1 -- 172.21.15.99:0/1465441689 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9ee8014070 con 0x7f9ef8100910 2023-08-25T08:58:22.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.111+0000 7f9ef67fc700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9ee009ffe0 0x7f9ee00a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:22.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.111+0000 7f9ef67fc700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9ee009ffe0 0x7f9ee00a24a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00b580 tx=0x7f9eec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:22.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.114+0000 7f9edffff700 1 -- 172.21.15.99:0/1465441689 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9ee80951e0 con 0x7f9ef8100910 2023-08-25T08:58:22.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.329+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ef810c390 con 0x7f9ef8100910 2023-08-25T08:58:22.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.330+0000 7f9edffff700 1 -- 172.21.15.99:0/1465441689 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ee8098800 con 0x7f9ef8100910 2023-08-25T08:58:22.331 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-08-25T08:58:22.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9ee009ffe0 msgr2=0x7f9ee00a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9ee009ffe0 0x7f9ee00a24a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00b580 tx=0x7f9eec005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef8100910 msgr2=0x7f9ef8196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef8100910 0x7f9ef8196610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800e970 tx=0x7f9ee800ec80 comp rx=0 tx=0).stop 2023-08-25T08:58:22.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 shutdown_connections 2023-08-25T08:58:22.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef81024a0 0x7f9ef819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef8100910 0x7f9ef8196610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f9ee009ffe0 0x7f9ee00a24a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 --2- 172.21.15.99:0/1465441689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef8101b00 0x7f9ef8196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 >> 172.21.15.99:0/1465441689 conn(0x7f9ef80fbe60 msgr2=0x7f9ef80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:22.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 shutdown_connections 2023-08-25T08:58:22.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.333+0000 7f9efd5ff700 1 -- 172.21.15.99:0/1465441689 wait complete. 2023-08-25T08:58:22.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T08:58:22.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 -- 172.21.15.99:0/1480496289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc620101b20 msgr2=0x7fc620101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 --2- 172.21.15.99:0/1480496289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc620101b20 0x7fc620101f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0098f0 tx=0x7fc60c009c00 comp rx=0 tx=0).stop 2023-08-25T08:58:22.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 -- 172.21.15.99:0/1480496289 shutdown_connections 2023-08-25T08:58:22.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 --2- 172.21.15.99:0/1480496289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6201024c0 0x7fc620108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 --2- 172.21.15.99:0/1480496289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc620100930 0x7fc620100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 --2- 172.21.15.99:0/1480496289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc620101b20 0x7fc620101f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 -- 172.21.15.99:0/1480496289 >> 172.21.15.99:0/1480496289 conn(0x7fc6200fbea0 msgr2=0x7fc6200fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 -- 172.21.15.99:0/1480496289 shutdown_connections 2023-08-25T08:58:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.566+0000 7fc626fff700 1 -- 172.21.15.99:0/1480496289 wait complete. 2023-08-25T08:58:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.567+0000 7fc626fff700 1 Processor -- start 2023-08-25T08:58:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc626fff700 1 -- start start 2023-08-25T08:58:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc626fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc620100930 0x7fc6201988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc626fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc620101b20 0x7fc620198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc626fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6201024c0 0x7fc62019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc626fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a0420 con 0x7fc620100930 2023-08-25T08:58:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc626fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a0560 con 0x7fc6201024c0 2023-08-25T08:58:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc626fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a06f0 con 0x7fc620101b20 2023-08-25T08:58:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc624d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc620100930 0x7fc6201988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc62559c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6201024c0 0x7fc62019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:22.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc624d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc620100930 0x7fc6201988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38186/0 (socket says 172.21.15.99:38186) 2023-08-25T08:58:22.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc624d9b700 1 -- 172.21.15.99:0/3072527629 learned_addr learned my addr 172.21.15.99:0/3072527629 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:22.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc61ffff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc620101b20 0x7fc620198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:22.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc624d9b700 1 -- 172.21.15.99:0/3072527629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc620101b20 msgr2=0x7fc620198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc624d9b700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc620101b20 0x7fc620198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc624d9b700 1 -- 172.21.15.99:0/3072527629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6201024c0 msgr2=0x7fc62019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc624d9b700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6201024c0 0x7fc62019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.568+0000 7fc624d9b700 1 -- 172.21.15.99:0/3072527629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c009520 con 0x7fc620100930 2023-08-25T08:58:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.569+0000 7fc624d9b700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc620100930 0x7fc6201988a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc61400e9f0 tx=0x7fc61400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:22.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.569+0000 7fc61dffb700 1 -- 172.21.15.99:0/3072527629 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614004220 con 0x7fc620100930 2023-08-25T08:58:22.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.569+0000 7fc61dffb700 1 -- 172.21.15.99:0/3072527629 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc614004380 con 0x7fc620100930 2023-08-25T08:58:22.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.569+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6201a09d0 con 0x7fc620100930 2023-08-25T08:58:22.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.569+0000 7fc61dffb700 1 -- 172.21.15.99:0/3072527629 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614003be0 con 0x7fc620100930 2023-08-25T08:58:22.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.569+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6201a0f30 con 0x7fc620100930 2023-08-25T08:58:22.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.570+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62010c190 con 0x7fc620100930 2023-08-25T08:58:22.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.572+0000 7fc61dffb700 1 -- 172.21.15.99:0/3072527629 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc614003d40 con 0x7fc620100930 2023-08-25T08:58:22.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.573+0000 7fc61dffb700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc6100a0060 0x7fc6100a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:22.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.573+0000 7fc61dffb700 1 -- 172.21.15.99:0/3072527629 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc614014070 con 0x7fc620100930 2023-08-25T08:58:22.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.574+0000 7fc61ffff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc6100a0060 0x7fc6100a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:22.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.574+0000 7fc61ffff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc6100a0060 0x7fc6100a2520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc60c005e90 tx=0x7fc60c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:22.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.577+0000 7fc61dffb700 1 -- 172.21.15.99:0/3072527629 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc614095030 con 0x7fc620100930 2023-08-25T08:58:22.799 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.798+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6201a0b60 con 0x7fc620100930 2023-08-25T08:58:22.799 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.799+0000 7fc61dffb700 1 -- 172.21.15.99:0/3072527629 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6201a0b60 con 0x7fc620100930 2023-08-25T08:58:22.800 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-08-25T08:58:22.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.801+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc6100a0060 msgr2=0x7fc6100a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.801+0000 7fc626fff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc6100a0060 0x7fc6100a2520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc60c005e90 tx=0x7fc60c019040 comp rx=0 tx=0).stop 2023-08-25T08:58:22.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.801+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc620100930 msgr2=0x7fc6201988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:22.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.801+0000 7fc626fff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc620100930 0x7fc6201988a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc61400e9f0 tx=0x7fc61400ed00 comp rx=0 tx=0).stop 2023-08-25T08:58:22.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.802+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 shutdown_connections 2023-08-25T08:58:22.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.802+0000 7fc626fff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc6100a0060 0x7fc6100a2520 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.802+0000 7fc626fff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6201024c0 0x7fc62019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.802+0000 7fc626fff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc620100930 0x7fc6201988a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.802+0000 7fc626fff700 1 --2- 172.21.15.99:0/3072527629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc620101b20 0x7fc620198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:22.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.802+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 >> 172.21.15.99:0/3072527629 conn(0x7fc6200fbea0 msgr2=0x7fc6200fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:22.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.802+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 shutdown_connections 2023-08-25T08:58:22.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:22.802+0000 7fc626fff700 1 -- 172.21.15.99:0/3072527629 wait complete. 2023-08-25T08:58:22.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:22 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3526339416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:22.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:22 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1465441689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:22.821 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:58:22.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:22 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3526339416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:22.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:22 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1465441689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:22.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:22 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3526339416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:22.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:22 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1465441689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:23.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 -- 172.21.15.99:0/376967057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c101b20 msgr2=0x7fa27c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 --2- 172.21.15.99:0/376967057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c101b20 0x7fa27c101f80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2680099c0 tx=0x7fa268009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 -- 172.21.15.99:0/376967057 shutdown_connections 2023-08-25T08:58:23.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 --2- 172.21.15.99:0/376967057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27c1024c0 0x7fa27c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 --2- 172.21.15.99:0/376967057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c100930 0x7fa27c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 --2- 172.21.15.99:0/376967057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c101b20 0x7fa27c101f80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 -- 172.21.15.99:0/376967057 >> 172.21.15.99:0/376967057 conn(0x7fa27c0fbe60 msgr2=0x7fa27c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 -- 172.21.15.99:0/376967057 shutdown_connections 2023-08-25T08:58:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.028+0000 7fa282dff700 1 -- 172.21.15.99:0/376967057 wait complete. 2023-08-25T08:58:23.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.029+0000 7fa282dff700 1 Processor -- start 2023-08-25T08:58:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa282dff700 1 -- start start 2023-08-25T08:58:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa282dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c100930 0x7fa27c196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa282dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27c101b20 0x7fa27c196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa282dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c1024c0 0x7fa27c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:23.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa282dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c19e1a0 con 0x7fa27c100930 2023-08-25T08:58:23.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa282dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c19e2e0 con 0x7fa27c101b20 2023-08-25T08:58:23.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa282dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c19e470 con 0x7fa27c1024c0 2023-08-25T08:58:23.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa280b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c100930 0x7fa27c196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:23.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa280b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c100930 0x7fa27c196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38196/0 (socket says 172.21.15.99:38196) 2023-08-25T08:58:23.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa280b9b700 1 -- 172.21.15.99:0/3877911231 learned_addr learned my addr 172.21.15.99:0/3877911231 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:23.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa27bfff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27c101b20 0x7fa27c196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:23.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.030+0000 7fa28139c700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c1024c0 0x7fa27c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:23.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa280b9b700 1 -- 172.21.15.99:0/3877911231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c1024c0 msgr2=0x7fa27c19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa280b9b700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c1024c0 0x7fa27c19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa280b9b700 1 -- 172.21.15.99:0/3877911231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27c101b20 msgr2=0x7fa27c196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa280b9b700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27c101b20 0x7fa27c196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa280b9b700 1 -- 172.21.15.99:0/3877911231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2680095f0 con 0x7fa27c100930 2023-08-25T08:58:23.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa27bfff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27c101b20 0x7fa27c196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:23.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa280b9b700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c100930 0x7fa27c196620 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e970 tx=0x7fa27000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:23.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa279ffb700 1 -- 172.21.15.99:0/3877911231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270009790 con 0x7fa27c100930 2023-08-25T08:58:23.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa279ffb700 1 -- 172.21.15.99:0/3877911231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa27000c7e0 con 0x7fa27c100930 2023-08-25T08:58:23.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c19e750 con 0x7fa27c100930 2023-08-25T08:58:23.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa279ffb700 1 -- 172.21.15.99:0/3877911231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270003c70 con 0x7fa27c100930 2023-08-25T08:58:23.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.031+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c19ec60 con 0x7fa27c100930 2023-08-25T08:58:23.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.032+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c107ed0 con 0x7fa27c100930 2023-08-25T08:58:23.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.034+0000 7fa279ffb700 1 -- 172.21.15.99:0/3877911231 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa27000cc50 con 0x7fa27c100930 2023-08-25T08:58:23.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.037+0000 7fa279ffb700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa26c0a0020 0x7fa26c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:23.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.037+0000 7fa279ffb700 1 -- 172.21.15.99:0/3877911231 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa270014070 con 0x7fa27c100930 2023-08-25T08:58:23.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.037+0000 7fa27bfff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa26c0a0020 0x7fa26c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:23.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.038+0000 7fa27bfff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa26c0a0020 0x7fa26c0a24e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b580 tx=0x7fa26801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:23.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.041+0000 7fa279ffb700 1 -- 172.21.15.99:0/3877911231 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa2700951b0 con 0x7fa27c100930 2023-08-25T08:58:23.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.297+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa27c10c3b0 con 0x7fa27c100930 2023-08-25T08:58:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.297+0000 7fa279ffb700 1 -- 172.21.15.99:0/3877911231 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11283 (secure 0 0 0) 0x7fa2700b6e60 con 0x7fa27c100930 2023-08-25T08:58:23.298 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:23.299 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T08:57:19.244738+0000","modified":"2023-08-25T08:57:25.030649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24547,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T08:58:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa26c0a0020 msgr2=0x7fa26c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa26c0a0020 0x7fa26c0a24e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b580 tx=0x7fa26801a040 comp rx=0 tx=0).stop 2023-08-25T08:58:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c100930 msgr2=0x7fa27c196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c100930 0x7fa27c196620 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e970 tx=0x7fa27000ec80 comp rx=0 tx=0).stop 2023-08-25T08:58:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 shutdown_connections 2023-08-25T08:58:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c1024c0 0x7fa27c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa27c100930 0x7fa27c196620 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fa26c0a0020 0x7fa26c0a24e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 --2- 172.21.15.99:0/3877911231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27c101b20 0x7fa27c196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 >> 172.21.15.99:0/3877911231 conn(0x7fa27c0fbe60 msgr2=0x7fa27c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 shutdown_connections 2023-08-25T08:58:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.300+0000 7fa282dff700 1 -- 172.21.15.99:0/3877911231 wait complete. 2023-08-25T08:58:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2023-08-25T08:58:23.321 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:58:23.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 -- 172.21.15.99:0/117632322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 msgr2=0x7f761c100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 --2- 172.21.15.99:0/117632322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 0x7f761c100d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f760c009990 tx=0x7f760c009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 -- 172.21.15.99:0/117632322 shutdown_connections 2023-08-25T08:58:23.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 --2- 172.21.15.99:0/117632322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1024b0 0x7f761c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 --2- 172.21.15.99:0/117632322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 0x7f761c100d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 --2- 172.21.15.99:0/117632322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f761c101b10 0x7f761c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 -- 172.21.15.99:0/117632322 >> 172.21.15.99:0/117632322 conn(0x7f761c0fbe50 msgr2=0x7f761c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:23.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 -- 172.21.15.99:0/117632322 shutdown_connections 2023-08-25T08:58:23.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.539+0000 7f76223ff700 1 -- 172.21.15.99:0/117632322 wait complete. 2023-08-25T08:58:23.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.540+0000 7f76223ff700 1 Processor -- start 2023-08-25T08:58:23.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f76223ff700 1 -- start start 2023-08-25T08:58:23.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f76223ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 0x7f761c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:23.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f76223ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f761c101b10 0x7f761c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:23.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f76223ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1024b0 0x7f761c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:23.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f76223ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1a03a0 con 0x7f761c1024b0 2023-08-25T08:58:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f76223ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1a04e0 con 0x7f761c101b10 2023-08-25T08:58:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f76223ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1a0670 con 0x7f761c100920 2023-08-25T08:58:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 0x7f761c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f761c101b10 0x7f761c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f762099c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1024b0 0x7f761c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 0x7f761c198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34168/0 (socket says 172.21.15.99:34168) 2023-08-25T08:58:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761bfff700 1 -- 172.21.15.99:0/3886418362 learned_addr learned my addr 172.21.15.99:0/3886418362 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761bfff700 1 -- 172.21.15.99:0/3886418362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f761c101b10 msgr2=0x7f761c198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761bfff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f761c101b10 0x7f761c198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761bfff700 1 -- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1024b0 msgr2=0x7f761c19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761bfff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1024b0 0x7f761c19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.541+0000 7f761bfff700 1 -- 172.21.15.99:0/3886418362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c0095f0 con 0x7f761c100920 2023-08-25T08:58:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.542+0000 7f762099c700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1024b0 0x7f761c19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.542+0000 7f761bfff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 0x7f761c198820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f760c005720 tx=0x7f760c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.542+0000 7f76197fa700 1 -- 172.21.15.99:0/3886418362 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c01d070 con 0x7f761c100920 2023-08-25T08:58:23.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.542+0000 7f76197fa700 1 -- 172.21.15.99:0/3886418362 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f760c021410 con 0x7f761c100920 2023-08-25T08:58:23.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.542+0000 7f76197fa700 1 -- 172.21.15.99:0/3886418362 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c017650 con 0x7f761c100920 2023-08-25T08:58:23.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.542+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c1a08f0 con 0x7f761c100920 2023-08-25T08:58:23.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.542+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c1a0d10 con 0x7f761c100920 2023-08-25T08:58:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.543+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c192f70 con 0x7f761c100920 2023-08-25T08:58:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.545+0000 7f76197fa700 1 -- 172.21.15.99:0/3886418362 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f760c021580 con 0x7f761c100920 2023-08-25T08:58:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.546+0000 7f76197fa700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f76040a00a0 0x7f76040a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.546+0000 7f76197fa700 1 -- 172.21.15.99:0/3886418362 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f760c0cc600 con 0x7f761c100920 2023-08-25T08:58:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.547+0000 7f761b7fe700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f76040a00a0 0x7f76040a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:23.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.549+0000 7f761b7fe700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f76040a00a0 0x7f76040a2560 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7610009c60 tx=0x7f7610009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:23.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.549+0000 7f76197fa700 1 -- 172.21.15.99:0/3886418362 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f760c0962d0 con 0x7f761c100920 2023-08-25T08:58:23.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.773+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f761c10c3d0 con 0x7f761c100920 2023-08-25T08:58:23.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.776+0000 7f76197fa700 1 -- 172.21.15.99:0/3886418362 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22646 (secure 0 0 0) 0x7f760c0998f0 con 0x7f761c100920 2023-08-25T08:58:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:23.777 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:56.041917+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:58:23.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f76040a00a0 msgr2=0x7f76040a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f76040a00a0 0x7f76040a2560 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7610009c60 tx=0x7f7610009260 comp rx=0 tx=0).stop 2023-08-25T08:58:23.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 msgr2=0x7f761c198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:23.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 0x7f761c198820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f760c005720 tx=0x7f760c009580 comp rx=0 tx=0).stop 2023-08-25T08:58:23.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 shutdown_connections 2023-08-25T08:58:23.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1024b0 0x7f761c19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f761c100920 0x7f761c198820 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f76040a00a0 0x7f76040a2560 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 --2- 172.21.15.99:0/3886418362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f761c101b10 0x7f761c198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:23.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 >> 172.21.15.99:0/3886418362 conn(0x7f761c0fbe50 msgr2=0x7f761c1073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:23.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.778+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 shutdown_connections 2023-08-25T08:58:23.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:23.779+0000 7f76223ff700 1 -- 172.21.15.99:0/3886418362 wait complete. 2023-08-25T08:58:23.796 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:23 smithi099 ceph-mon[97641]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:23.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:23 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3072527629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:23.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:23 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3877911231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:23.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:23.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:23 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:23.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T08:57:17.389400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T08:57:18.695578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T08:58:23.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T08:58:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:23 smithi157 ceph-mon[130879]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:23 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3072527629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:23 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3877911231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:23.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:23 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:23.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:23 smithi181 ceph-mon[131296]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:23.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:23 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3072527629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:23.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:23 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3877911231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:23.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:23.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:23 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.009+0000 7fb894dff700 1 -- 172.21.15.99:0/2707304670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 msgr2=0x7fb890101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.009+0000 7fb894dff700 1 --2- 172.21.15.99:0/2707304670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 0x7fb890101f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb884009920 tx=0x7fb884009c30 comp rx=0 tx=0).stop 2023-08-25T08:58:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.010+0000 7fb894dff700 1 -- 172.21.15.99:0/2707304670 shutdown_connections 2023-08-25T08:58:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.010+0000 7fb894dff700 1 --2- 172.21.15.99:0/2707304670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8901024c0 0x7fb890108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.010+0000 7fb894dff700 1 --2- 172.21.15.99:0/2707304670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb890100930 0x7fb890100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.010+0000 7fb894dff700 1 --2- 172.21.15.99:0/2707304670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 0x7fb890101f80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.010+0000 7fb894dff700 1 -- 172.21.15.99:0/2707304670 >> 172.21.15.99:0/2707304670 conn(0x7fb8900fbea0 msgr2=0x7fb8900fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.010+0000 7fb894dff700 1 -- 172.21.15.99:0/2707304670 shutdown_connections 2023-08-25T08:58:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.010+0000 7fb894dff700 1 -- 172.21.15.99:0/2707304670 wait complete. 2023-08-25T08:58:24.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb894dff700 1 Processor -- start 2023-08-25T08:58:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb894dff700 1 -- start start 2023-08-25T08:58:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb894dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb890100930 0x7fb890198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb894dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 0x7fb890198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb894dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8901024c0 0x7fb89019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb894dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901a0410 con 0x7fb890100930 2023-08-25T08:58:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb894dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901a0550 con 0x7fb890101b20 2023-08-25T08:58:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb894dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901a06e0 con 0x7fb8901024c0 2023-08-25T08:58:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb88e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb890100930 0x7fb890198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb88e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb890100930 0x7fb890198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38230/0 (socket says 172.21.15.99:38230) 2023-08-25T08:58:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb88e59c700 1 -- 172.21.15.99:0/4078962407 learned_addr learned my addr 172.21.15.99:0/4078962407 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb88dd9b700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 0x7fb890198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.011+0000 7fb88ed9d700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8901024c0 0x7fb89019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb88e59c700 1 -- 172.21.15.99:0/4078962407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8901024c0 msgr2=0x7fb89019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb88e59c700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8901024c0 0x7fb89019fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb88e59c700 1 -- 172.21.15.99:0/4078962407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 msgr2=0x7fb890198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb88e59c700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 0x7fb890198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb88e59c700 1 -- 172.21.15.99:0/4078962407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb884009520 con 0x7fb890100930 2023-08-25T08:58:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb88dd9b700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 0x7fb890198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb88e59c700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb890100930 0x7fb890198890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9f0 tx=0x7fb88000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb87f7fe700 1 -- 172.21.15.99:0/4078962407 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880004220 con 0x7fb890100930 2023-08-25T08:58:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8901a09c0 con 0x7fb890100930 2023-08-25T08:58:24.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb87f7fe700 1 -- 172.21.15.99:0/4078962407 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb880004380 con 0x7fb890100930 2023-08-25T08:58:24.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb87f7fe700 1 -- 172.21.15.99:0/4078962407 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880003be0 con 0x7fb890100930 2023-08-25T08:58:24.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.012+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8901a0f20 con 0x7fb890100930 2023-08-25T08:58:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.013+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb890192f30 con 0x7fb890100930 2023-08-25T08:58:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.015+0000 7fb87f7fe700 1 -- 172.21.15.99:0/4078962407 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb880003d40 con 0x7fb890100930 2023-08-25T08:58:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.016+0000 7fb87f7fe700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb8780a0060 0x7fb8780a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.017+0000 7fb87f7fe700 1 -- 172.21.15.99:0/4078962407 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb880014070 con 0x7fb890100930 2023-08-25T08:58:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.017+0000 7fb88dd9b700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb8780a0060 0x7fb8780a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.017+0000 7fb88dd9b700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb8780a0060 0x7fb8780a2520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb88400fa40 tx=0x7fb884019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:24.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.020+0000 7fb87f7fe700 1 -- 172.21.15.99:0/4078962407 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb880095040 con 0x7fb890100930 2023-08-25T08:58:24.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.247+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8901a0b50 con 0x7fb890100930 2023-08-25T08:58:24.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.247+0000 7fb87f7fe700 1 -- 172.21.15.99:0/4078962407 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8901a0b50 con 0x7fb890100930 2023-08-25T08:58:24.249 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T08:58:24.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.249+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb8780a0060 msgr2=0x7fb8780a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.249+0000 7fb894dff700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb8780a0060 0x7fb8780a2520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb88400fa40 tx=0x7fb884019040 comp rx=0 tx=0).stop 2023-08-25T08:58:24.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.249+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb890100930 msgr2=0x7fb890198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.249+0000 7fb894dff700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb890100930 0x7fb890198890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9f0 tx=0x7fb88000ed00 comp rx=0 tx=0).stop 2023-08-25T08:58:24.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.250+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 shutdown_connections 2023-08-25T08:58:24.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.250+0000 7fb894dff700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8901024c0 0x7fb89019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.250+0000 7fb894dff700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb890100930 0x7fb890198890 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.250+0000 7fb894dff700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb8780a0060 0x7fb8780a2520 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.250+0000 7fb894dff700 1 --2- 172.21.15.99:0/4078962407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb890101b20 0x7fb890198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.250+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 >> 172.21.15.99:0/4078962407 conn(0x7fb8900fbea0 msgr2=0x7fb8901074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:24.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.250+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 shutdown_connections 2023-08-25T08:58:24.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.250+0000 7fb894dff700 1 -- 172.21.15.99:0/4078962407 wait complete. 2023-08-25T08:58:24.271 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T08:58:24.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 -- 172.21.15.99:0/3254712398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84701024a0 msgr2=0x7f8470108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 --2- 172.21.15.99:0/3254712398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84701024a0 0x7f8470108d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f846800b0d0 tx=0x7f846800b3e0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 -- 172.21.15.99:0/3254712398 shutdown_connections 2023-08-25T08:58:24.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 --2- 172.21.15.99:0/3254712398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84701024a0 0x7f8470108d50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 --2- 172.21.15.99:0/3254712398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8470100910 0x7f8470100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 --2- 172.21.15.99:0/3254712398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8470101b00 0x7f8470101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 -- 172.21.15.99:0/3254712398 >> 172.21.15.99:0/3254712398 conn(0x7f84700fbe60 msgr2=0x7f84700fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:24.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 -- 172.21.15.99:0/3254712398 shutdown_connections 2023-08-25T08:58:24.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.480+0000 7f8476bff700 1 -- 172.21.15.99:0/3254712398 wait complete. 2023-08-25T08:58:24.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.481+0000 7f8476bff700 1 Processor -- start 2023-08-25T08:58:24.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.481+0000 7f8476bff700 1 -- start start 2023-08-25T08:58:24.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.481+0000 7f8476bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8470100910 0x7f8470198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.481+0000 7f8476bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8470101b00 0x7f8470198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.481+0000 7f8476bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701024a0 0x7f847019ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.481+0000 7f8476bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84701a0500 con 0x7f8470100910 2023-08-25T08:58:24.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.481+0000 7f8476bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84701a0640 con 0x7f84701024a0 2023-08-25T08:58:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.481+0000 7f8476bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84701a07d0 con 0x7f8470101b00 2023-08-25T08:58:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f846ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8470101b00 0x7f8470198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8470100910 0x7f8470198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f846ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8470101b00 0x7f8470198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34190/0 (socket says 172.21.15.99:34190) 2023-08-25T08:58:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f846ffff700 1 -- 172.21.15.99:0/4280272047 learned_addr learned my addr 172.21.15.99:0/4280272047 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847519c700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701024a0 0x7f847019ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847499b700 1 -- 172.21.15.99:0/4280272047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8470101b00 msgr2=0x7f8470198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847499b700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8470101b00 0x7f8470198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847499b700 1 -- 172.21.15.99:0/4280272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701024a0 msgr2=0x7f847019ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847499b700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701024a0 0x7f847019ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847499b700 1 -- 172.21.15.99:0/4280272047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468009d00 con 0x7f8470100910 2023-08-25T08:58:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847519c700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701024a0 0x7f847019ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.482+0000 7f847499b700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8470100910 0x7f8470198980 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f846400e9c0 tx=0x7f846400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.483+0000 7f846dffb700 1 -- 172.21.15.99:0/4280272047 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464004220 con 0x7f8470100910 2023-08-25T08:58:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.483+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84701a0ab0 con 0x7f8470100910 2023-08-25T08:58:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.483+0000 7f846dffb700 1 -- 172.21.15.99:0/4280272047 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8464004380 con 0x7f8470100910 2023-08-25T08:58:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.483+0000 7f846dffb700 1 -- 172.21.15.99:0/4280272047 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464003be0 con 0x7f8470100910 2023-08-25T08:58:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.483+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84701a1010 con 0x7f8470100910 2023-08-25T08:58:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.484+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8470192f20 con 0x7f8470100910 2023-08-25T08:58:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.486+0000 7f846dffb700 1 -- 172.21.15.99:0/4280272047 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f8464003d40 con 0x7f8470100910 2023-08-25T08:58:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.487+0000 7f846dffb700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f846009ffe0 0x7f84600a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.487+0000 7f846dffb700 1 -- 172.21.15.99:0/4280272047 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8464014070 con 0x7f8470100910 2023-08-25T08:58:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.487+0000 7f846ffff700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f846009ffe0 0x7f84600a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.488+0000 7f846ffff700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f846009ffe0 0x7f84600a24a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f845c005950 tx=0x7f845c00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:24.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.495+0000 7f846dffb700 1 -- 172.21.15.99:0/4280272047 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f84640952c0 con 0x7f8470100910 2023-08-25T08:58:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.710+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f847010c130 con 0x7f8470100910 2023-08-25T08:58:24.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.710+0000 7f846dffb700 1 -- 172.21.15.99:0/4280272047 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84640988e0 con 0x7f8470100910 2023-08-25T08:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-08-25T08:58:24.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.712+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f846009ffe0 msgr2=0x7f84600a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.712+0000 7f8476bff700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f846009ffe0 0x7f84600a24a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f845c005950 tx=0x7f845c00b500 comp rx=0 tx=0).stop 2023-08-25T08:58:24.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.712+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8470100910 msgr2=0x7f8470198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.712+0000 7f8476bff700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8470100910 0x7f8470198980 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f846400e9c0 tx=0x7f846400ecd0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.713+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 shutdown_connections 2023-08-25T08:58:24.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.713+0000 7f8476bff700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f846009ffe0 0x7f84600a24a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.713+0000 7f8476bff700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701024a0 0x7f847019ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.713+0000 7f8476bff700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8470100910 0x7f8470198980 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.713+0000 7f8476bff700 1 --2- 172.21.15.99:0/4280272047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8470101b00 0x7f8470198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.713+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 >> 172.21.15.99:0/4280272047 conn(0x7f84700fbe60 msgr2=0x7f84701073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.713+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 shutdown_connections 2023-08-25T08:58:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.713+0000 7f8476bff700 1 -- 172.21.15.99:0/4280272047 wait complete. 2023-08-25T08:58:24.732 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T08:58:24.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:24 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:24 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:24 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3886418362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:24.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:24 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4078962407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:24.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:24 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:24 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:24 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:24 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:24 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3886418362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:24.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:24 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4078962407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:24.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:24 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:24 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 -- 172.21.15.99:0/4110269491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c074960 msgr2=0x7f4f8c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 --2- 172.21.15.99:0/4110269491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c074960 0x7f4f8c072db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009990 tx=0x7f4f88009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 -- 172.21.15.99:0/4110269491 shutdown_connections 2023-08-25T08:58:24.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 --2- 172.21.15.99:0/4110269491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f8c10cf00 0x7f4f8c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 --2- 172.21.15.99:0/4110269491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c074960 0x7f4f8c072db0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 --2- 172.21.15.99:0/4110269491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8c073380 0x7f4f8c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 -- 172.21.15.99:0/4110269491 >> 172.21.15.99:0/4110269491 conn(0x7f4f8c0fbea0 msgr2=0x7f4f8c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:24.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 -- 172.21.15.99:0/4110269491 shutdown_connections 2023-08-25T08:58:24.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.940+0000 7f4f945ff700 1 -- 172.21.15.99:0/4110269491 wait complete. 2023-08-25T08:58:24.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.941+0000 7f4f945ff700 1 Processor -- start 2023-08-25T08:58:24.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.941+0000 7f4f945ff700 1 -- start start 2023-08-25T08:58:24.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f945ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f8c073380 0x7f4f8c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f945ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8c074960 0x7f4f8c19d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f945ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c10cf00 0x7f4f8c1a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f945ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c1a47c0 con 0x7f4f8c10cf00 2023-08-25T08:58:24.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f945ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c1a4930 con 0x7f4f8c074960 2023-08-25T08:58:24.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f945ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c1a4af0 con 0x7f4f8c073380 2023-08-25T08:58:24.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f92b9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c10cf00 0x7f4f8c1a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f91b9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8c074960 0x7f4f8c19d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f92b9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c10cf00 0x7f4f8c1a41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38272/0 (socket says 172.21.15.99:38272) 2023-08-25T08:58:24.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f92b9c700 1 -- 172.21.15.99:0/1549174074 learned_addr learned my addr 172.21.15.99:0/1549174074 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:24.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f9239b700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f8c073380 0x7f4f8c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f92b9c700 1 -- 172.21.15.99:0/1549174074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f8c073380 msgr2=0x7f4f8c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f92b9c700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f8c073380 0x7f4f8c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f92b9c700 1 -- 172.21.15.99:0/1549174074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8c074960 msgr2=0x7f4f8c19d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:24.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f92b9c700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8c074960 0x7f4f8c19d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:24.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.942+0000 7f4f92b9c700 1 -- 172.21.15.99:0/1549174074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c009520 con 0x7f4f8c10cf00 2023-08-25T08:58:24.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.943+0000 7f4f92b9c700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c10cf00 0x7f4f8c1a41f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400ef30 tx=0x7f4f8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:24.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.943+0000 7f4f837fe700 1 -- 172.21.15.99:0/1549174074 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84018070 con 0x7f4f8c10cf00 2023-08-25T08:58:24.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.943+0000 7f4f837fe700 1 -- 172.21.15.99:0/1549174074 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8400f040 con 0x7f4f8c10cf00 2023-08-25T08:58:24.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.943+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f880095f0 con 0x7f4f8c10cf00 2023-08-25T08:58:24.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.943+0000 7f4f837fe700 1 -- 172.21.15.99:0/1549174074 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8400cd00 con 0x7f4f8c10cf00 2023-08-25T08:58:24.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.943+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c1a50a0 con 0x7f4f8c10cf00 2023-08-25T08:58:24.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.944+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c102a50 con 0x7f4f8c10cf00 2023-08-25T08:58:24.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.946+0000 7f4f837fe700 1 -- 172.21.15.99:0/1549174074 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f4f84003a00 con 0x7f4f8c10cf00 2023-08-25T08:58:24.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.947+0000 7f4f837fe700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f7809ffe0 0x7f4f780a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:24.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.947+0000 7f4f837fe700 1 -- 172.21.15.99:0/1549174074 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4f840cafb0 con 0x7f4f8c10cf00 2023-08-25T08:58:24.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.947+0000 7f4f9239b700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f7809ffe0 0x7f4f780a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:24.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.948+0000 7f4f9239b700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f7809ffe0 0x7f4f780a24a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b2e0 tx=0x7f4f88005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:24.951 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:24.951+0000 7f4f837fe700 1 -- 172.21.15.99:0/1549174074 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4f84094bf0 con 0x7f4f8c10cf00 2023-08-25T08:58:24.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:24 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:24 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:24 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3886418362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:24.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:24 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4078962407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:24.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:24 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:24.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:24 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:25.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.171+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f8c112980 con 0x7f4f8c10cf00 2023-08-25T08:58:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.171+0000 7f4f837fe700 1 -- 172.21.15.99:0/1549174074 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f84098210 con 0x7f4f8c10cf00 2023-08-25T08:58:25.172 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-08-25T08:58:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f7809ffe0 msgr2=0x7f4f780a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f7809ffe0 0x7f4f780a24a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b2e0 tx=0x7f4f88005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c10cf00 msgr2=0x7f4f8c1a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c10cf00 0x7f4f8c1a41f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400ef30 tx=0x7f4f8400c3a0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 shutdown_connections 2023-08-25T08:58:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f8c10cf00 0x7f4f8c1a41f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f8c073380 0x7f4f8c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4f7809ffe0 0x7f4f780a24a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 --2- 172.21.15.99:0/1549174074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8c074960 0x7f4f8c19d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.173+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 >> 172.21.15.99:0/1549174074 conn(0x7f4f8c0fbea0 msgr2=0x7f4f8c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:25.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.174+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 shutdown_connections 2023-08-25T08:58:25.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.174+0000 7f4f945ff700 1 -- 172.21.15.99:0/1549174074 wait complete. 2023-08-25T08:58:25.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:58:25.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 -- 172.21.15.99:0/2393220633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 msgr2=0x7f55c0072d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 --2- 172.21.15.99:0/2393220633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 0x7f55c0072d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009990 tx=0x7f55bc009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 -- 172.21.15.99:0/2393220633 shutdown_connections 2023-08-25T08:58:25.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 --2- 172.21.15.99:0/2393220633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c0108680 0x7f55c010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 --2- 172.21.15.99:0/2393220633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 0x7f55c0072d30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 --2- 172.21.15.99:0/2393220633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c0073300 0x7f55c0073780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 -- 172.21.15.99:0/2393220633 >> 172.21.15.99:0/2393220633 conn(0x7f55c00fbce0 msgr2=0x7f55c00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:25.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 -- 172.21.15.99:0/2393220633 shutdown_connections 2023-08-25T08:58:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.398+0000 7f55c699d700 1 -- 172.21.15.99:0/2393220633 wait complete. 2023-08-25T08:58:25.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c699d700 1 Processor -- start 2023-08-25T08:58:25.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c699d700 1 -- start start 2023-08-25T08:58:25.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c699d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c0073300 0x7f55c0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:25.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c699d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 0x7f55c0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c699d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c0108680 0x7f55c019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c699d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01a00d0 con 0x7f55c00748e0 2023-08-25T08:58:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c699d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01a0210 con 0x7f55c0108680 2023-08-25T08:58:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c699d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01a03a0 con 0x7f55c0073300 2023-08-25T08:58:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c519a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 0x7f55c0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c519a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 0x7f55c0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38288/0 (socket says 172.21.15.99:38288) 2023-08-25T08:58:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.399+0000 7f55c519a700 1 -- 172.21.15.99:0/317154424 learned_addr learned my addr 172.21.15.99:0/317154424 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c599b700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c0073300 0x7f55c0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c519a700 1 -- 172.21.15.99:0/317154424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c0073300 msgr2=0x7f55c0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c519a700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c0073300 0x7f55c0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c619c700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c0108680 0x7f55c019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:25.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c519a700 1 -- 172.21.15.99:0/317154424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c0108680 msgr2=0x7f55c019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c519a700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c0108680 0x7f55c019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c519a700 1 -- 172.21.15.99:0/317154424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0095f0 con 0x7f55c00748e0 2023-08-25T08:58:25.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c619c700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c0108680 0x7f55c019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:58:25.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c519a700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 0x7f55c0198a90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e9a0 tx=0x7f55b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:25.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55b6ffd700 1 -- 172.21.15.99:0/317154424 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0009790 con 0x7f55c00748e0 2023-08-25T08:58:25.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c01a0680 con 0x7f55c00748e0 2023-08-25T08:58:25.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55b6ffd700 1 -- 172.21.15.99:0/317154424 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b000cbd0 con 0x7f55c00748e0 2023-08-25T08:58:25.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55b6ffd700 1 -- 172.21.15.99:0/317154424 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0018680 con 0x7f55c00748e0 2023-08-25T08:58:25.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.400+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c01a0be0 con 0x7f55c00748e0 2023-08-25T08:58:25.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.401+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c0106d80 con 0x7f55c00748e0 2023-08-25T08:58:25.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.403+0000 7f55b6ffd700 1 -- 172.21.15.99:0/317154424 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f55b000cd40 con 0x7f55c00748e0 2023-08-25T08:58:25.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.405+0000 7f55b6ffd700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f55ac0a0160 0x7f55ac0a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:25.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.405+0000 7f55b6ffd700 1 -- 172.21.15.99:0/317154424 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f55b0014070 con 0x7f55c00748e0 2023-08-25T08:58:25.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.405+0000 7f55c599b700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f55ac0a0160 0x7f55ac0a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:25.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.406+0000 7f55c599b700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f55ac0a0160 0x7f55ac0a2620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b2e0 tx=0x7f55bc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:25.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.408+0000 7f55b6ffd700 1 -- 172.21.15.99:0/317154424 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f55b0051e00 con 0x7f55c00748e0 2023-08-25T08:58:25.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.656+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55c010e1d0 con 0x7f55c00748e0 2023-08-25T08:58:25.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.656+0000 7f55b6ffd700 1 -- 172.21.15.99:0/317154424 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11283 (secure 0 0 0) 0x7f55b0095230 con 0x7f55c00748e0 2023-08-25T08:58:25.660 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:25.661 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T08:57:19.244738+0000","modified":"2023-08-25T08:57:25.030649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24547,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T08:58:25.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f55ac0a0160 msgr2=0x7f55ac0a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f55ac0a0160 0x7f55ac0a2620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b2e0 tx=0x7f55bc005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 msgr2=0x7f55c0198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 0x7f55c0198a90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e9a0 tx=0x7f55b000ecb0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 shutdown_connections 2023-08-25T08:58:25.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f55ac0a0160 0x7f55ac0a2620 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c0108680 0x7f55c019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c0073300 0x7f55c0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 --2- 172.21.15.99:0/317154424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c00748e0 0x7f55c0198a90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 >> 172.21.15.99:0/317154424 conn(0x7f55c00fbce0 msgr2=0x7f55c00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 shutdown_connections 2023-08-25T08:58:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.659+0000 7f55c699d700 1 -- 172.21.15.99:0/317154424 wait complete. 2023-08-25T08:58:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2023-08-25T08:58:25.678 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:58:25.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:25 smithi099 ceph-mon[97641]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:25.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:25 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4280272047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:25.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:25 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:58:25.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:25 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:58:25.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:25 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:25.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:25 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1549174074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:25.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 -- 172.21.15.99:0/2416942786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 msgr2=0x7fb424108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/2416942786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 0x7fb424108d70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b0d0 tx=0x7fb41c00b3e0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 -- 172.21.15.99:0/2416942786 shutdown_connections 2023-08-25T08:58:25.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/2416942786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 0x7fb424108d70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/2416942786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424100930 0x7fb424100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/2416942786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb424101b20 0x7fb424101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 -- 172.21.15.99:0/2416942786 >> 172.21.15.99:0/2416942786 conn(0x7fb4240fbe60 msgr2=0x7fb4240fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:25.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 -- 172.21.15.99:0/2416942786 shutdown_connections 2023-08-25T08:58:25.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.876+0000 7fb42a7ff700 1 -- 172.21.15.99:0/2416942786 wait complete. 2023-08-25T08:58:25.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:25 smithi157 ceph-mon[130879]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:25.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:25 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4280272047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:25.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:25 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:58:25.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:25 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:58:25.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:25 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:25.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:25 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1549174074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:25.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.877+0000 7fb42a7ff700 1 Processor -- start 2023-08-25T08:58:25.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.877+0000 7fb42a7ff700 1 -- start start 2023-08-25T08:58:25.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb42a7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424100930 0x7fb4241966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:25.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb42a7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb424101b20 0x7fb424196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:25.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb42a7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 0x7fb42419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:25.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb42a7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42419e240 con 0x7fb4241024c0 2023-08-25T08:58:25.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb42a7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42419e380 con 0x7fb424101b20 2023-08-25T08:58:25.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb42a7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42419e510 con 0x7fb424100930 2023-08-25T08:58:25.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 0x7fb42419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:25.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 0x7fb42419dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38304/0 (socket says 172.21.15.99:38304) 2023-08-25T08:58:25.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 -- 172.21.15.99:0/4217693824 learned_addr learned my addr 172.21.15.99:0/4217693824 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:25.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb423fff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424100930 0x7fb4241966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:25.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb4237fe700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb424101b20 0x7fb424196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:25.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 -- 172.21.15.99:0/4217693824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424100930 msgr2=0x7fb4241966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424100930 0x7fb4241966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 -- 172.21.15.99:0/4217693824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb424101b20 msgr2=0x7fb424196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:25.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb424101b20 0x7fb424196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:25.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 -- 172.21.15.99:0/4217693824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009d00 con 0x7fb4241024c0 2023-08-25T08:58:25.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.878+0000 7fb428d9c700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 0x7fb42419dc70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb41c008eb0 tx=0x7fb41c008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:25.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.879+0000 7fb4217fa700 1 -- 172.21.15.99:0/4217693824 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00e040 con 0x7fb4241024c0 2023-08-25T08:58:25.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.879+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42419e790 con 0x7fb4241024c0 2023-08-25T08:58:25.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.879+0000 7fb4217fa700 1 -- 172.21.15.99:0/4217693824 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb41c005020 con 0x7fb4241024c0 2023-08-25T08:58:25.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.879+0000 7fb4217fa700 1 -- 172.21.15.99:0/4217693824 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c01d6d0 con 0x7fb4241024c0 2023-08-25T08:58:25.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.879+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42419ec10 con 0x7fb4241024c0 2023-08-25T08:58:25.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.880+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42410c1a0 con 0x7fb4241024c0 2023-08-25T08:58:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.882+0000 7fb4217fa700 1 -- 172.21.15.99:0/4217693824 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb41c005420 con 0x7fb4241024c0 2023-08-25T08:58:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.883+0000 7fb4217fa700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb40c0a0020 0x7fb40c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.883+0000 7fb4217fa700 1 -- 172.21.15.99:0/4217693824 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb41c0cc490 con 0x7fb4241024c0 2023-08-25T08:58:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.883+0000 7fb423fff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb40c0a0020 0x7fb40c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.884+0000 7fb423fff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb40c0a0020 0x7fb40c0a24e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb424101980 tx=0x7fb414009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:25.887+0000 7fb4217fa700 1 -- 172.21.15.99:0/4217693824 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb41c0960d0 con 0x7fb4241024c0 2023-08-25T08:58:25.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:25 smithi181 ceph-mon[131296]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:25.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:25 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4280272047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:25.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:25 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:58:25.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:25 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:58:25.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:25 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' 2023-08-25T08:58:25.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:25 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1549174074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:26.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.088+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb424107730 con 0x7fb4241024c0 2023-08-25T08:58:26.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.090+0000 7fb4217fa700 1 -- 172.21.15.99:0/4217693824 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22646 (secure 0 0 0) 0x7fb41c0996f0 con 0x7fb4241024c0 2023-08-25T08:58:26.091 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:26.092 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:56.041917+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:58:26.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb40c0a0020 msgr2=0x7fb40c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb40c0a0020 0x7fb40c0a24e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb424101980 tx=0x7fb414009260 comp rx=0 tx=0).stop 2023-08-25T08:58:26.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 msgr2=0x7fb42419dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 0x7fb42419dc70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb41c008eb0 tx=0x7fb41c008f90 comp rx=0 tx=0).stop 2023-08-25T08:58:26.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 shutdown_connections 2023-08-25T08:58:26.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4241024c0 0x7fb42419dc70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424100930 0x7fb4241966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fb40c0a0020 0x7fb40c0a24e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 --2- 172.21.15.99:0/4217693824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb424101b20 0x7fb424196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 >> 172.21.15.99:0/4217693824 conn(0x7fb4240fbe60 msgr2=0x7fb4240fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 shutdown_connections 2023-08-25T08:58:26.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.093+0000 7fb42a7ff700 1 -- 172.21.15.99:0/4217693824 wait complete. 2023-08-25T08:58:26.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:58:26.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 -- 172.21.15.99:0/3571174118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28101b10 msgr2=0x7ffa28101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/3571174118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28101b10 0x7ffa28101f70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0099c0 tx=0x7ffa1c009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 -- 172.21.15.99:0/3571174118 shutdown_connections 2023-08-25T08:58:26.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/3571174118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa281024b0 0x7ffa28108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/3571174118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa28100920 0x7ffa28100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/3571174118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28101b10 0x7ffa28101f70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 -- 172.21.15.99:0/3571174118 >> 172.21.15.99:0/3571174118 conn(0x7ffa280fbe50 msgr2=0x7ffa280fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 -- 172.21.15.99:0/3571174118 shutdown_connections 2023-08-25T08:58:26.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.309+0000 7ffa2ddff700 1 -- 172.21.15.99:0/3571174118 wait complete. 2023-08-25T08:58:26.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.310+0000 7ffa2ddff700 1 Processor -- start 2023-08-25T08:58:26.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.310+0000 7ffa2ddff700 1 -- start start 2023-08-25T08:58:26.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa2ddff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28100920 0x7ffa281987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa2ddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa28101b10 0x7ffa28198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa2ddff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa281024b0 0x7ffa2819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa2ddff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa281a0360 con 0x7ffa28100920 2023-08-25T08:58:26.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa2ddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa281a04a0 con 0x7ffa28101b10 2023-08-25T08:58:26.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa2ddff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa281a0630 con 0x7ffa281024b0 2023-08-25T08:58:26.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28100920 0x7ffa281987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28100920 0x7ffa281987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38322/0 (socket says 172.21.15.99:38322) 2023-08-25T08:58:26.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa277fe700 1 -- 172.21.15.99:0/4201054525 learned_addr learned my addr 172.21.15.99:0/4201054525 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:26.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa26ffd700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa28101b10 0x7ffa28198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa27fff700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa281024b0 0x7ffa2819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa277fe700 1 -- 172.21.15.99:0/4201054525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa281024b0 msgr2=0x7ffa2819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa277fe700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa281024b0 0x7ffa2819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa277fe700 1 -- 172.21.15.99:0/4201054525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa28101b10 msgr2=0x7ffa28198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa277fe700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa28101b10 0x7ffa28198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa277fe700 1 -- 172.21.15.99:0/4201054525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c0095f0 con 0x7ffa28100920 2023-08-25T08:58:26.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa26ffd700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa28101b10 0x7ffa28198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:26.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.311+0000 7ffa27fff700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa281024b0 0x7ffa2819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:26.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.312+0000 7ffa277fe700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28100920 0x7ffa281987e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800e970 tx=0x7ffa1800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:26.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.312+0000 7ffa24ff9700 1 -- 172.21.15.99:0/4201054525 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18009790 con 0x7ffa28100920 2023-08-25T08:58:26.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.312+0000 7ffa24ff9700 1 -- 172.21.15.99:0/4201054525 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa180044e0 con 0x7ffa28100920 2023-08-25T08:58:26.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.312+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa281a0910 con 0x7ffa28100920 2023-08-25T08:58:26.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.312+0000 7ffa24ff9700 1 -- 172.21.15.99:0/4201054525 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18003e30 con 0x7ffa28100920 2023-08-25T08:58:26.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.312+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa281a0e70 con 0x7ffa28100920 2023-08-25T08:58:26.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.313+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2810c130 con 0x7ffa28100920 2023-08-25T08:58:26.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.315+0000 7ffa24ff9700 1 -- 172.21.15.99:0/4201054525 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ffa18003f90 con 0x7ffa28100920 2023-08-25T08:58:26.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.316+0000 7ffa24ff9700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ffa100a00a0 0x7ffa100a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.317+0000 7ffa24ff9700 1 -- 172.21.15.99:0/4201054525 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ffa18014070 con 0x7ffa28100920 2023-08-25T08:58:26.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.317+0000 7ffa26ffd700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ffa100a00a0 0x7ffa100a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.317+0000 7ffa26ffd700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ffa100a00a0 0x7ffa100a2560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b580 tx=0x7ffa1c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:26.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.320+0000 7ffa24ff9700 1 -- 172.21.15.99:0/4201054525 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ffa180952a0 con 0x7ffa28100920 2023-08-25T08:58:26.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.569+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa281a0aa0 con 0x7ffa28100920 2023-08-25T08:58:26.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.570+0000 7ffa24ff9700 1 -- 172.21.15.99:0/4201054525 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11283 (secure 0 0 0) 0x7ffa281a0aa0 con 0x7ffa28100920 2023-08-25T08:58:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:26.571 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T08:57:19.244738+0000","modified":"2023-08-25T08:57:25.030649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24547,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T08:58:26.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.572+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ffa100a00a0 msgr2=0x7ffa100a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.572+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ffa100a00a0 0x7ffa100a2560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b580 tx=0x7ffa1c01a040 comp rx=0 tx=0).stop 2023-08-25T08:58:26.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.572+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28100920 msgr2=0x7ffa281987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.572+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28100920 0x7ffa281987e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800e970 tx=0x7ffa1800ed30 comp rx=0 tx=0).stop 2023-08-25T08:58:26.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.572+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 shutdown_connections 2023-08-25T08:58:26.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.573+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa281024b0 0x7ffa2819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.573+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa28100920 0x7ffa281987e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.573+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ffa100a00a0 0x7ffa100a2560 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.573+0000 7ffa2ddff700 1 --2- 172.21.15.99:0/4201054525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa28101b10 0x7ffa28198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.573+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 >> 172.21.15.99:0/4201054525 conn(0x7ffa280fbe50 msgr2=0x7ffa280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.573+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 shutdown_connections 2023-08-25T08:58:26.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.573+0000 7ffa2ddff700 1 -- 172.21.15.99:0/4201054525 wait complete. 2023-08-25T08:58:26.575 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2023-08-25T08:58:26.593 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-08-25T08:57:19.244738+0000', 'modified': '2023-08-25T08:57:25.030649+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24593, 'mds_1': 24547, 'mds_2': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/1963370106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 1963370106}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 1963370106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.181:6835/3995966648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3995966648}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 3995966648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6837/2291370984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2291370984}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2291370984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24545': {'gid': 24545, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/135740173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 135740173}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 135740173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/2903255438', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 2903255438}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 2903255438}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6839/1062424301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1062424301}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1062424301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T08:58:26.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-08-25T08:58:26.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-08-25T08:58:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 -- 172.21.15.99:0/4167615295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a8100930 msgr2=0x7f62a8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/4167615295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a8100930 0x7f62a8100d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f62980099c0 tx=0x7f6298009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 -- 172.21.15.99:0/4167615295 shutdown_connections 2023-08-25T08:58:26.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/4167615295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a81024c0 0x7f62a8108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/4167615295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a8100930 0x7f62a8100d50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/4167615295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a8101b20 0x7f62a8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 -- 172.21.15.99:0/4167615295 >> 172.21.15.99:0/4167615295 conn(0x7f62a80fbea0 msgr2=0x7f62a80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 -- 172.21.15.99:0/4167615295 shutdown_connections 2023-08-25T08:58:26.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.793+0000 7f62ac9ff700 1 -- 172.21.15.99:0/4167615295 wait complete. 2023-08-25T08:58:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.794+0000 7f62ac9ff700 1 Processor -- start 2023-08-25T08:58:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62ac9ff700 1 -- start start 2023-08-25T08:58:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62ac9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a8100930 0x7f62a8196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62ac9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a8101b20 0x7f62a8196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62ac9ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a81024c0 0x7f62a819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62ac9ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a819e1d0 con 0x7f62a81024c0 2023-08-25T08:58:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62ac9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a819e310 con 0x7f62a8101b20 2023-08-25T08:58:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62ac9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a819e4a0 con 0x7f62a8100930 2023-08-25T08:58:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62a659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a8100930 0x7f62a8196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62a659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a8100930 0x7f62a8196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34904/0 (socket says 172.21.15.99:34904) 2023-08-25T08:58:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62a659c700 1 -- 172.21.15.99:0/3690624336 learned_addr learned my addr 172.21.15.99:0/3690624336 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:26.799 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a81024c0 0x7f62a819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.799 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.795+0000 7f629ffff700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a8101b20 0x7f62a8196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f62a659c700 1 -- 172.21.15.99:0/3690624336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a8101b20 msgr2=0x7f62a8196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f62a659c700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a8101b20 0x7f62a8196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f62a659c700 1 -- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a81024c0 msgr2=0x7f62a819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f62a659c700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a81024c0 0x7f62a819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f62a659c700 1 -- 172.21.15.99:0/3690624336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980095f0 con 0x7f62a8100930 2023-08-25T08:58:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a81024c0 0x7f62a819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f629ffff700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a8101b20 0x7f62a8196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f62a659c700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a8100930 0x7f62a8196650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6298009990 tx=0x7f629800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f629f7fe700 1 -- 172.21.15.99:0/3690624336 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629801d070 con 0x7f62a8100930 2023-08-25T08:58:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f629f7fe700 1 -- 172.21.15.99:0/3690624336 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6298005030 con 0x7f62a8100930 2023-08-25T08:58:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f62ac9ff700 1 -- 172.21.15.99:0/3690624336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a819e720 con 0x7f62a8100930 2023-08-25T08:58:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.796+0000 7f629f7fe700 1 -- 172.21.15.99:0/3690624336 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298004b30 con 0x7f62a8100930 2023-08-25T08:58:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.797+0000 7f62ac9ff700 1 -- 172.21.15.99:0/3690624336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a819eb40 con 0x7f62a8100930 2023-08-25T08:58:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.799+0000 7f629f7fe700 1 -- 172.21.15.99:0/3690624336 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f6298004cd0 con 0x7f62a8100930 2023-08-25T08:58:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.801+0000 7f62c03bcb80 1 Processor -- start 2023-08-25T08:58:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.801+0000 7f62c03bcb80 1 -- start start 2023-08-25T08:58:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d991980 0x55c59d991da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d9922e0 0x55c59d992720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f629f7fe700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f629409ffe0 0x7f62940a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d992d00 0x55c59d993180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d8f9760 con 0x55c59d991980 2023-08-25T08:58:26.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d8ea730 con 0x55c59d992d00 2023-08-25T08:58:26.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d8e8050 con 0x55c59d9922e0 2023-08-25T08:58:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f629f7fe700 1 -- 172.21.15.99:0/3690624336 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f62980cc260 con 0x7f62a8100930 2023-08-25T08:58:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d991980 0x55c59d991da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d991980 0x55c59d991da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51950/0 (socket says 172.21.15.99:51950) 2023-08-25T08:58:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62a6d9d700 1 -- 172.21.15.99:0/3628206854 learned_addr learned my addr 172.21.15.99:0/3628206854 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62a659c700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f629409ffe0 0x7f62940a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d992d00 0x55c59d993180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f629ffff700 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d9922e0 0x55c59d992720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62a6d9d700 1 -- 172.21.15.99:0/3628206854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d9922e0 msgr2=0x55c59d992720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.802+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d9922e0 0x55c59d992720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f62a6d9d700 1 -- 172.21.15.99:0/3628206854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d992d00 msgr2=0x55c59d993180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d992d00 0x55c59d993180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f62a6d9d700 1 -- 172.21.15.99:0/3628206854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c59d7635e0 con 0x55c59d991980 2023-08-25T08:58:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f629ffff700 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d9922e0 0x55c59d992720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d991980 0x55c59d991da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007d10 tx=0x7f62a0009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b24335c5ec7c08 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f62a659c700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f629409ffe0 0x7f62940a24a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62980a13b0 tx=0x7f6298004f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f629cff9700 1 -- 172.21.15.99:0/3628206854 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0003a00 con 0x55c59d991980 2023-08-25T08:58:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f629cff9700 1 -- 172.21.15.99:0/3628206854 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62a000f040 con 0x55c59d991980 2023-08-25T08:58:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.803+0000 7f629cff9700 1 -- 172.21.15.99:0/3628206854 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0004450 con 0x55c59d991980 2023-08-25T08:58:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 -- 172.21.15.99:0/3628206854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d991980 msgr2=0x55c59d991da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d991980 0x55c59d991da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007d10 tx=0x7f62a0009cf0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 -- 172.21.15.99:0/3628206854 shutdown_connections 2023-08-25T08:58:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d992d00 0x55c59d993180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d991980 0x55c59d991da0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/3628206854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d9922e0 0x55c59d992720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 -- 172.21.15.99:0/3628206854 >> 172.21.15.99:0/3628206854 conn(0x55c59d906090 msgr2=0x55c59d98c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 -- 172.21.15.99:0/3628206854 shutdown_connections 2023-08-25T08:58:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 -- 172.21.15.99:0/3628206854 wait complete. 2023-08-25T08:58:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.804+0000 7f62c03bcb80 1 -- start start 2023-08-25T08:58:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62c03bcb80 1 Processor -- start 2023-08-25T08:58:26.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d9922e0 0x55c59d99b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d992d00 0x55c59d9a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d8e3940 con 0x55c59d992d00 2023-08-25T08:58:26.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d881b60 con 0x55c59d9922e0 2023-08-25T08:58:26.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d881860 con 0x55c59d991980 2023-08-25T08:58:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62c03bcb80 10 client.0 _ll_register_callbacks cb 0x55c59d7cba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d992d00 0x55c59d9a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f629ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d9922e0 0x55c59d99b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d992d00 0x55c59d9a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51954/0 (socket says 172.21.15.99:51954) 2023-08-25T08:58:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62a6d9d700 1 -- 172.21.15.99:0/1374565455 learned_addr learned my addr 172.21.15.99:0/1374565455 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.805+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f629ffff700 1 -- 172.21.15.99:0/1374565455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 msgr2=0x55c59d99b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f629ffff700 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f629ffff700 1 -- 172.21.15.99:0/1374565455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d992d00 msgr2=0x55c59d9a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f629ffff700 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d992d00 0x55c59d9a2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f629ffff700 1 -- 172.21.15.99:0/1374565455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a0007930 con 0x55c59d9922e0 2023-08-25T08:58:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f629ffff700 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d9922e0 0x55c59d99b900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6290011d10 tx=0x7f629000f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=589374490f22598e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f628dffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T08:58:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f628dffb700 1 -- 172.21.15.99:0/1374565455 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290016070 con 0x55c59d9922e0 2023-08-25T08:58:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1374565455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c59d961170 con 0x55c59d9922e0 2023-08-25T08:58:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.806+0000 7f628dffb700 1 -- 172.21.15.99:0/1374565455 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f629000fe10 con 0x55c59d9922e0 2023-08-25T08:58:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f628dffb700 1 -- 172.21.15.99:0/1374565455 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6290004030 con 0x55c59d9922e0 2023-08-25T08:58:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f62c03bcb80 10 client.24977 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1374565455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c59d70fd90 con 0x55c59d9922e0 2023-08-25T08:58:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f628dffb700 1 -- 172.21.15.99:0/1374565455 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c59d961170 con 0x55c59d9922e0 2023-08-25T08:58:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f628dffb700 1 -- 172.21.15.99:0/1374565455 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f629001a430 con 0x55c59d9922e0 2023-08-25T08:58:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f62c03bcb80 10 client.24977 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f62c03bcb80 10 client.24977 resolve_mds: resolved name 'f' to daemon mds.f 2023-08-25T08:58:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f62c03bcb80 20 client.24977 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.807+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55c59d9a6ff0 0x55c59d9a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.808+0000 7f62c03bcb80 4 client.24977 mds_command: new command op to 24517 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:58:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.808+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1374565455 --> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c59d889b50 con 0x55c59d9a6ff0 2023-08-25T08:58:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.808+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55c59d9a6ff0 0x55c59d9a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.808+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55c59d9a6ff0 0x55c59d9a5a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc0d15b64de81c76 server_cookie=45ff1d05c799c43 in_seq=0 out_seq=0 2023-08-25T08:58:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.808+0000 7f628dffb700 10 client.24977 ms_handle_connect on v2:172.21.15.181:6834/3995966648 2023-08-25T08:58:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f628dffb700 1 -- 172.21.15.99:0/1374565455 <== mds.2 v2:172.21.15.181:6834/3995966648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c59d889b50 con 0x55c59d9a6ff0 2023-08-25T08:58:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f628dffb700 10 client.24977 handle_command_reply: tid=0 2023-08-25T08:58:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 client.24977 shutdown 2023-08-25T08:58:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1374565455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d9922e0 msgr2=0x55c59d99b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d9922e0 0x55c59d99b900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6290011d10 tx=0x7f629000f6a0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1374565455 shutdown_connections 2023-08-25T08:58:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d992d00 0x55c59d9a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55c59d9a6ff0 0x55c59d9a5a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f628dffb700 0 client.24977 ms_handle_reset on v2:172.21.15.181:6834/3995966648 2023-08-25T08:58:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1374565455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d9922e0 0x55c59d99b900 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.809+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1374565455 >> 172.21.15.99:0/1374565455 conn(0x55c59d906090 msgr2=0x55c59d986af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.810+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1374565455 shutdown_connections 2023-08-25T08:58:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.810+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1374565455 wait complete. 2023-08-25T08:58:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.810+0000 7f62c03bcb80 20 client.24977 trim_cache size 0 max 16384 2023-08-25T08:58:26.826 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:26 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/317154424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:26.827 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:26 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4217693824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.828+0000 7f62c03bcb80 1 Processor -- start 2023-08-25T08:58:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.828+0000 7f62c03bcb80 1 -- start start 2023-08-25T08:58:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.828+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.828+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d9a2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.828+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.828+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d7658d0 con 0x55c59d9922e0 2023-08-25T08:58:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.828+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d771de0 con 0x55c59d906090 2023-08-25T08:58:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.828+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d89ff00 con 0x55c59d991980 2023-08-25T08:58:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52220/0 (socket says 172.21.15.99:52220) 2023-08-25T08:58:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 -- 172.21.15.99:0/1986342793 learned_addr learned my addr 172.21.15.99:0/1986342793 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f629ffff700 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d9a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 -- 172.21.15.99:0/1986342793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 msgr2=0x55c59d9a2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d9a2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 -- 172.21.15.99:0/1986342793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 msgr2=0x55c59d9991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 -- 172.21.15.99:0/1986342793 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c59d6f8c10 con 0x55c59d906090 2023-08-25T08:58:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.829+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99a0f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009dd0 tx=0x7f62a001b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71df794145a74e7c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f629d7fa700 1 -- 172.21.15.99:0/1986342793 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0014980 con 0x55c59d906090 2023-08-25T08:58:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f629d7fa700 1 -- 172.21.15.99:0/1986342793 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62a000f040 con 0x55c59d906090 2023-08-25T08:58:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f629d7fa700 1 -- 172.21.15.99:0/1986342793 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0005860 con 0x55c59d906090 2023-08-25T08:58:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1986342793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 msgr2=0x55c59d99a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99a0f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009dd0 tx=0x7f62a001b040 comp rx=0 tx=0).stop 2023-08-25T08:58:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1986342793 shutdown_connections 2023-08-25T08:58:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99a0f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/1986342793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d9a2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1986342793 >> 172.21.15.99:0/1986342793 conn(0x55c59d98c580 msgr2=0x55c59d985ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1986342793 shutdown_connections 2023-08-25T08:58:26.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.830+0000 7f62c03bcb80 1 -- 172.21.15.99:0/1986342793 wait complete. 2023-08-25T08:58:26.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.831+0000 7f62c03bcb80 1 -- start start 2023-08-25T08:58:26.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.831+0000 7f62c03bcb80 1 Processor -- start 2023-08-25T08:58:26.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.831+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.831+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.831+0000 7f62c03bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.831+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d9a28d0 con 0x55c59d9922e0 2023-08-25T08:58:26.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.831+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d9a2a40 con 0x55c59d906090 2023-08-25T08:58:26.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.831+0000 7f62c03bcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59d9a2c00 con 0x55c59d991980 2023-08-25T08:58:26.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62c03bcb80 10 client.0 _ll_register_callbacks cb 0x7f62900117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:26.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9fedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51972/0 (socket says 172.21.15.99:51972) 2023-08-25T08:58:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 -- 172.21.15.99:0/4126263095 learned_addr learned my addr 172.21.15.99:0/4126263095 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f629ffff700 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 -- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 msgr2=0x55c59d99f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 -- 172.21.15.99:0/4126263095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 msgr2=0x55c59d99f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 -- 172.21.15.99:0/4126263095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a0031040 con 0x55c59d9922e0 2023-08-25T08:58:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f629ffff700 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f62a6d9d700 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9fedf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007720 tx=0x7f62a0008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d110f6749d08421d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.832+0000 7f628dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:58:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f628dffb700 1 -- 172.21.15.99:0/4126263095 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0005340 con 0x55c59d9922e0 2023-08-25T08:58:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 1 -- 172.21.15.99:0/4126263095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c59d984860 con 0x55c59d9922e0 2023-08-25T08:58:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f628dffb700 1 -- 172.21.15.99:0/4126263095 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f62a00057e0 con 0x55c59d9922e0 2023-08-25T08:58:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f628dffb700 1 -- 172.21.15.99:0/4126263095 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a00054a0 con 0x55c59d9922e0 2023-08-25T08:58:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 10 client.14928 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 1 -- 172.21.15.99:0/4126263095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c59d736200 con 0x55c59d9922e0 2023-08-25T08:58:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f628dffb700 1 -- 172.21.15.99:0/4126263095 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c59d984860 con 0x55c59d9922e0 2023-08-25T08:58:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f628dffb700 1 -- 172.21.15.99:0/4126263095 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f62a00277a0 con 0x55c59d9922e0 2023-08-25T08:58:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 10 client.14928 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 10 client.14928 resolve_mds: resolved name 'f' to daemon mds.f 2023-08-25T08:58:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 20 client.14928 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:26.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55c59d992d00 0x55c59da00400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:26.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 4 client.14928 mds_command: new command op to 24517 tid=0[{"prefix": "status"}] 2023-08-25T08:58:26.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.833+0000 7f62c03bcb80 1 -- 172.21.15.99:0/4126263095 --> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c59d8dc920 con 0x55c59d992d00 2023-08-25T08:58:26.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.834+0000 7f629ffff700 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55c59d992d00 0x55c59da00400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:26.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.834+0000 7f628dffb700 10 client.14928 ms_handle_connect on v2:172.21.15.181:6834/3995966648 2023-08-25T08:58:26.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.834+0000 7f629ffff700 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55c59d992d00 0x55c59da00400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a003a9f8210bbb03 server_cookie=3b25cc2b7068070b in_seq=0 out_seq=0 2023-08-25T08:58:26.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.834+0000 7f628dffb700 1 -- 172.21.15.99:0/4126263095 <== mds.2 v2:172.21.15.181:6834/3995966648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c59d8dc920 con 0x55c59d992d00 2023-08-25T08:58:26.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.834+0000 7f628dffb700 10 client.14928 handle_command_reply: tid=0 2023-08-25T08:58:26.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.834+0000 7f62c03bcb80 1 client.14928 shutdown 2023-08-25T08:58:26.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 -- 172.21.15.99:0/4126263095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 msgr2=0x55c59d9fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9fedf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007720 tx=0x7f62a0008f10 comp rx=0 tx=0).stop 2023-08-25T08:58:26.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 -- 172.21.15.99:0/4126263095 shutdown_connections 2023-08-25T08:58:26.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c59d9922e0 0x55c59d9fedf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59d906090 0x55c59d99f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55c59d992d00 0x55c59da00400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 --2- 172.21.15.99:0/4126263095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c59d991980 0x55c59d99f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f628dffb700 0 client.14928 ms_handle_reset on v2:172.21.15.181:6834/3995966648 2023-08-25T08:58:26.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 -- 172.21.15.99:0/4126263095 >> 172.21.15.99:0/4126263095 conn(0x55c59d98c580 msgr2=0x55c59d9991d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 -- 172.21.15.99:0/4126263095 shutdown_connections 2023-08-25T08:58:26.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 1 -- 172.21.15.99:0/4126263095 wait complete. 2023-08-25T08:58:26.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62c03bcb80 20 client.14928 trim_cache size 0 max 16384 2023-08-25T08:58:26.848 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T08:58:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:58:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2023-08-25T08:58:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24517, 2023-08-25T08:58:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-08-25T08:58:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-08-25T08:58:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-08-25T08:58:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 62.806703550000002, 2023-08-25T08:58:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2023-08-25T08:58:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2023-08-25T08:58:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2023-08-25T08:58:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 89.039917755999994 2023-08-25T08:58:26.850 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T08:58:26.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62ac9ff700 1 -- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f629409ffe0 msgr2=0x7f62940a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f629409ffe0 0x7f62940a24a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62980a13b0 tx=0x7f6298004f80 comp rx=0 tx=0).stop 2023-08-25T08:58:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62ac9ff700 1 -- 172.21.15.99:0/3690624336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a8100930 msgr2=0x7f62a8196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.835+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a8100930 0x7f62a8196650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6298009990 tx=0x7f629800f690 comp rx=0 tx=0).stop 2023-08-25T08:58:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.836+0000 7f62ac9ff700 1 -- 172.21.15.99:0/3690624336 shutdown_connections 2023-08-25T08:58:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.836+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a81024c0 0x7f62a819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.836+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a8100930 0x7f62a8196650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.836+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f629409ffe0 0x7f62940a24a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.836+0000 7f62ac9ff700 1 --2- 172.21.15.99:0/3690624336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a8101b20 0x7f62a8196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.836+0000 7f62ac9ff700 1 -- 172.21.15.99:0/3690624336 >> 172.21.15.99:0/3690624336 conn(0x7f62a80fbea0 msgr2=0x7f62a80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.836+0000 7f62ac9ff700 1 -- 172.21.15.99:0/3690624336 shutdown_connections 2023-08-25T08:58:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:26.836+0000 7f62ac9ff700 1 -- 172.21.15.99:0/3690624336 wait complete. 2023-08-25T08:58:26.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-08-25T08:58:26.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:26 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/317154424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:26.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:26 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4217693824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:26.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:26 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/317154424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:26.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:26 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4217693824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 -- 172.21.15.99:0/3412914728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 msgr2=0x7fd330100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 --2- 172.21.15.99:0/3412914728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 0x7fd330100d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0099c0 tx=0x7fd32c009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 -- 172.21.15.99:0/3412914728 shutdown_connections 2023-08-25T08:58:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 --2- 172.21.15.99:0/3412914728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3301024c0 0x7fd330108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 --2- 172.21.15.99:0/3412914728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 0x7fd330100d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 --2- 172.21.15.99:0/3412914728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd330101b20 0x7fd330101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 -- 172.21.15.99:0/3412914728 >> 172.21.15.99:0/3412914728 conn(0x7fd3300fbea0 msgr2=0x7fd3300fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 -- 172.21.15.99:0/3412914728 shutdown_connections 2023-08-25T08:58:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.052+0000 7fd337dff700 1 -- 172.21.15.99:0/3412914728 wait complete. 2023-08-25T08:58:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd337dff700 1 Processor -- start 2023-08-25T08:58:27.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd337dff700 1 -- start start 2023-08-25T08:58:27.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd337dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 0x7fd3300746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd337dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd330101b20 0x7fd330072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd337dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3301024c0 0x7fd330073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd337dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300739f0 con 0x7fd3301024c0 2023-08-25T08:58:27.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd337dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330073b60 con 0x7fd330100930 2023-08-25T08:58:27.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd337dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3301a0650 con 0x7fd330101b20 2023-08-25T08:58:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.053+0000 7fd335b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 0x7fd3300746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd33639c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3301024c0 0x7fd330073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd335b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 0x7fd3300746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52240/0 (socket says 172.21.15.99:52240) 2023-08-25T08:58:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd335b9b700 1 -- 172.21.15.99:0/477203065 learned_addr learned my addr 172.21.15.99:0/477203065 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd33539a700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd330101b20 0x7fd330072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd335b9b700 1 -- 172.21.15.99:0/477203065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd330101b20 msgr2=0x7fd330072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd335b9b700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd330101b20 0x7fd330072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd335b9b700 1 -- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3301024c0 msgr2=0x7fd330073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd335b9b700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3301024c0 0x7fd330073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd335b9b700 1 -- 172.21.15.99:0/477203065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0095f0 con 0x7fd330100930 2023-08-25T08:58:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd33639c700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3301024c0 0x7fd330073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd335b9b700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 0x7fd3300746d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0058e0 tx=0x7fd32c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd326ffd700 1 -- 172.21.15.99:0/477203065 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c01d070 con 0x7fd330100930 2023-08-25T08:58:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd326ffd700 1 -- 172.21.15.99:0/477203065 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd32c005050 con 0x7fd330100930 2023-08-25T08:58:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd337dff700 1 -- 172.21.15.99:0/477203065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3301a08d0 con 0x7fd330100930 2023-08-25T08:58:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.054+0000 7fd326ffd700 1 -- 172.21.15.99:0/477203065 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c004b30 con 0x7fd330100930 2023-08-25T08:58:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.055+0000 7fd337dff700 1 -- 172.21.15.99:0/477203065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3301a0cf0 con 0x7fd330100930 2023-08-25T08:58:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.058+0000 7fd326ffd700 1 -- 172.21.15.99:0/477203065 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fd32c0051c0 con 0x7fd330100930 2023-08-25T08:58:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.059+0000 7fd34b7a1b80 1 Processor -- start 2023-08-25T08:58:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.059+0000 7fd34b7a1b80 1 -- start start 2023-08-25T08:58:27.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 0x55596b903da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b9042e0 0x55596b904720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b904d00 0x55596b905180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b86b760 con 0x55596b904d00 2023-08-25T08:58:27.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b85c730 con 0x55596b9042e0 2023-08-25T08:58:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b85a050 con 0x55596b903980 2023-08-25T08:58:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33539a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b904d00 0x55596b905180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33539a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b904d00 0x55596b905180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52002/0 (socket says 172.21.15.99:52002) 2023-08-25T08:58:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33539a700 1 -- 172.21.15.99:0/668362442 learned_addr learned my addr 172.21.15.99:0/668362442 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd326ffd700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd31c09ffe0 0x7fd31c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd335b9b700 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b9042e0 0x55596b904720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33639c700 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 0x55596b903da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd326ffd700 1 -- 172.21.15.99:0/477203065 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd32c0cc220 con 0x7fd330100930 2023-08-25T08:58:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33639c700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd31c09ffe0 0x7fd31c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33539a700 1 -- 172.21.15.99:0/668362442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 msgr2=0x55596b903da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33539a700 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 0x55596b903da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33539a700 1 -- 172.21.15.99:0/668362442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b9042e0 msgr2=0x55596b904720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33539a700 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b9042e0 0x55596b904720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.060+0000 7fd33539a700 1 -- 172.21.15.99:0/668362442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55596b6d55e0 con 0x55596b904d00 2023-08-25T08:58:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.061+0000 7fd33539a700 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b904d00 0x55596b905180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd320009d80 tx=0x7fd320011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa15815a2b9d70d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.061+0000 7fd33639c700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd31c09ffe0 0x7fd31c0a24a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd328009fd0 tx=0x7fd32800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.061+0000 7fd31bfff700 1 -- 172.21.15.99:0/668362442 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320004230 con 0x55596b904d00 2023-08-25T08:58:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.061+0000 7fd31bfff700 1 -- 172.21.15.99:0/668362442 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd320004390 con 0x55596b904d00 2023-08-25T08:58:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.061+0000 7fd31bfff700 1 -- 172.21.15.99:0/668362442 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320019860 con 0x55596b904d00 2023-08-25T08:58:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/668362442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b904d00 msgr2=0x55596b905180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b904d00 0x55596b905180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd320009d80 tx=0x7fd320011720 comp rx=0 tx=0).stop 2023-08-25T08:58:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/668362442 shutdown_connections 2023-08-25T08:58:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b904d00 0x55596b905180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 0x55596b903da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/668362442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b9042e0 0x55596b904720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/668362442 >> 172.21.15.99:0/668362442 conn(0x55596b878090 msgr2=0x55596b8fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/668362442 shutdown_connections 2023-08-25T08:58:27.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/668362442 wait complete. 2023-08-25T08:58:27.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.062+0000 7fd34b7a1b80 1 -- start start 2023-08-25T08:58:27.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.063+0000 7fd34b7a1b80 1 Processor -- start 2023-08-25T08:58:27.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.063+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 0x55596b90d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.063+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b90d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.063+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b904d00 0x55596b9148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.063+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b855940 con 0x55596b9042e0 2023-08-25T08:58:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.063+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b7f3b60 con 0x55596b904d00 2023-08-25T08:58:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.063+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b7f3860 con 0x55596b903980 2023-08-25T08:58:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd34b7a1b80 10 client.0 _ll_register_callbacks cb 0x55596b73da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b90d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd335b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 0x55596b90d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b90d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52012/0 (socket says 172.21.15.99:52012) 2023-08-25T08:58:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 -- 172.21.15.99:0/1186361456 learned_addr learned my addr 172.21.15.99:0/1186361456 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33639c700 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b904d00 0x55596b9148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 -- 172.21.15.99:0/1186361456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 msgr2=0x55596b90d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 0x55596b90d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 -- 172.21.15.99:0/1186361456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b904d00 msgr2=0x55596b9148a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b904d00 0x55596b9148a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 -- 172.21.15.99:0/1186361456 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3200099e0 con 0x55596b9042e0 2023-08-25T08:58:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33639c700 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b904d00 0x55596b9148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd33539a700 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b90d860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3200041e0 tx=0x7fd320004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434e540809a5adf3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.064+0000 7fd3197fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:58:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd3197fa700 1 -- 172.21.15.99:0/1186361456 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320005240 con 0x55596b9042e0 2023-08-25T08:58:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/1186361456 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55596b8d3170 con 0x55596b9042e0 2023-08-25T08:58:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd3197fa700 1 -- 172.21.15.99:0/1186361456 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd320005ce0 con 0x55596b9042e0 2023-08-25T08:58:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd3197fa700 1 -- 172.21.15.99:0/1186361456 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd320003da0 con 0x55596b9042e0 2023-08-25T08:58:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 10 client.14940 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/1186361456 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55596b681d90 con 0x55596b9042e0 2023-08-25T08:58:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd3197fa700 1 -- 172.21.15.99:0/1186361456 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55596b8d3170 con 0x55596b9042e0 2023-08-25T08:58:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd3197fa700 1 -- 172.21.15.99:0/1186361456 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd320021040 con 0x55596b9042e0 2023-08-25T08:58:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 10 client.14940 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 10 client.14940 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-25T08:58:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 20 client.14940 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x55596b918e90 0x55596b917950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 4 client.14940 mds_command: new command op to 24547 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:58:27.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.065+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/1186361456 --> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55596b7fbb50 con 0x55596b918e90 2023-08-25T08:58:27.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.066+0000 7fd335b9b700 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x55596b918e90 0x55596b917950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.066+0000 7fd335b9b700 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x55596b918e90 0x55596b917950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7ab19d1a5afbce3 server_cookie=68618dba17bad914 in_seq=0 out_seq=0 2023-08-25T08:58:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.066+0000 7fd3197fa700 10 client.14940 ms_handle_connect on v2:172.21.15.181:6838/2903255438 2023-08-25T08:58:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd3197fa700 1 -- 172.21.15.99:0/1186361456 <== mds.1 v2:172.21.15.181:6838/2903255438 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55596b7fbb50 con 0x55596b918e90 2023-08-25T08:58:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd3197fa700 10 client.14940 handle_command_reply: tid=0 2023-08-25T08:58:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 client.14940 shutdown 2023-08-25T08:58:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/1186361456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 msgr2=0x55596b90d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b90d860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3200041e0 tx=0x7fd320004210 comp rx=0 tx=0).stop 2023-08-25T08:58:27.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/1186361456 shutdown_connections 2023-08-25T08:58:27.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x55596b918e90 0x55596b917950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b904d00 0x55596b9148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b903980 0x55596b90d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd3197fa700 0 client.14940 ms_handle_reset on v2:172.21.15.181:6838/2903255438 2023-08-25T08:58:27.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/1186361456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b90d860 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/1186361456 >> 172.21.15.99:0/1186361456 conn(0x55596b878090 msgr2=0x55596b8f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/1186361456 shutdown_connections 2023-08-25T08:58:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.067+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/1186361456 wait complete. 2023-08-25T08:58:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.068+0000 7fd34b7a1b80 20 client.14940 trim_cache size 0 max 16384 2023-08-25T08:58:27.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd34b7a1b80 1 Processor -- start 2023-08-25T08:58:27.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd34b7a1b80 1 -- start start 2023-08-25T08:58:27.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b903980 0x55596b8fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b914860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 0x55596b907bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b6d78d0 con 0x55596b9042e0 2023-08-25T08:58:27.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b7efe00 con 0x55596b903980 2023-08-25T08:58:27.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b7fd470 con 0x55596b904d00 2023-08-25T08:58:27.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd33539a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b914860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.086+0000 7fd33539a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b914860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52014/0 (socket says 172.21.15.99:52014) 2023-08-25T08:58:27.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd33539a700 1 -- 172.21.15.99:0/811275410 learned_addr learned my addr 172.21.15.99:0/811275410 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd335b9b700 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b903980 0x55596b8fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd33639c700 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 0x55596b907bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd33539a700 1 -- 172.21.15.99:0/811275410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 msgr2=0x55596b907bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd33539a700 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 0x55596b907bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd33539a700 1 -- 172.21.15.99:0/811275410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b903980 msgr2=0x55596b8fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd33539a700 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b903980 0x55596b8fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd33539a700 1 -- 172.21.15.99:0/811275410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55596b66ac10 con 0x55596b9042e0 2023-08-25T08:58:27.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd335b9b700 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b903980 0x55596b8fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:27.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd33539a700 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b914860 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd320004e50 tx=0x7fd320019510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bae8e5a136757c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd324bff700 1 -- 172.21.15.99:0/811275410 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320004330 con 0x55596b9042e0 2023-08-25T08:58:27.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd324bff700 1 -- 172.21.15.99:0/811275410 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd320003bd0 con 0x55596b9042e0 2023-08-25T08:58:27.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.087+0000 7fd324bff700 1 -- 172.21.15.99:0/811275410 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320004fa0 con 0x55596b9042e0 2023-08-25T08:58:27.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/811275410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 msgr2=0x55596b914860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b914860 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd320004e50 tx=0x7fd320019510 comp rx=0 tx=0).stop 2023-08-25T08:58:27.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/811275410 shutdown_connections 2023-08-25T08:58:27.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 0x55596b907bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b903980 0x55596b8fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/811275410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b9042e0 0x55596b914860 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/811275410 >> 172.21.15.99:0/811275410 conn(0x55596b878090 msgr2=0x55596b8f7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/811275410 shutdown_connections 2023-08-25T08:58:27.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/811275410 wait complete. 2023-08-25T08:58:27.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.088+0000 7fd34b7a1b80 1 -- start start 2023-08-25T08:58:27.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd34b7a1b80 1 Processor -- start 2023-08-25T08:58:27.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b903980 0x55596b911890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 0x55596b913de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd34b7a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b914320 0x55596b914780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b96c880 con 0x55596b903980 2023-08-25T08:58:27.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b96c9c0 con 0x55596b914320 2023-08-25T08:58:27.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd34b7a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b96cb00 con 0x55596b904d00 2023-08-25T08:58:27.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd34b7a1b80 10 client.0 _ll_register_callbacks cb 0x7fd32c009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:27.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd335b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b903980 0x55596b911890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd335b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b903980 0x55596b911890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52020/0 (socket says 172.21.15.99:52020) 2023-08-25T08:58:27.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd335b9b700 1 -- 172.21.15.99:0/2374441130 learned_addr learned my addr 172.21.15.99:0/2374441130 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd33539a700 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 0x55596b913de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.089+0000 7fd33639c700 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b914320 0x55596b914780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd335b9b700 1 -- 172.21.15.99:0/2374441130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 msgr2=0x55596b913de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd335b9b700 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 0x55596b913de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd335b9b700 1 -- 172.21.15.99:0/2374441130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b914320 msgr2=0x55596b914780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd335b9b700 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b914320 0x55596b914780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd335b9b700 1 -- 172.21.15.99:0/2374441130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55596b681d90 con 0x55596b903980 2023-08-25T08:58:27.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd335b9b700 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b903980 0x55596b911890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0c9b80 tx=0x7fd32c069430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f5645aa149a8cf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd3197fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:58:27.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd3197fa700 1 -- 172.21.15.99:0/2374441130 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c004e90 con 0x55596b903980 2023-08-25T08:58:27.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/2374441130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55596b8f6860 con 0x55596b903980 2023-08-25T08:58:27.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd3197fa700 1 -- 172.21.15.99:0/2374441130 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd32c09d950 con 0x55596b903980 2023-08-25T08:58:27.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd3197fa700 1 -- 172.21.15.99:0/2374441130 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd32c0a18a0 con 0x55596b903980 2023-08-25T08:58:27.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd34b7a1b80 10 client.14946 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:27.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/2374441130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55596b6a8200 con 0x55596b903980 2023-08-25T08:58:27.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.090+0000 7fd3197fa700 1 -- 172.21.15.99:0/2374441130 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55596b8f6860 con 0x55596b903980 2023-08-25T08:58:27.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd3197fa700 1 -- 172.21.15.99:0/2374441130 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd32c065670 con 0x55596b903980 2023-08-25T08:58:27.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd34b7a1b80 10 client.14946 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:27.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd34b7a1b80 10 client.14946 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-25T08:58:27.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd34b7a1b80 20 client.14946 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:27.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x55596b9719f0 0x55596b96de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd34b7a1b80 4 client.14946 mds_command: new command op to 24547 tid=0[{"prefix": "status"}] 2023-08-25T08:58:27.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/2374441130 --> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55596b84e920 con 0x55596b9719f0 2023-08-25T08:58:27.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd33539a700 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x55596b9719f0 0x55596b96de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd3197fa700 10 client.14946 ms_handle_connect on v2:172.21.15.181:6838/2903255438 2023-08-25T08:58:27.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.091+0000 7fd33539a700 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x55596b9719f0 0x55596b96de40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c705d469836584a server_cookie=d68a3d7c416fe80b in_seq=0 out_seq=0 2023-08-25T08:58:27.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd3197fa700 1 -- 172.21.15.99:0/2374441130 <== mds.1 v2:172.21.15.181:6838/2903255438 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55596b84e920 con 0x55596b9719f0 2023-08-25T08:58:27.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd3197fa700 10 client.14946 handle_command_reply: tid=0 2023-08-25T08:58:27.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 client.14946 shutdown 2023-08-25T08:58:27.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/2374441130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b903980 msgr2=0x55596b911890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b903980 0x55596b911890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0c9b80 tx=0x7fd32c069430 comp rx=0 tx=0).stop 2023-08-25T08:58:27.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/2374441130 shutdown_connections 2023-08-25T08:58:27.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x55596b9719f0 0x55596b96de40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55596b914320 0x55596b914780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd3197fa700 0 client.14946 ms_handle_reset on v2:172.21.15.181:6838/2903255438 2023-08-25T08:58:27.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55596b903980 0x55596b911890 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 --2- 172.21.15.99:0/2374441130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55596b904d00 0x55596b913de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/2374441130 >> 172.21.15.99:0/2374441130 conn(0x55596b878090 msgr2=0x55596b907bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/2374441130 shutdown_connections 2023-08-25T08:58:27.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 1 -- 172.21.15.99:0/2374441130 wait complete. 2023-08-25T08:58:27.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.092+0000 7fd34b7a1b80 20 client.14946 trim_cache size 0 max 16384 2023-08-25T08:58:27.102 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T08:58:27.102 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:58:27.102 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2023-08-25T08:58:27.102 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24547, 2023-08-25T08:58:27.102 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-08-25T08:58:27.102 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-08-25T08:58:27.102 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-08-25T08:58:27.103 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 64.040223948999994, 2023-08-25T08:58:27.103 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2023-08-25T08:58:27.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2023-08-25T08:58:27.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2023-08-25T08:58:27.103 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 82.648114110999998 2023-08-25T08:58:27.103 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T08:58:27.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 -- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd31c09ffe0 msgr2=0x7fd31c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd31c09ffe0 0x7fd31c0a24a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd328009fd0 tx=0x7fd32800b040 comp rx=0 tx=0).stop 2023-08-25T08:58:27.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 -- 172.21.15.99:0/477203065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 msgr2=0x7fd3300746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 0x7fd3300746d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0058e0 tx=0x7fd32c00f690 comp rx=0 tx=0).stop 2023-08-25T08:58:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 -- 172.21.15.99:0/477203065 shutdown_connections 2023-08-25T08:58:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3301024c0 0x7fd330073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd31c09ffe0 0x7fd31c0a24a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd330100930 0x7fd3300746d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 --2- 172.21.15.99:0/477203065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd330101b20 0x7fd330072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 -- 172.21.15.99:0/477203065 >> 172.21.15.99:0/477203065 conn(0x7fd3300fbea0 msgr2=0x7fd3300fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 -- 172.21.15.99:0/477203065 shutdown_connections 2023-08-25T08:58:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.093+0000 7fd337dff700 1 -- 172.21.15.99:0/477203065 wait complete. 2023-08-25T08:58:27.112 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-25T08:58:27.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 -- 172.21.15.99:0/693658378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb8101990 msgr2=0x7facb8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 --2- 172.21.15.99:0/693658378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb8101990 0x7facb8101df0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faca80099c0 tx=0x7faca8009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 -- 172.21.15.99:0/693658378 shutdown_connections 2023-08-25T08:58:27.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 --2- 172.21.15.99:0/693658378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facb8102330 0x7facb8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 --2- 172.21.15.99:0/693658378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb81007a0 0x7facb8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 --2- 172.21.15.99:0/693658378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb8101990 0x7facb8101df0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 -- 172.21.15.99:0/693658378 >> 172.21.15.99:0/693658378 conn(0x7facb80fbd30 msgr2=0x7facb80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 -- 172.21.15.99:0/693658378 shutdown_connections 2023-08-25T08:58:27.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.309+0000 7facbe59d700 1 -- 172.21.15.99:0/693658378 wait complete. 2023-08-25T08:58:27.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.310+0000 7facbe59d700 1 Processor -- start 2023-08-25T08:58:27.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.310+0000 7facbe59d700 1 -- start start 2023-08-25T08:58:27.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.310+0000 7facbe59d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb81007a0 0x7facb8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbe59d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb8101990 0x7facb8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbe59d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facb8102330 0x7facb819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbe59d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb81a00f0 con 0x7facb81007a0 2023-08-25T08:58:27.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbe59d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb81a0230 con 0x7facb8101990 2023-08-25T08:58:27.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbe59d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb81a03c0 con 0x7facb8102330 2023-08-25T08:58:27.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb81007a0 0x7facb8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb81007a0 0x7facb8198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52040/0 (socket says 172.21.15.99:52040) 2023-08-25T08:58:27.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 -- 172.21.15.99:0/3630886979 learned_addr learned my addr 172.21.15.99:0/3630886979 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbdd9c700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facb8102330 0x7facb819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbcd9a700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb8101990 0x7facb8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 -- 172.21.15.99:0/3630886979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facb8102330 msgr2=0x7facb819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facb8102330 0x7facb819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 -- 172.21.15.99:0/3630886979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb8101990 msgr2=0x7facb8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb8101990 0x7facb8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 -- 172.21.15.99:0/3630886979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca80095f0 con 0x7facb81007a0 2023-08-25T08:58:27.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.311+0000 7facbd59b700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb81007a0 0x7facb8198570 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7facb400d980 tx=0x7facb400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.312+0000 7facae7fc700 1 -- 172.21.15.99:0/3630886979 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb400b950 con 0x7facb81007a0 2023-08-25T08:58:27.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.312+0000 7facae7fc700 1 -- 172.21.15.99:0/3630886979 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb4007620 con 0x7facb81007a0 2023-08-25T08:58:27.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.312+0000 7facbe59d700 1 -- 172.21.15.99:0/3630886979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb81a06a0 con 0x7facb81007a0 2023-08-25T08:58:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.312+0000 7facae7fc700 1 -- 172.21.15.99:0/3630886979 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb400f3c0 con 0x7facb81007a0 2023-08-25T08:58:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.312+0000 7facbe59d700 1 -- 172.21.15.99:0/3630886979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb81a0c00 con 0x7facb81007a0 2023-08-25T08:58:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.315+0000 7facae7fc700 1 -- 172.21.15.99:0/3630886979 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7facb4010640 con 0x7facb81007a0 2023-08-25T08:58:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.316+0000 7facd3155b80 1 Processor -- start 2023-08-25T08:58:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.317+0000 7facd3155b80 1 -- start start 2023-08-25T08:58:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.317+0000 7facd3155b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d3980 0x55abca3d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.317+0000 7facd3155b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3d4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.317+0000 7facd3155b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.317+0000 7facd3155b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca33b760 con 0x55abca3d42e0 2023-08-25T08:58:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.317+0000 7facd3155b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca32c730 con 0x55abca3d3980 2023-08-25T08:58:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.317+0000 7facd3155b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca32a050 con 0x55abca3d4f00 2023-08-25T08:58:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.317+0000 7facbd59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3d4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbdd9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d3980 0x55abca3d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbdd9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d3980 0x55abca3d3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52304/0 (socket says 172.21.15.99:52304) 2023-08-25T08:58:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbdd9c700 1 -- 172.21.15.99:0/2063633278 learned_addr learned my addr 172.21.15.99:0/2063633278 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbcd9a700 1 --2- 172.21.15.99:0/2063633278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facae7fc700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faca40a0110 0x7faca40a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbcd9a700 1 -- 172.21.15.99:0/2063633278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d3980 msgr2=0x55abca3d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbcd9a700 1 --2- 172.21.15.99:0/2063633278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d3980 0x55abca3d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbcd9a700 1 -- 172.21.15.99:0/2063633278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 msgr2=0x55abca3d4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbcd9a700 1 --2- 172.21.15.99:0/2063633278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3d4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbcd9a700 1 -- 172.21.15.99:0/2063633278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abca1a55e0 con 0x55abca3d4f00 2023-08-25T08:58:27.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbdd9c700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faca40a0110 0x7faca40a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facae7fc700 1 -- 172.21.15.99:0/3630886979 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7facb40cb290 con 0x7facb81007a0 2023-08-25T08:58:27.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.318+0000 7facbcd9a700 1 --2- 172.21.15.99:0/2063633278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3d5360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faca80098e0 tx=0x7faca8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c23c07fadd72d5a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facbdd9c700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faca40a0110 0x7faca40a25d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7facb00075c0 tx=0x7facb0009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7faca37fe700 1 -- 172.21.15.99:0/2063633278 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca801d040 con 0x55abca3d4f00 2023-08-25T08:58:27.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7faca37fe700 1 -- 172.21.15.99:0/2063633278 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faca8005070 con 0x55abca3d4f00 2023-08-25T08:58:27.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7faca37fe700 1 -- 172.21.15.99:0/2063633278 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca8003ac0 con 0x55abca3d4f00 2023-08-25T08:58:27.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 -- 172.21.15.99:0/2063633278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 msgr2=0x55abca3d5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 --2- 172.21.15.99:0/2063633278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3d5360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faca80098e0 tx=0x7faca8005e80 comp rx=0 tx=0).stop 2023-08-25T08:58:27.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 -- 172.21.15.99:0/2063633278 shutdown_connections 2023-08-25T08:58:27.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 --2- 172.21.15.99:0/2063633278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3d5360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 --2- 172.21.15.99:0/2063633278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d3980 0x55abca3d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 --2- 172.21.15.99:0/2063633278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 -- 172.21.15.99:0/2063633278 >> 172.21.15.99:0/2063633278 conn(0x55abca348090 msgr2=0x55abca3ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 -- 172.21.15.99:0/2063633278 shutdown_connections 2023-08-25T08:58:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.319+0000 7facd3155b80 1 -- 172.21.15.99:0/2063633278 wait complete. 2023-08-25T08:58:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.320+0000 7facd3155b80 1 -- start start 2023-08-25T08:58:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.320+0000 7facd3155b80 1 Processor -- start 2023-08-25T08:58:27.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facd3155b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d3980 0x55abca3dd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facd3155b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3ddab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facd3155b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facd3155b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca325940 con 0x55abca3d3980 2023-08-25T08:58:27.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facd3155b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca2c3b60 con 0x55abca3d42e0 2023-08-25T08:58:27.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facd3155b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca2c3860 con 0x55abca3d4f00 2023-08-25T08:58:27.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facd3155b80 10 client.0 _ll_register_callbacks cb 0x55abca20da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:27.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d3980 0x55abca3dd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d3980 0x55abca3dd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52058/0 (socket says 172.21.15.99:52058) 2023-08-25T08:58:27.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 -- 172.21.15.99:0/1345066343 learned_addr learned my addr 172.21.15.99:0/1345066343 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbdd9c700 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbcd9a700 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3ddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 -- 172.21.15.99:0/1345066343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 msgr2=0x55abca3e4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3e4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 -- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 msgr2=0x55abca3ddab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3ddab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 -- 172.21.15.99:0/1345066343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb8109530 con 0x55abca3d3980 2023-08-25T08:58:27.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbcd9a700 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:27.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7facbd59b700 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d3980 0x55abca3dd570 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7facb400b540 tx=0x7facb40986c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7bcd7ee9f2271a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.321+0000 7faca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:58:27.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7faca0ff9700 1 -- 172.21.15.99:0/1345066343 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4099200 con 0x55abca3d3980 2023-08-25T08:58:27.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 1 -- 172.21.15.99:0/1345066343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abca3a3170 con 0x55abca3d3980 2023-08-25T08:58:27.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7faca0ff9700 1 -- 172.21.15.99:0/1345066343 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7facb40a0910 con 0x55abca3d3980 2023-08-25T08:58:27.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7faca0ff9700 1 -- 172.21.15.99:0/1345066343 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb4098b00 con 0x55abca3d3980 2023-08-25T08:58:27.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 10 client.14958 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:27.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 1 -- 172.21.15.99:0/1345066343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55abca151d90 con 0x55abca3d3980 2023-08-25T08:58:27.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7faca0ff9700 1 -- 172.21.15.99:0/1345066343 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abca3a3170 con 0x55abca3d3980 2023-08-25T08:58:27.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7faca0ff9700 1 -- 172.21.15.99:0/1345066343 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7facb4098d60 con 0x55abca3d3980 2023-08-25T08:58:27.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 10 client.14958 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:27.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 10 client.14958 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T08:58:27.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 20 client.14958 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:27.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55abca3e9170 0x55abca3e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 4 client.14958 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:58:27.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.322+0000 7facd3155b80 1 -- 172.21.15.99:0/1345066343 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abca2cbb50 con 0x55abca3e9170 2023-08-25T08:58:27.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.323+0000 7facbcd9a700 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55abca3e9170 0x55abca3e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.323+0000 7facbcd9a700 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55abca3e9170 0x55abca3e7bf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599689eeee1a8e94 server_cookie=e50cf657d7d342d in_seq=0 out_seq=0 2023-08-25T08:58:27.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.323+0000 7faca0ff9700 10 client.14958 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:58:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7faca0ff9700 1 -- 172.21.15.99:0/1345066343 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55abca2cbb50 con 0x55abca3e9170 2023-08-25T08:58:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7faca0ff9700 10 client.14958 handle_command_reply: tid=0 2023-08-25T08:58:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 client.14958 shutdown 2023-08-25T08:58:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 -- 172.21.15.99:0/1345066343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d3980 msgr2=0x55abca3dd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d3980 0x55abca3dd570 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7facb400b540 tx=0x7facb40986c0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 -- 172.21.15.99:0/1345066343 shutdown_connections 2023-08-25T08:58:27.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d4f00 0x55abca3e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d3980 0x55abca3dd570 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55abca3e9170 0x55abca3e7bf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 --2- 172.21.15.99:0/1345066343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7faca0ff9700 0 client.14958 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:58:27.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 -- 172.21.15.99:0/1345066343 >> 172.21.15.99:0/1345066343 conn(0x55abca348090 msgr2=0x55abca3d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 -- 172.21.15.99:0/1345066343 shutdown_connections 2023-08-25T08:58:27.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.324+0000 7facd3155b80 1 -- 172.21.15.99:0/1345066343 wait complete. 2023-08-25T08:58:27.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.325+0000 7facd3155b80 20 client.14958 trim_cache size 0 max 16384 2023-08-25T08:58:27.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.342+0000 7facd3155b80 1 Processor -- start 2023-08-25T08:58:27.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.342+0000 7facd3155b80 1 -- start start 2023-08-25T08:58:27.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facd3155b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3dcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facd3155b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facd3155b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d4f00 0x55abca3d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facd3155b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca1a78d0 con 0x55abca3d4f00 2023-08-25T08:58:27.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facd3155b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca2e1f00 con 0x55abca3d42e0 2023-08-25T08:58:27.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facd3155b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca2bfe00 con 0x55abca3d3980 2023-08-25T08:58:27.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbdd9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d4f00 0x55abca3d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbdd9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d4f00 0x55abca3d80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52062/0 (socket says 172.21.15.99:52062) 2023-08-25T08:58:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbdd9c700 1 -- 172.21.15.99:0/1927872456 learned_addr learned my addr 172.21.15.99:0/1927872456 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbd59b700 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3dcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbcd9a700 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbdd9c700 1 -- 172.21.15.99:0/1927872456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 msgr2=0x55abca3dcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbdd9c700 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3dcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbdd9c700 1 -- 172.21.15.99:0/1927872456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 msgr2=0x55abca3e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbdd9c700 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbdd9c700 1 -- 172.21.15.99:0/1927872456 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abca13ac10 con 0x55abca3d4f00 2023-08-25T08:58:27.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.343+0000 7facbd59b700 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3dcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:27.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7facbdd9c700 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d4f00 0x55abca3d80d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7facb0017930 tx=0x7facb0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e6fdb1bc9ae9d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7faca3fff700 1 -- 172.21.15.99:0/1927872456 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0028070 con 0x55abca3d4f00 2023-08-25T08:58:27.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7faca3fff700 1 -- 172.21.15.99:0/1927872456 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb000f040 con 0x55abca3d4f00 2023-08-25T08:58:27.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7faca3fff700 1 -- 172.21.15.99:0/1927872456 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb000e040 con 0x55abca3d4f00 2023-08-25T08:58:27.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7facd3155b80 1 -- 172.21.15.99:0/1927872456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d4f00 msgr2=0x55abca3d80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7facd3155b80 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d4f00 0x55abca3d80d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7facb0017930 tx=0x7facb0004b40 comp rx=0 tx=0).stop 2023-08-25T08:58:27.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7facd3155b80 1 -- 172.21.15.99:0/1927872456 shutdown_connections 2023-08-25T08:58:27.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7facd3155b80 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d4f00 0x55abca3d80d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7facd3155b80 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3dcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7facd3155b80 1 --2- 172.21.15.99:0/1927872456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d42e0 0x55abca3e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.344+0000 7facd3155b80 1 -- 172.21.15.99:0/1927872456 >> 172.21.15.99:0/1927872456 conn(0x55abca348090 msgr2=0x55abca3dc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.345+0000 7facd3155b80 1 -- 172.21.15.99:0/1927872456 shutdown_connections 2023-08-25T08:58:27.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.345+0000 7facd3155b80 1 -- 172.21.15.99:0/1927872456 wait complete. 2023-08-25T08:58:27.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.345+0000 7facd3155b80 1 -- start start 2023-08-25T08:58:27.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.345+0000 7facd3155b80 1 Processor -- start 2023-08-25T08:58:27.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.345+0000 7facd3155b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facd3155b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facd3155b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d4f00 0x55abca3e2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facd3155b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca43cb90 con 0x55abca3d42e0 2023-08-25T08:58:27.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facd3155b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca43cd00 con 0x55abca3d4f00 2023-08-25T08:58:27.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facd3155b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abca43cec0 con 0x55abca3d3980 2023-08-25T08:58:27.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facd3155b80 10 client.0 _ll_register_callbacks cb 0x55abca20da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:27.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3e1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52078/0 (socket says 172.21.15.99:52078) 2023-08-25T08:58:27.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 -- 172.21.15.99:0/1300718292 learned_addr learned my addr 172.21.15.99:0/1300718292 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:27.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbdd9c700 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d4f00 0x55abca3e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbd59b700 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 -- 172.21.15.99:0/1300718292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 msgr2=0x55abca3e4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3e4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 -- 172.21.15.99:0/1300718292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d4f00 msgr2=0x55abca3e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:27.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d4f00 0x55abca3e2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 -- 172.21.15.99:0/1300718292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abca151d90 con 0x55abca3d42e0 2023-08-25T08:58:27.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbd59b700 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:58:27.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7facbcd9a700 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3e1c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faca800b580 tx=0x7faca80094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52af25b206ec56dd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:27.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7faca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:58:27.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.346+0000 7faca0ff9700 1 -- 172.21.15.99:0/1300718292 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca801d040 con 0x55abca3d42e0 2023-08-25T08:58:27.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 1 -- 172.21.15.99:0/1300718292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abca3c6860 con 0x55abca3d42e0 2023-08-25T08:58:27.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7faca0ff9700 1 -- 172.21.15.99:0/1300718292 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faca800bd10 con 0x55abca3d42e0 2023-08-25T08:58:27.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7faca0ff9700 1 -- 172.21.15.99:0/1300718292 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faca8030150 con 0x55abca3d42e0 2023-08-25T08:58:27.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 10 client.14964 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:27.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 1 -- 172.21.15.99:0/1300718292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7facb00175b0 con 0x55abca3d42e0 2023-08-25T08:58:27.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7faca0ff9700 1 -- 172.21.15.99:0/1300718292 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faca8040080 con 0x55abca3d42e0 2023-08-25T08:58:27.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7faca0ff9700 1 -- 172.21.15.99:0/1300718292 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faca803a030 con 0x55abca3d42e0 2023-08-25T08:58:27.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 10 client.14964 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:27.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 10 client.14964 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T08:58:27.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 20 client.14964 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:27.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55abca441cb0 0x55abca43e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:27.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 4 client.14964 mds_command: new command op to 24593 tid=0[{"prefix": "status"}] 2023-08-25T08:58:27.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.347+0000 7facd3155b80 1 -- 172.21.15.99:0/1300718292 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55abca31e920 con 0x55abca441cb0 2023-08-25T08:58:27.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.348+0000 7facbd59b700 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55abca441cb0 0x55abca43e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:27.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.348+0000 7faca0ff9700 10 client.14964 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:58:27.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.348+0000 7facbd59b700 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55abca441cb0 0x55abca43e110 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc65f987642dea3b server_cookie=3a937bde67056933 in_seq=0 out_seq=0 2023-08-25T08:58:27.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.348+0000 7faca0ff9700 1 -- 172.21.15.99:0/1300718292 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55abca31e920 con 0x55abca441cb0 2023-08-25T08:58:27.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.348+0000 7faca0ff9700 10 client.14964 handle_command_reply: tid=0 2023-08-25T08:58:27.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.348+0000 7facd3155b80 1 client.14964 shutdown 2023-08-25T08:58:27.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 -- 172.21.15.99:0/1300718292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 msgr2=0x55abca3e1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3e1c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faca800b580 tx=0x7faca80094b0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 -- 172.21.15.99:0/1300718292 shutdown_connections 2023-08-25T08:58:27.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55abca441cb0 0x55abca43e110 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55abca3d4f00 0x55abca3e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abca3d3980 0x55abca3e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 --2- 172.21.15.99:0/1300718292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abca3d42e0 0x55abca3e1c90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7faca0ff9700 0 client.14964 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:58:27.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 -- 172.21.15.99:0/1300718292 >> 172.21.15.99:0/1300718292 conn(0x55abca348090 msgr2=0x55abca3dc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 -- 172.21.15.99:0/1300718292 shutdown_connections 2023-08-25T08:58:27.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 1 -- 172.21.15.99:0/1300718292 wait complete. 2023-08-25T08:58:27.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facd3155b80 20 client.14964 trim_cache size 0 max 16384 2023-08-25T08:58:27.362 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T08:58:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:58:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-08-25T08:58:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24593, 2023-08-25T08:58:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-08-25T08:58:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-08-25T08:58:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-08-25T08:58:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 67.325127190000003, 2023-08-25T08:58:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2023-08-25T08:58:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2023-08-25T08:58:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2023-08-25T08:58:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 74.579340490000007 2023-08-25T08:58:27.366 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T08:58:27.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facbe59d700 1 -- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faca40a0110 msgr2=0x7faca40a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facbe59d700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faca40a0110 0x7faca40a25d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7facb00075c0 tx=0x7facb0009040 comp rx=0 tx=0).stop 2023-08-25T08:58:27.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facbe59d700 1 -- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb81007a0 msgr2=0x7facb8198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:27.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.349+0000 7facbe59d700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb81007a0 0x7facb8198570 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7facb400d980 tx=0x7facb400dc90 comp rx=0 tx=0).stop 2023-08-25T08:58:27.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.350+0000 7facbe59d700 1 -- 172.21.15.99:0/3630886979 shutdown_connections 2023-08-25T08:58:27.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.350+0000 7facbe59d700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facb8102330 0x7facb819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.350+0000 7facbe59d700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facb81007a0 0x7facb8198570 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.350+0000 7facbe59d700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7faca40a0110 0x7faca40a25d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.350+0000 7facbe59d700 1 --2- 172.21.15.99:0/3630886979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb8101990 0x7facb8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:27.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.350+0000 7facbe59d700 1 -- 172.21.15.99:0/3630886979 >> 172.21.15.99:0/3630886979 conn(0x7facb80fbd30 msgr2=0x7facb80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:27.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.350+0000 7facbe59d700 1 -- 172.21.15.99:0/3630886979 shutdown_connections 2023-08-25T08:58:27.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:27.350+0000 7facbe59d700 1 -- 172.21.15.99:0/3630886979 wait complete. 2023-08-25T08:58:27.370 INFO:tasks.cephadm:Setup complete, yielding 2023-08-25T08:58:27.371 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-08-25T08:58:27.384 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T08:58:27.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch status' 2023-08-25T08:58:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:27 smithi099 ceph-mon[97641]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:27 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/4201054525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:27 smithi157 ceph-mon[130879]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:27 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/4201054525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:27.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:27 smithi181 ceph-mon[131296]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:27.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:27 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/4201054525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:28.101 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:29.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 -- 172.21.15.99:0/3546322961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c102550 msgr2=0x7f322c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:29.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 --2- 172.21.15.99:0/3546322961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c102550 0x7f322c108e00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f322400b0d0 tx=0x7f322400b3e0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 -- 172.21.15.99:0/3546322961 shutdown_connections 2023-08-25T08:58:29.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 --2- 172.21.15.99:0/3546322961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c102550 0x7f322c108e00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 --2- 172.21.15.99:0/3546322961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f322c1009b0 0x7f322c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 --2- 172.21.15.99:0/3546322961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c101bb0 0x7f322c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 -- 172.21.15.99:0/3546322961 >> 172.21.15.99:0/3546322961 conn(0x7f322c0fbf10 msgr2=0x7f322c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:29.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 -- 172.21.15.99:0/3546322961 shutdown_connections 2023-08-25T08:58:29.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.637+0000 7f3231d9c700 1 -- 172.21.15.99:0/3546322961 wait complete. 2023-08-25T08:58:29.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.638+0000 7f3231d9c700 1 Processor -- start 2023-08-25T08:58:29.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f3231d9c700 1 -- start start 2023-08-25T08:58:29.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f3231d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f322c1009b0 0x7f322c196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:29.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f3231d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c101bb0 0x7f322c196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:29.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f3231d9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c102550 0x7f322c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:29.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f3231d9c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19e320 con 0x7f322c101bb0 2023-08-25T08:58:29.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f3231d9c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19e460 con 0x7f322c102550 2023-08-25T08:58:29.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f3231d9c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19e5f0 con 0x7f322c1009b0 2023-08-25T08:58:29.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f322affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c101bb0 0x7f322c196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:29.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f322affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c101bb0 0x7f322c196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52092/0 (socket says 172.21.15.99:52092) 2023-08-25T08:58:29.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f322affd700 1 -- 172.21.15.99:0/2764312446 learned_addr learned my addr 172.21.15.99:0/2764312446 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:29.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f322bfff700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c102550 0x7f322c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:29.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.639+0000 7f322b7fe700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f322c1009b0 0x7f322c196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:29.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f322affd700 1 -- 172.21.15.99:0/2764312446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f322c1009b0 msgr2=0x7f322c196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:29.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f322affd700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f322c1009b0 0x7f322c196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f322affd700 1 -- 172.21.15.99:0/2764312446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c102550 msgr2=0x7f322c19dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:29.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f322affd700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c102550 0x7f322c19dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f322affd700 1 -- 172.21.15.99:0/2764312446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224009d00 con 0x7f322c101bb0 2023-08-25T08:58:29.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f322bfff700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c102550 0x7f322c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:29.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f322affd700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c101bb0 0x7f322c196cd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9d0 tx=0x7f322000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:29.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f3228ff9700 1 -- 172.21.15.99:0/2764312446 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220009790 con 0x7f322c101bb0 2023-08-25T08:58:29.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f3228ff9700 1 -- 172.21.15.99:0/2764312446 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f322000ee70 con 0x7f322c101bb0 2023-08-25T08:58:29.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c19e8d0 con 0x7f322c101bb0 2023-08-25T08:58:29.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f3228ff9700 1 -- 172.21.15.99:0/2764312446 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220010ea0 con 0x7f322c101bb0 2023-08-25T08:58:29.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.640+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c19ee20 con 0x7f322c101bb0 2023-08-25T08:58:29.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.641+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c107ef0 con 0x7f322c101bb0 2023-08-25T08:58:29.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.643+0000 7f3228ff9700 1 -- 172.21.15.99:0/2764312446 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f322000cbd0 con 0x7f322c101bb0 2023-08-25T08:58:29.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.645+0000 7f3228ff9700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32140a0060 0x7f32140a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:29.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.645+0000 7f3228ff9700 1 -- 172.21.15.99:0/2764312446 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3220014070 con 0x7f322c101bb0 2023-08-25T08:58:29.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.645+0000 7f322b7fe700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32140a0060 0x7f32140a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:29.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.645+0000 7f322b7fe700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32140a0060 0x7f32140a2520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f322c101a10 tx=0x7f321c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:29.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.649+0000 7f3228ff9700 1 -- 172.21.15.99:0/2764312446 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3220051e00 con 0x7f322c101bb0 2023-08-25T08:58:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:29 smithi099 ceph-mon[97641]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.864+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f322c02cdc0 con 0x7f32140a0060 2023-08-25T08:58:29.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.866+0000 7f3228ff9700 1 -- 172.21.15.99:0/2764312446 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f322c02cdc0 con 0x7f32140a0060 2023-08-25T08:58:29.867 INFO:teuthology.orchestra.run.smithi099.stdout:Backend: cephadm 2023-08-25T08:58:29.867 INFO:teuthology.orchestra.run.smithi099.stdout:Available: Yes 2023-08-25T08:58:29.867 INFO:teuthology.orchestra.run.smithi099.stdout:Paused: No 2023-08-25T08:58:29.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32140a0060 msgr2=0x7f32140a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:29.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32140a0060 0x7f32140a2520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f322c101a10 tx=0x7f321c009260 comp rx=0 tx=0).stop 2023-08-25T08:58:29.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c101bb0 msgr2=0x7f322c196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:29.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c101bb0 0x7f322c196cd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9d0 tx=0x7f322000ece0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 shutdown_connections 2023-08-25T08:58:29.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32140a0060 0x7f32140a2520 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c102550 0x7f322c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f322c1009b0 0x7f322c196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 --2- 172.21.15.99:0/2764312446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c101bb0 0x7f322c196cd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:29.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 >> 172.21.15.99:0/2764312446 conn(0x7f322c0fbf10 msgr2=0x7f322c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:29.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 shutdown_connections 2023-08-25T08:58:29.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:29.868+0000 7f3231d9c700 1 -- 172.21.15.99:0/2764312446 wait complete. 2023-08-25T08:58:29.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:29 smithi157 ceph-mon[130879]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:29.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:29 smithi181 ceph-mon[131296]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:30.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch ps' 2023-08-25T08:58:30.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:30 smithi157 ceph-mon[130879]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:30.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:30 smithi181 ceph-mon[131296]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:31.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:30 smithi099 ceph-mon[97641]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:31.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:31 smithi099 ceph-mon[97641]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:31 smithi157 ceph-mon[130879]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:31.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:31 smithi181 ceph-mon[131296]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:32.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 -- 172.21.15.99:0/71207764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8101bb0 msgr2=0x7f32f8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:32.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 --2- 172.21.15.99:0/71207764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8101bb0 0x7f32f8102010 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f32e80099c0 tx=0x7f32e8009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 -- 172.21.15.99:0/71207764 shutdown_connections 2023-08-25T08:58:32.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 --2- 172.21.15.99:0/71207764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f8102550 0x7f32f8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 --2- 172.21.15.99:0/71207764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32f81009b0 0x7f32f8100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 --2- 172.21.15.99:0/71207764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8101bb0 0x7f32f8102010 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 -- 172.21.15.99:0/71207764 >> 172.21.15.99:0/71207764 conn(0x7f32f80fbf10 msgr2=0x7f32f80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:32.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 -- 172.21.15.99:0/71207764 shutdown_connections 2023-08-25T08:58:32.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.501+0000 7f3300446700 1 -- 172.21.15.99:0/71207764 wait complete. 2023-08-25T08:58:32.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.502+0000 7f3300446700 1 Processor -- start 2023-08-25T08:58:32.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.502+0000 7f3300446700 1 -- start start 2023-08-25T08:58:32.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f3300446700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f81009b0 0x7f32f80746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:32.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f3300446700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32f8101bb0 0x7f32f8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:32.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f3300446700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8102550 0x7f32f8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:32.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f3300446700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80739d0 con 0x7f32f8102550 2023-08-25T08:58:32.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f3300446700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8073b40 con 0x7f32f8101bb0 2023-08-25T08:58:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f3300446700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f81a0710 con 0x7f32f81009b0 2023-08-25T08:58:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f32fe9e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8102550 0x7f32f8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f32fe9e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8102550 0x7f32f8073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52096/0 (socket says 172.21.15.99:52096) 2023-08-25T08:58:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f32fe9e3700 1 -- 172.21.15.99:0/3660176645 learned_addr learned my addr 172.21.15.99:0/3660176645 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f32fd9e1700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32f8101bb0 0x7f32f8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.503+0000 7f32fe1e2700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f81009b0 0x7f32f80746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32fe9e3700 1 -- 172.21.15.99:0/3660176645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f81009b0 msgr2=0x7f32f80746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32fe9e3700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f81009b0 0x7f32f80746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32fe9e3700 1 -- 172.21.15.99:0/3660176645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32f8101bb0 msgr2=0x7f32f8072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:32.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32fe9e3700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32f8101bb0 0x7f32f8072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32fe9e3700 1 -- 172.21.15.99:0/3660176645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e80095f0 con 0x7f32f8102550 2023-08-25T08:58:32.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32fe9e3700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8102550 0x7f32f8073250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f32f000ef30 tx=0x7f32f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:32.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32ef7fe700 1 -- 172.21.15.99:0/3660176645 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0018070 con 0x7f32f8102550 2023-08-25T08:58:32.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32ef7fe700 1 -- 172.21.15.99:0/3660176645 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32f000f040 con 0x7f32f8102550 2023-08-25T08:58:32.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f81a0990 con 0x7f32f8102550 2023-08-25T08:58:32.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f32ef7fe700 1 -- 172.21.15.99:0/3660176645 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000cd00 con 0x7f32f8102550 2023-08-25T08:58:32.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.504+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f81a0e90 con 0x7f32f8102550 2023-08-25T08:58:32.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.505+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f81078f0 con 0x7f32f8102550 2023-08-25T08:58:32.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.507+0000 7f32ef7fe700 1 -- 172.21.15.99:0/3660176645 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f32f0003a00 con 0x7f32f8102550 2023-08-25T08:58:32.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.508+0000 7f32ef7fe700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32e409ffe0 0x7f32e40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:32.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.508+0000 7f32ef7fe700 1 -- 172.21.15.99:0/3660176645 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f32f00caf30 con 0x7f32f8102550 2023-08-25T08:58:32.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.509+0000 7f32fe1e2700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32e409ffe0 0x7f32e40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:32.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.509+0000 7f32fe1e2700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32e409ffe0 0x7f32e40a24a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f32f8101a10 tx=0x7f32f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:32.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.512+0000 7f32ef7fe700 1 -- 172.21.15.99:0/3660176645 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f32f0094b70 con 0x7f32f8102550 2023-08-25T08:58:32.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.722+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f32f802cde0 con 0x7f32e409ffe0 2023-08-25T08:58:32.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.735+0000 7f32ef7fe700 1 -- 172.21.15.99:0/3660176645 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f32f802cde0 con 0x7f32e409ffe0 2023-08-25T08:58:32.736 INFO:teuthology.orchestra.run.smithi099.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-08-25T08:58:32.736 INFO:teuthology.orchestra.run.smithi099.stdout:mds.a smithi099 running (82s) - 82s 13.7M - 18.0.0-5703-gba6b39e4 95c9c69e1344 5e8997ffbf0c 2023-08-25T08:58:32.736 INFO:teuthology.orchestra.run.smithi099.stdout:mds.b smithi157 running (80s) - 79s 32.2M - 18.0.0-5703-gba6b39e4 95c9c69e1344 2d1d798cc847 2023-08-25T08:58:32.737 INFO:teuthology.orchestra.run.smithi099.stdout:mds.c smithi181 running (93s) - 92s 19.4M - 18.0.0-5703-gba6b39e4 95c9c69e1344 f341dae8b111 2023-08-25T08:58:32.737 INFO:teuthology.orchestra.run.smithi099.stdout:mds.d smithi099 running (99s) - 99s 13.7M - 18.0.0-5703-gba6b39e4 95c9c69e1344 aa62f45c0b6c 2023-08-25T08:58:32.737 INFO:teuthology.orchestra.run.smithi099.stdout:mds.e smithi157 running (97s) - 97s 19.4M - 18.0.0-5703-gba6b39e4 95c9c69e1344 458f75894c41 2023-08-25T08:58:32.737 INFO:teuthology.orchestra.run.smithi099.stdout:mds.f smithi181 running (95s) - 94s 21.2M - 18.0.0-5703-gba6b39e4 95c9c69e1344 1da113e7caa0 2023-08-25T08:58:32.737 INFO:teuthology.orchestra.run.smithi099.stdout:mds.g smithi099 running (85s) - 85s 13.7M - 18.0.0-5703-gba6b39e4 95c9c69e1344 4fccd29d3f22 2023-08-25T08:58:32.737 INFO:teuthology.orchestra.run.smithi099.stdout:mds.h smithi157 running (90s) - 90s 19.4M - 18.0.0-5703-gba6b39e4 95c9c69e1344 20a140486cbe 2023-08-25T08:58:32.738 INFO:teuthology.orchestra.run.smithi099.stdout:mds.i smithi181 running (88s) - 88s 20.7M - 18.0.0-5703-gba6b39e4 95c9c69e1344 1e5028919808 2023-08-25T08:58:32.738 INFO:teuthology.orchestra.run.smithi099.stdout:mds.j smithi099 running (102s) - 102s 13.8M - 18.0.0-5703-gba6b39e4 95c9c69e1344 370be917da72 2023-08-25T08:58:32.738 INFO:teuthology.orchestra.run.smithi099.stdout:mds.k smithi157 running (106s) - 105s 16.5M - 18.0.0-5703-gba6b39e4 95c9c69e1344 fbf0b74d2a22 2023-08-25T08:58:32.738 INFO:teuthology.orchestra.run.smithi099.stdout:mds.l smithi181 running (108s) - 107s 17.4M - 18.0.0-5703-gba6b39e4 95c9c69e1344 3deff3cf2ff4 2023-08-25T08:58:32.738 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.x smithi099 *:9283,8765 running (8m) - 8m 525M - 18.0.0-5703-gba6b39e4 95c9c69e1344 50bd29f18fb2 2023-08-25T08:58:32.738 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.y smithi157 *:8443,8765 running (6m) - 6m 449M - 18.0.0-5703-gba6b39e4 95c9c69e1344 c131e149a663 2023-08-25T08:58:32.738 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.z smithi181 *:8443,8765 running (6m) - 6m 452M - 18.0.0-5703-gba6b39e4 95c9c69e1344 cff29435c3d9 2023-08-25T08:58:32.739 INFO:teuthology.orchestra.run.smithi099.stdout:mon.a smithi099 running (8m) - 8m 92.3M 2048M 18.0.0-5703-gba6b39e4 95c9c69e1344 eeef5ed71b05 2023-08-25T08:58:32.739 INFO:teuthology.orchestra.run.smithi099.stdout:mon.b smithi157 running (6m) - 6m 62.6M 2048M 18.0.0-5703-gba6b39e4 95c9c69e1344 c9f6ce89376f 2023-08-25T08:58:32.739 INFO:teuthology.orchestra.run.smithi099.stdout:mon.c smithi181 running (6m) - 6m 65.8M 2048M 18.0.0-5703-gba6b39e4 95c9c69e1344 8c6cb3773acd 2023-08-25T08:58:32.739 INFO:teuthology.orchestra.run.smithi099.stdout:osd.0 smithi099 running (6m) - 6m 67.0M 4316M 18.0.0-5703-gba6b39e4 95c9c69e1344 67e23f4f026e 2023-08-25T08:58:32.739 INFO:teuthology.orchestra.run.smithi099.stdout:osd.1 smithi157 running (5m) - 5m 77.2M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 015ff3125d87 2023-08-25T08:58:32.739 INFO:teuthology.orchestra.run.smithi099.stdout:osd.2 smithi181 running (5m) - 5m 74.0M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 ceff3925d1a9 2023-08-25T08:58:32.739 INFO:teuthology.orchestra.run.smithi099.stdout:osd.3 smithi099 running (5m) - 5m 61.8M 4316M 18.0.0-5703-gba6b39e4 95c9c69e1344 f63b2255c689 2023-08-25T08:58:32.740 INFO:teuthology.orchestra.run.smithi099.stdout:osd.4 smithi157 running (4m) - 4m 60.7M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 06601a9db029 2023-08-25T08:58:32.740 INFO:teuthology.orchestra.run.smithi099.stdout:osd.5 smithi181 running (4m) - 4m 64.7M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 0b789570b401 2023-08-25T08:58:32.740 INFO:teuthology.orchestra.run.smithi099.stdout:osd.6 smithi099 running (3m) - 3m 58.5M 4316M 18.0.0-5703-gba6b39e4 95c9c69e1344 75ae79b00111 2023-08-25T08:58:32.740 INFO:teuthology.orchestra.run.smithi099.stdout:osd.7 smithi157 running (3m) - 3m 60.9M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 1960d0d2af8f 2023-08-25T08:58:32.740 INFO:teuthology.orchestra.run.smithi099.stdout:osd.8 smithi181 running (3m) - 3m 60.2M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 8aac95d928f9 2023-08-25T08:58:32.740 INFO:teuthology.orchestra.run.smithi099.stdout:osd.9 smithi099 running (2m) - 2m 58.2M 4316M 18.0.0-5703-gba6b39e4 95c9c69e1344 c846e946eb69 2023-08-25T08:58:32.740 INFO:teuthology.orchestra.run.smithi099.stdout:osd.10 smithi157 running (2m) - 2m 64.4M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 c96e85fb9ca0 2023-08-25T08:58:32.741 INFO:teuthology.orchestra.run.smithi099.stdout:osd.11 smithi181 running (2m) - 2m 51.3M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 1750052bb80c 2023-08-25T08:58:32.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32e409ffe0 msgr2=0x7f32e40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:32.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32e409ffe0 0x7f32e40a24a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f32f8101a10 tx=0x7f32f4009260 comp rx=0 tx=0).stop 2023-08-25T08:58:32.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8102550 msgr2=0x7f32f8073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:32.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8102550 0x7f32f8073250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f32f000ef30 tx=0x7f32f000c3a0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 shutdown_connections 2023-08-25T08:58:32.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32f8102550 0x7f32f8073250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f81009b0 0x7f32f80746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f32e409ffe0 0x7f32e40a24a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 --2- 172.21.15.99:0/3660176645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32f8101bb0 0x7f32f8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:32.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 >> 172.21.15.99:0/3660176645 conn(0x7f32f80fbf10 msgr2=0x7f32f80fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:32.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 shutdown_connections 2023-08-25T08:58:32.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:32.737+0000 7f3300446700 1 -- 172.21.15.99:0/3660176645 wait complete. 2023-08-25T08:58:33.273 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch ls' 2023-08-25T08:58:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:33 smithi099 ceph-mon[97641]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:33 smithi099 ceph-mon[97641]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:33 smithi157 ceph-mon[130879]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:33 smithi157 ceph-mon[130879]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:33.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:33 smithi181 ceph-mon[131296]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:33.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:33 smithi181 ceph-mon[131296]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:34.152 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:35.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:35 smithi099 ceph-mon[97641]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:35 smithi157 ceph-mon[130879]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:35 smithi181 ceph-mon[131296]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:35.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.676+0000 7f744745a700 1 -- 172.21.15.99:0/2965114069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440073380 msgr2=0x7f7440073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:35.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.676+0000 7f744745a700 1 --2- 172.21.15.99:0/2965114069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440073380 0x7f7440073800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7430009990 tx=0x7f7430009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.676+0000 7f744745a700 1 -- 172.21.15.99:0/2965114069 shutdown_connections 2023-08-25T08:58:35.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.676+0000 7f744745a700 1 --2- 172.21.15.99:0/2965114069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7440108880 0x7f744010ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.676+0000 7f744745a700 1 --2- 172.21.15.99:0/2965114069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7440074950 0x7f7440072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.676+0000 7f744745a700 1 --2- 172.21.15.99:0/2965114069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440073380 0x7f7440073800 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.676+0000 7f744745a700 1 -- 172.21.15.99:0/2965114069 >> 172.21.15.99:0/2965114069 conn(0x7f74400fbf10 msgr2=0x7f74400fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:35.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.676+0000 7f744745a700 1 -- 172.21.15.99:0/2965114069 shutdown_connections 2023-08-25T08:58:35.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.677+0000 7f744745a700 1 -- 172.21.15.99:0/2965114069 wait complete. 2023-08-25T08:58:35.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f744745a700 1 Processor -- start 2023-08-25T08:58:35.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f744745a700 1 -- start start 2023-08-25T08:58:35.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f744745a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7440073380 0x7f7440198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f744745a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440074950 0x7f7440198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f744745a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7440108880 0x7f744019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f744745a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74401a04b0 con 0x7f7440074950 2023-08-25T08:58:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f744745a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74401a05f0 con 0x7f7440073380 2023-08-25T08:58:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f744745a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74401a0780 con 0x7f7440108880 2023-08-25T08:58:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.678+0000 7f74449f5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440074950 0x7f7440198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74451f6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7440073380 0x7f7440198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74451f6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7440073380 0x7f7440198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52380/0 (socket says 172.21.15.99:52380) 2023-08-25T08:58:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74449f5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440074950 0x7f7440198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52114/0 (socket says 172.21.15.99:52114) 2023-08-25T08:58:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74451f6700 1 -- 172.21.15.99:0/922021450 learned_addr learned my addr 172.21.15.99:0/922021450 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74459f7700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7440108880 0x7f744019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:35.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74449f5700 1 -- 172.21.15.99:0/922021450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7440108880 msgr2=0x7f744019fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:35.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74449f5700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7440108880 0x7f744019fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74449f5700 1 -- 172.21.15.99:0/922021450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7440073380 msgr2=0x7f7440198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:35.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74449f5700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7440073380 0x7f7440198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74449f5700 1 -- 172.21.15.99:0/922021450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74300095f0 con 0x7f7440074950 2023-08-25T08:58:35.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74459f7700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7440108880 0x7f744019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:35.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74451f6700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7440073380 0x7f7440198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:35.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74449f5700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440074950 0x7f7440198e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f743000b680 tx=0x7f7430004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:35.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74367fc700 1 -- 172.21.15.99:0/922021450 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743001d070 con 0x7f7440074950 2023-08-25T08:58:35.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74367fc700 1 -- 172.21.15.99:0/922021450 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74300054f0 con 0x7f7440074950 2023-08-25T08:58:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74401a0a00 con 0x7f7440074950 2023-08-25T08:58:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.679+0000 7f74367fc700 1 -- 172.21.15.99:0/922021450 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430004c60 con 0x7f7440074950 2023-08-25T08:58:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.680+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74401a0e10 con 0x7f7440074950 2023-08-25T08:58:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.681+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7440106f30 con 0x7f7440074950 2023-08-25T08:58:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.682+0000 7f74367fc700 1 -- 172.21.15.99:0/922021450 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f743000fb80 con 0x7f7440074950 2023-08-25T08:58:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.684+0000 7f74367fc700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f742c0a0060 0x7f742c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.684+0000 7f74367fc700 1 -- 172.21.15.99:0/922021450 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f74300cc5e0 con 0x7f7440074950 2023-08-25T08:58:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.684+0000 7f74451f6700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f742c0a0060 0x7f742c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:35.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.684+0000 7f74451f6700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f742c0a0060 0x7f742c0a2520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f743c005950 tx=0x7f743c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:35.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.688+0000 7f74367fc700 1 -- 172.21.15.99:0/922021450 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7430096220 con 0x7f7440074950 2023-08-25T08:58:35.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.894+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f744010e420 con 0x7f742c0a0060 2023-08-25T08:58:35.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.899+0000 7f74367fc700 1 -- 172.21.15.99:0/922021450 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f744010e420 con 0x7f742c0a0060 2023-08-25T08:58:35.900 INFO:teuthology.orchestra.run.smithi099.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-08-25T08:58:35.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.all 12/12 - 113s smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi157=b;smithi157=e;smithi157=h;smithi157=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-08-25T08:58:35.900 INFO:teuthology.orchestra.run.smithi099.stdout:mgr 3/3 - 6m smithi099=x;smithi157=y;smithi181=z;count:3 2023-08-25T08:58:35.900 INFO:teuthology.orchestra.run.smithi099.stdout:mon 3/3 - 7m smithi099:172.21.15.99=a;smithi157:172.21.15.157=b;smithi181:172.21.15.181=c;count:3 2023-08-25T08:58:35.900 INFO:teuthology.orchestra.run.smithi099.stdout:osd 12 - - 2023-08-25T08:58:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f742c0a0060 msgr2=0x7f742c0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f742c0a0060 0x7f742c0a2520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f743c005950 tx=0x7f743c009260 comp rx=0 tx=0).stop 2023-08-25T08:58:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440074950 msgr2=0x7f7440198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440074950 0x7f7440198e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f743000b680 tx=0x7f7430004730 comp rx=0 tx=0).stop 2023-08-25T08:58:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 shutdown_connections 2023-08-25T08:58:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7440108880 0x7f744019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f742c0a0060 0x7f742c0a2520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7440073380 0x7f7440198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 --2- 172.21.15.99:0/922021450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7440074950 0x7f7440198e60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 >> 172.21.15.99:0/922021450 conn(0x7f74400fbf10 msgr2=0x7f74400fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 shutdown_connections 2023-08-25T08:58:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:35.901+0000 7f744745a700 1 -- 172.21.15.99:0/922021450 wait complete. 2023-08-25T08:58:36.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:36 smithi099 ceph-mon[97641]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:36.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:36 smithi157 ceph-mon[130879]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:36.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch host ls' 2023-08-25T08:58:36.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:36 smithi181 ceph-mon[131296]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:37.147 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:37.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:37 smithi099 ceph-mon[97641]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:37.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:37 smithi157 ceph-mon[130879]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:37.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:37 smithi181 ceph-mon[131296]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 -- 172.21.15.99:0/1262435838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c101bb0 msgr2=0x7f414c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 --2- 172.21.15.99:0/1262435838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c101bb0 0x7f414c102010 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f41400099c0 tx=0x7f4140009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 -- 172.21.15.99:0/1262435838 shutdown_connections 2023-08-25T08:58:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 --2- 172.21.15.99:0/1262435838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414c102550 0x7f414c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 --2- 172.21.15.99:0/1262435838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c1009b0 0x7f414c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 --2- 172.21.15.99:0/1262435838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c101bb0 0x7f414c102010 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 -- 172.21.15.99:0/1262435838 >> 172.21.15.99:0/1262435838 conn(0x7f414c0fbf10 msgr2=0x7f414c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 -- 172.21.15.99:0/1262435838 shutdown_connections 2023-08-25T08:58:38.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.686+0000 7f4151501700 1 -- 172.21.15.99:0/1262435838 wait complete. 2023-08-25T08:58:38.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.687+0000 7f4151501700 1 Processor -- start 2023-08-25T08:58:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.687+0000 7f4151501700 1 -- start start 2023-08-25T08:58:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.687+0000 7f4151501700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c1009b0 0x7f414c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.687+0000 7f4151501700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c101bb0 0x7f414c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.687+0000 7f4151501700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414c102550 0x7f414c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.687+0000 7f4151501700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c1a03f0 con 0x7f414c1009b0 2023-08-25T08:58:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f4151501700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c1a0530 con 0x7f414c101bb0 2023-08-25T08:58:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f4151501700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c1a06c0 con 0x7f414c102550 2023-08-25T08:58:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c1009b0 0x7f414c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c1009b0 0x7f414c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49542/0 (socket says 172.21.15.99:49542) 2023-08-25T08:58:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 -- 172.21.15.99:0/2485799550 learned_addr learned my addr 172.21.15.99:0/2485799550 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414b7fe700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414c102550 0x7f414c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414a7fc700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c101bb0 0x7f414c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 -- 172.21.15.99:0/2485799550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414c102550 msgr2=0x7f414c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414c102550 0x7f414c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 -- 172.21.15.99:0/2485799550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c101bb0 msgr2=0x7f414c198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c101bb0 0x7f414c198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 -- 172.21.15.99:0/2485799550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41400095f0 con 0x7f414c1009b0 2023-08-25T08:58:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.688+0000 7f414affd700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c1009b0 0x7f414c198860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f413c00d9b0 tx=0x7f413c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.689+0000 7f4133fff700 1 -- 172.21.15.99:0/2485799550 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c004020 con 0x7f414c1009b0 2023-08-25T08:58:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.689+0000 7f4133fff700 1 -- 172.21.15.99:0/2485799550 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f413c009730 con 0x7f414c1009b0 2023-08-25T08:58:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.689+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c1a09a0 con 0x7f414c1009b0 2023-08-25T08:58:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.689+0000 7f4133fff700 1 -- 172.21.15.99:0/2485799550 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c0039e0 con 0x7f414c1009b0 2023-08-25T08:58:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.689+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c1a0ef0 con 0x7f414c1009b0 2023-08-25T08:58:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.690+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414c10c1b0 con 0x7f414c1009b0 2023-08-25T08:58:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.692+0000 7f4133fff700 1 -- 172.21.15.99:0/2485799550 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f413c0098a0 con 0x7f414c1009b0 2023-08-25T08:58:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.693+0000 7f4133fff700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f41340a0060 0x7f41340a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.693+0000 7f4133fff700 1 -- 172.21.15.99:0/2485799550 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f413c0cad40 con 0x7f414c1009b0 2023-08-25T08:58:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.694+0000 7f414a7fc700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f41340a0060 0x7f41340a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.694+0000 7f414a7fc700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f41340a0060 0x7f41340a2520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f414000b580 tx=0x7f4140005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.695+0000 7f4133fff700 1 -- 172.21.15.99:0/2485799550 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f413c094970 con 0x7f414c1009b0 2023-08-25T08:58:38.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.904+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f414c1085a0 con 0x7f41340a0060 2023-08-25T08:58:38.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.906+0000 7f4133fff700 1 -- 172.21.15.99:0/2485799550 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f414c1085a0 con 0x7f41340a0060 2023-08-25T08:58:38.910 INFO:teuthology.orchestra.run.smithi099.stdout:HOST ADDR LABELS STATUS 2023-08-25T08:58:38.911 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099 172.21.15.99 2023-08-25T08:58:38.911 INFO:teuthology.orchestra.run.smithi099.stdout:smithi157 172.21.15.157 2023-08-25T08:58:38.911 INFO:teuthology.orchestra.run.smithi099.stdout:smithi181 172.21.15.181 2023-08-25T08:58:38.911 INFO:teuthology.orchestra.run.smithi099.stdout:3 hosts in cluster 2023-08-25T08:58:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f41340a0060 msgr2=0x7f41340a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f41340a0060 0x7f41340a2520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f414000b580 tx=0x7f4140005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c1009b0 msgr2=0x7f414c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c1009b0 0x7f414c198860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f413c00d9b0 tx=0x7f413c00dcc0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 shutdown_connections 2023-08-25T08:58:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414c102550 0x7f414c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f414c1009b0 0x7f414c198860 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f41340a0060 0x7f41340a2520 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 --2- 172.21.15.99:0/2485799550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c101bb0 0x7f414c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 >> 172.21.15.99:0/2485799550 conn(0x7f414c0fbf10 msgr2=0x7f414c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 shutdown_connections 2023-08-25T08:58:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:38.909+0000 7f4151501700 1 -- 172.21.15.99:0/2485799550 wait complete. 2023-08-25T08:58:39.423 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch device ls' 2023-08-25T08:58:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:39 smithi099 ceph-mon[97641]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:39 smithi099 ceph-mon[97641]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:39 smithi157 ceph-mon[130879]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:39 smithi157 ceph-mon[130879]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:39.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:39 smithi181 ceph-mon[131296]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:39.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:39 smithi181 ceph-mon[131296]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:40.147 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:41.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 -- 172.21.15.99:0/1449688711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008101b80 msgr2=0x7fc008101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:41.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 --2- 172.21.15.99:0/1449688711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008101b80 0x7fc008101fe0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0099c0 tx=0x7fbffc009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 -- 172.21.15.99:0/1449688711 shutdown_connections 2023-08-25T08:58:41.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 --2- 172.21.15.99:0/1449688711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008102520 0x7fc008108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 --2- 172.21.15.99:0/1449688711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc008100980 0x7fc008100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 --2- 172.21.15.99:0/1449688711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008101b80 0x7fc008101fe0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 -- 172.21.15.99:0/1449688711 >> 172.21.15.99:0/1449688711 conn(0x7fc0080fbf00 msgr2=0x7fc0080fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:41.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 -- 172.21.15.99:0/1449688711 shutdown_connections 2023-08-25T08:58:41.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.678+0000 7fc00e645700 1 -- 172.21.15.99:0/1449688711 wait complete. 2023-08-25T08:58:41.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.679+0000 7fc00e645700 1 Processor -- start 2023-08-25T08:58:41.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.679+0000 7fc00e645700 1 -- start start 2023-08-25T08:58:41.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc00e645700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008100980 0x7fc008196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:41.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc00e645700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008101b80 0x7fc008196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:41.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc00e645700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc008102520 0x7fc00819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:41.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc00e645700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00819e1a0 con 0x7fc008100980 2023-08-25T08:58:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc00e645700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00819e2e0 con 0x7fc008102520 2023-08-25T08:58:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc00e645700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00819e470 con 0x7fc008101b80 2023-08-25T08:58:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc007fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008100980 0x7fc008196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc007fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008100980 0x7fc008196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49548/0 (socket says 172.21.15.99:49548) 2023-08-25T08:58:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc007fff700 1 -- 172.21.15.99:0/3725429343 learned_addr learned my addr 172.21.15.99:0/3725429343 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc0077fe700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008101b80 0x7fc008196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc00cbe2700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc008102520 0x7fc00819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc007fff700 1 -- 172.21.15.99:0/3725429343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008101b80 msgr2=0x7fc008196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc007fff700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008101b80 0x7fc008196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc007fff700 1 -- 172.21.15.99:0/3725429343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc008102520 msgr2=0x7fc00819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc007fff700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc008102520 0x7fc00819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc007fff700 1 -- 172.21.15.99:0/3725429343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc0095f0 con 0x7fc008100980 2023-08-25T08:58:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.680+0000 7fc00cbe2700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc008102520 0x7fc00819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:58:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.681+0000 7fc007fff700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008100980 0x7fc008196610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e9a0 tx=0x7fbff800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.681+0000 7fc0057fa700 1 -- 172.21.15.99:0/3725429343 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8009790 con 0x7fc008100980 2023-08-25T08:58:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.681+0000 7fc0057fa700 1 -- 172.21.15.99:0/3725429343 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbff80044e0 con 0x7fc008100980 2023-08-25T08:58:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.681+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00819e750 con 0x7fc008100980 2023-08-25T08:58:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.681+0000 7fc0057fa700 1 -- 172.21.15.99:0/3725429343 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8003d20 con 0x7fc008100980 2023-08-25T08:58:41.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.681+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00819ec50 con 0x7fc008100980 2023-08-25T08:58:41.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.682+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00810c1d0 con 0x7fc008100980 2023-08-25T08:58:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.684+0000 7fc0057fa700 1 -- 172.21.15.99:0/3725429343 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fbff8003e80 con 0x7fc008100980 2023-08-25T08:58:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.685+0000 7fc0057fa700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fbff009ffe0 0x7fbff00a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.685+0000 7fc0057fa700 1 -- 172.21.15.99:0/3725429343 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbff8014070 con 0x7fc008100980 2023-08-25T08:58:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.685+0000 7fc0077fe700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fbff009ffe0 0x7fbff00a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.686+0000 7fc0077fe700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fbff009ffe0 0x7fbff00a24a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b580 tx=0x7fbffc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:41.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.689+0000 7fc0057fa700 1 -- 172.21.15.99:0/3725429343 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fbff8095290 con 0x7fc008100980 2023-08-25T08:58:41.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:41 smithi099 ceph-mon[97641]: pgmap v308: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:41 smithi157 ceph-mon[130879]: pgmap v308: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:41.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.894+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc008060d60 con 0x7fbff009ffe0 2023-08-25T08:58:41.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.896+0000 7fc0057fa700 1 -- 172.21.15.99:0/3725429343 <== mgr.14150 v2:172.21.15.99:6800/4068336524 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7fc008060d60 con 0x7fbff009ffe0 2023-08-25T08:58:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-08-25T08:58:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 2m ago LVM detected, locked 2023-08-25T08:58:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 2m ago LVM detected, locked 2023-08-25T08:58:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 2m ago LVM detected, locked 2023-08-25T08:58:41.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fbff009ffe0 msgr2=0x7fbff00a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:41.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fbff009ffe0 0x7fbff00a24a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b580 tx=0x7fbffc005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008100980 msgr2=0x7fc008196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:41.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008100980 0x7fc008196610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e9a0 tx=0x7fbff800ecb0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 shutdown_connections 2023-08-25T08:58:41.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fbff009ffe0 0x7fbff00a24a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc008102520 0x7fc00819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc008100980 0x7fc008196610 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 --2- 172.21.15.99:0/3725429343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008101b80 0x7fc008196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:41.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 >> 172.21.15.99:0/3725429343 conn(0x7fc0080fbf00 msgr2=0x7fc0080fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:41.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 shutdown_connections 2023-08-25T08:58:41.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:41.899+0000 7fc00e645700 1 -- 172.21.15.99:0/3725429343 wait complete. 2023-08-25T08:58:41.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:41 smithi181 ceph-mon[131296]: pgmap v308: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:42.428 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-08-25T08:58:42.437 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T08:58:42.438 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- bash -c 'ceph fs dump' 2023-08-25T08:58:42.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:42 smithi099 ceph-mon[97641]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:42.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:42 smithi157 ceph-mon[130879]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:42.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:42 smithi181 ceph-mon[131296]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:58:43.160 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:43 smithi099 ceph-mon[97641]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:43 smithi157 ceph-mon[130879]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:43 smithi181 ceph-mon[131296]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:44.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 -- 172.21.15.99:0/2254641667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101bb0 msgr2=0x7f42fc102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:44.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 --2- 172.21.15.99:0/2254641667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101bb0 0x7f42fc102010 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42f00099c0 tx=0x7f42f0009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 -- 172.21.15.99:0/2254641667 shutdown_connections 2023-08-25T08:58:44.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 --2- 172.21.15.99:0/2254641667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42fc102550 0x7f42fc108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 --2- 172.21.15.99:0/2254641667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42fc1009b0 0x7f42fc100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 --2- 172.21.15.99:0/2254641667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101bb0 0x7f42fc102010 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 -- 172.21.15.99:0/2254641667 >> 172.21.15.99:0/2254641667 conn(0x7f42fc0fbf10 msgr2=0x7f42fc0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:44.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 -- 172.21.15.99:0/2254641667 shutdown_connections 2023-08-25T08:58:44.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.732+0000 7f43023cf700 1 -- 172.21.15.99:0/2254641667 wait complete. 2023-08-25T08:58:44.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.733+0000 7f43023cf700 1 Processor -- start 2023-08-25T08:58:44.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.733+0000 7f43023cf700 1 -- start start 2023-08-25T08:58:44.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.733+0000 7f43023cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42fc1009b0 0x7f42fc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:44.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.733+0000 7f43023cf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42fc101bb0 0x7f42fc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:44.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f43023cf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc102550 0x7f42fc19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:44.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f43023cf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc1a0400 con 0x7f42fc102550 2023-08-25T08:58:44.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f43023cf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc1a0540 con 0x7f42fc101bb0 2023-08-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f43023cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc1a06d0 con 0x7f42fc1009b0 2023-08-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42fc1009b0 0x7f42fc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f430096c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc102550 0x7f42fc19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42fc1009b0 0x7f42fc198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34264/0 (socket says 172.21.15.99:34264) 2023-08-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fbfff700 1 -- 172.21.15.99:0/183297521 learned_addr learned my addr 172.21.15.99:0/183297521 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fb7fe700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42fc101bb0 0x7f42fc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fbfff700 1 -- 172.21.15.99:0/183297521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42fc101bb0 msgr2=0x7f42fc198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fbfff700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42fc101bb0 0x7f42fc198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fbfff700 1 -- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc102550 msgr2=0x7f42fc19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fbfff700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc102550 0x7f42fc19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f42fbfff700 1 -- 172.21.15.99:0/183297521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00095f0 con 0x7f42fc1009b0 2023-08-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.734+0000 7f430096c700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc102550 0x7f42fc19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.735+0000 7f42fbfff700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42fc1009b0 0x7f42fc198870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00e9a0 tx=0x7f42ec00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.735+0000 7f42f97fa700 1 -- 172.21.15.99:0/183297521 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec009790 con 0x7f42fc1009b0 2023-08-25T08:58:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.735+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc1a09b0 con 0x7f42fc1009b0 2023-08-25T08:58:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.735+0000 7f42f97fa700 1 -- 172.21.15.99:0/183297521 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42ec00c810 con 0x7f42fc1009b0 2023-08-25T08:58:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.735+0000 7f42f97fa700 1 -- 172.21.15.99:0/183297521 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec003c70 con 0x7f42fc1009b0 2023-08-25T08:58:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.735+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42fc1a0eb0 con 0x7f42fc1009b0 2023-08-25T08:58:44.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.736+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42fc192fa0 con 0x7f42fc1009b0 2023-08-25T08:58:44.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.738+0000 7f42f97fa700 1 -- 172.21.15.99:0/183297521 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f42ec00cc80 con 0x7f42fc1009b0 2023-08-25T08:58:44.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.739+0000 7f42f97fa700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f42e409ffe0 0x7f42e40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:44.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.739+0000 7f42f97fa700 1 -- 172.21.15.99:0/183297521 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f42ec014070 con 0x7f42fc1009b0 2023-08-25T08:58:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.739+0000 7f42fb7fe700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f42e409ffe0 0x7f42e40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.740+0000 7f42fb7fe700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f42e409ffe0 0x7f42e40a24a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42f000b580 tx=0x7f42f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.741+0000 7f42f97fa700 1 -- 172.21.15.99:0/183297521 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f42ec095250 con 0x7f42fc1009b0 2023-08-25T08:58:44.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.981+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f42fc10c1b0 con 0x7f42fc1009b0 2023-08-25T08:58:44.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.982+0000 7f42f97fa700 1 -- 172.21.15.99:0/183297521 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2943 (secure 0 0 0) 0x7f42ec098870 con 0x7f42fc1009b0 2023-08-25T08:58:44.983 INFO:teuthology.orchestra.run.smithi099.stdout:e23 2023-08-25T08:58:44.983 INFO:teuthology.orchestra.run.smithi099.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-08-25T08:58:44.983 INFO:teuthology.orchestra.run.smithi099.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T08:58:44.983 INFO:teuthology.orchestra.run.smithi099.stdout:legacy client fscid: 1 2023-08-25T08:58:44.983 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:44.983 INFO:teuthology.orchestra.run.smithi099.stdout:Filesystem 'cephfs' (1) 2023-08-25T08:58:44.984 INFO:teuthology.orchestra.run.smithi099.stdout:fs_name cephfs 2023-08-25T08:58:44.984 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 23 2023-08-25T08:58:44.984 INFO:teuthology.orchestra.run.smithi099.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-08-25T08:58:44.984 INFO:teuthology.orchestra.run.smithi099.stdout:created 2023-08-25T08:57:19.244738+0000 2023-08-25T08:58:44.984 INFO:teuthology.orchestra.run.smithi099.stdout:modified 2023-08-25T08:57:25.030649+0000 2023-08-25T08:58:44.985 INFO:teuthology.orchestra.run.smithi099.stdout:tableserver 0 2023-08-25T08:58:44.985 INFO:teuthology.orchestra.run.smithi099.stdout:root 0 2023-08-25T08:58:44.985 INFO:teuthology.orchestra.run.smithi099.stdout:session_timeout 300 2023-08-25T08:58:44.985 INFO:teuthology.orchestra.run.smithi099.stdout:session_autoclose 300 2023-08-25T08:58:44.985 INFO:teuthology.orchestra.run.smithi099.stdout:max_file_size 1099511627776 2023-08-25T08:58:44.985 INFO:teuthology.orchestra.run.smithi099.stdout:max_xattr_size 65536 2023-08-25T08:58:44.985 INFO:teuthology.orchestra.run.smithi099.stdout:required_client_features {} 2023-08-25T08:58:44.985 INFO:teuthology.orchestra.run.smithi099.stdout:last_failure 0 2023-08-25T08:58:44.986 INFO:teuthology.orchestra.run.smithi099.stdout:last_failure_osd_epoch 0 2023-08-25T08:58:44.986 INFO:teuthology.orchestra.run.smithi099.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T08:58:44.986 INFO:teuthology.orchestra.run.smithi099.stdout:max_mds 3 2023-08-25T08:58:44.986 INFO:teuthology.orchestra.run.smithi099.stdout:in 0,1,2 2023-08-25T08:58:44.986 INFO:teuthology.orchestra.run.smithi099.stdout:up {0=24593,1=24547,2=24517} 2023-08-25T08:58:44.986 INFO:teuthology.orchestra.run.smithi099.stdout:failed 2023-08-25T08:58:44.986 INFO:teuthology.orchestra.run.smithi099.stdout:damaged 2023-08-25T08:58:44.987 INFO:teuthology.orchestra.run.smithi099.stdout:stopped 2023-08-25T08:58:44.987 INFO:teuthology.orchestra.run.smithi099.stdout:data_pools [3] 2023-08-25T08:58:44.987 INFO:teuthology.orchestra.run.smithi099.stdout:metadata_pool 2 2023-08-25T08:58:44.987 INFO:teuthology.orchestra.run.smithi099.stdout:inline_data disabled 2023-08-25T08:58:44.987 INFO:teuthology.orchestra.run.smithi099.stdout:balancer 2023-08-25T08:58:44.987 INFO:teuthology.orchestra.run.smithi099.stdout:bal_rank_mask -1 2023-08-25T08:58:44.987 INFO:teuthology.orchestra.run.smithi099.stdout:standby_count_wanted 0 2023-08-25T08:58:44.987 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b{0:24593} state up:active seq 3 addr [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.988 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.h{0:24545} state up:standby-replay seq 1 addr [v2:172.21.15.157:6836/135740173,v1:172.21.15.157:6837/135740173] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.988 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.i{1:24547} state up:active seq 6 addr [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.988 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c{1:24529} state up:standby-replay seq 1 addr [v2:172.21.15.181:6836/2291370984,v1:172.21.15.181:6837/2291370984] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.988 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.f{2:24517} state up:active seq 8 addr [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.988 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.e{2:24512} state up:standby-replay seq 1 addr [v2:172.21.15.157:6834/1963370106,v1:172.21.15.157:6835/1963370106] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.988 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:44.988 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:44.988 INFO:teuthology.orchestra.run.smithi099.stdout:Standby daemons: 2023-08-25T08:58:44.989 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:44.989 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.j{-1:14451} state up:standby seq 1 addr [v2:172.21.15.99:6834/2749583644,v1:172.21.15.99:6835/2749583644] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.989 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.g{-1:14514} state up:standby seq 1 addr [v2:172.21.15.99:6838/2687363843,v1:172.21.15.99:6839/2687363843] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.989 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a{-1:14526} state up:standby seq 1 addr [v2:172.21.15.99:6840/1596550306,v1:172.21.15.99:6841/1596550306] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.989 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.l{-1:24463} state up:standby seq 1 addr [v2:172.21.15.181:6832/829780458,v1:172.21.15.181:6833/829780458] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.989 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.k{-1:24470} state up:standby seq 1 addr [v2:172.21.15.157:6832/3715294590,v1:172.21.15.157:6833/3715294590] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.989 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d{-1:24496} state up:standby seq 1 addr [v2:172.21.15.99:6836/654200682,v1:172.21.15.99:6837/654200682] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T08:58:44.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f42e409ffe0 msgr2=0x7f42e40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:44.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f42e409ffe0 0x7f42e40a24a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42f000b580 tx=0x7f42f0005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42fc1009b0 msgr2=0x7f42fc198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:44.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42fc1009b0 0x7f42fc198870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00e9a0 tx=0x7f42ec00ecb0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 shutdown_connections 2023-08-25T08:58:44.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc102550 0x7f42fc19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42fc1009b0 0x7f42fc198870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f42e409ffe0 0x7f42e40a24a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 --2- 172.21.15.99:0/183297521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42fc101bb0 0x7f42fc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:44.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.984+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 >> 172.21.15.99:0/183297521 conn(0x7f42fc0fbf10 msgr2=0x7f42fc107510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:44.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.985+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 shutdown_connections 2023-08-25T08:58:44.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:44.985+0000 7f43023cf700 1 -- 172.21.15.99:0/183297521 wait complete. 2023-08-25T08:58:44.992 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2023-08-25T08:58:45.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:45 smithi099 ceph-mon[97641]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:45.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:45 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/183297521' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-25T08:58:45.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:45 smithi157 ceph-mon[130879]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:45.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:45 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/183297521' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-25T08:58:45.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- bash -c 'ceph osd dump' 2023-08-25T08:58:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:45 smithi181 ceph-mon[131296]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:45 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/183297521' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-25T08:58:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:47.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 -- 172.21.15.99:0/586032160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 msgr2=0x7f8e84072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:47.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 --2- 172.21.15.99:0/586032160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 0x7f8e84072db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009990 tx=0x7f8e74009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 -- 172.21.15.99:0/586032160 shutdown_connections 2023-08-25T08:58:47.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 --2- 172.21.15.99:0/586032160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e84108880 0x7f8e8410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 --2- 172.21.15.99:0/586032160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 0x7f8e84072db0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 --2- 172.21.15.99:0/586032160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e84073380 0x7f8e84073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 -- 172.21.15.99:0/586032160 >> 172.21.15.99:0/586032160 conn(0x7f8e840fbf10 msgr2=0x7f8e840fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:47.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 -- 172.21.15.99:0/586032160 shutdown_connections 2023-08-25T08:58:47.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.733+0000 7f8e89d38700 1 -- 172.21.15.99:0/586032160 wait complete. 2023-08-25T08:58:47.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.734+0000 7f8e89d38700 1 Processor -- start 2023-08-25T08:58:47.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.734+0000 7f8e89d38700 1 -- start start 2023-08-25T08:58:47.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.734+0000 7f8e89d38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e84073380 0x7f8e84198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:47.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.734+0000 7f8e89d38700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 0x7f8e84198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:47.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.734+0000 7f8e89d38700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e84108880 0x7f8e8419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:47.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.734+0000 7f8e89d38700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841a0410 con 0x7f8e84074950 2023-08-25T08:58:47.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.734+0000 7f8e89d38700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841a0550 con 0x7f8e84108880 2023-08-25T08:58:47.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.734+0000 7f8e89d38700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841a06e0 con 0x7f8e84073380 2023-08-25T08:58:47.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e84073380 0x7f8e84198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:47.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e82ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 0x7f8e84198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:47.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e84073380 0x7f8e84198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:36776/0 (socket says 172.21.15.99:36776) 2023-08-25T08:58:47.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e837fe700 1 -- 172.21.15.99:0/3306736838 learned_addr learned my addr 172.21.15.99:0/3306736838 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:47.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e83fff700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e84108880 0x7f8e8419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:47.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e82ffd700 1 -- 172.21.15.99:0/3306736838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e84073380 msgr2=0x7f8e84198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:47.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e82ffd700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e84073380 0x7f8e84198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e82ffd700 1 -- 172.21.15.99:0/3306736838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e84108880 msgr2=0x7f8e8419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:47.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e82ffd700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e84108880 0x7f8e8419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e82ffd700 1 -- 172.21.15.99:0/3306736838 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740095f0 con 0x7f8e84074950 2023-08-25T08:58:47.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e83fff700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e84108880 0x7f8e8419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:47.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e837fe700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e84073380 0x7f8e84198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:47.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.735+0000 7f8e82ffd700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 0x7f8e84198dc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800e9f0 tx=0x7f8e7800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:47.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.736+0000 7f8e80ff9700 1 -- 172.21.15.99:0/3306736838 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800eee0 con 0x7f8e84074950 2023-08-25T08:58:47.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.736+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e841a09c0 con 0x7f8e84074950 2023-08-25T08:58:47.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.736+0000 7f8e80ff9700 1 -- 172.21.15.99:0/3306736838 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e780044e0 con 0x7f8e84074950 2023-08-25T08:58:47.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.736+0000 7f8e80ff9700 1 -- 172.21.15.99:0/3306736838 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78003c70 con 0x7f8e84074950 2023-08-25T08:58:47.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.736+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e841a0ec0 con 0x7f8e84074950 2023-08-25T08:58:47.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.737+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e84106f30 con 0x7f8e84074950 2023-08-25T08:58:47.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.739+0000 7f8e80ff9700 1 -- 172.21.15.99:0/3306736838 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f8e7800ccd0 con 0x7f8e84074950 2023-08-25T08:58:47.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.740+0000 7f8e80ff9700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6c09ffe0 0x7f8e6c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:47.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.740+0000 7f8e80ff9700 1 -- 172.21.15.99:0/3306736838 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8e78014070 con 0x7f8e84074950 2023-08-25T08:58:47.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.740+0000 7f8e837fe700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6c09ffe0 0x7f8e6c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:47.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.741+0000 7f8e837fe700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6c09ffe0 0x7f8e6c0a24a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400b680 tx=0x7f8e74005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:47.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.744+0000 7f8e80ff9700 1 -- 172.21.15.99:0/3306736838 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8e78095290 con 0x7f8e84074950 2023-08-25T08:58:47.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:47 smithi099 ceph-mon[97641]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:47 smithi157 ceph-mon[130879]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.944+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f8e8410e3b0 con 0x7f8e84074950 2023-08-25T08:58:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.945+0000 7f8e80ff9700 1 -- 172.21.15.99:0/3306736838 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5334 (secure 0 0 0) 0x7f8e780988b0 con 0x7f8e84074950 2023-08-25T08:58:47.945 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 85 2023-08-25T08:58:47.945 INFO:teuthology.orchestra.run.smithi099.stdout:fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T08:58:47.945 INFO:teuthology.orchestra.run.smithi099.stdout:created 2023-08-25T08:49:52.251496+0000 2023-08-25T08:58:47.945 INFO:teuthology.orchestra.run.smithi099.stdout:modified 2023-08-25T08:57:56.041917+0000 2023-08-25T08:58:47.945 INFO:teuthology.orchestra.run.smithi099.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-08-25T08:58:47.946 INFO:teuthology.orchestra.run.smithi099.stdout:crush_version 26 2023-08-25T08:58:47.946 INFO:teuthology.orchestra.run.smithi099.stdout:full_ratio 0.95 2023-08-25T08:58:47.946 INFO:teuthology.orchestra.run.smithi099.stdout:backfillfull_ratio 0.9 2023-08-25T08:58:47.946 INFO:teuthology.orchestra.run.smithi099.stdout:nearfull_ratio 0.85 2023-08-25T08:58:47.946 INFO:teuthology.orchestra.run.smithi099.stdout:require_min_compat_client luminous 2023-08-25T08:58:47.946 INFO:teuthology.orchestra.run.smithi099.stdout:min_compat_client luminous 2023-08-25T08:58:47.946 INFO:teuthology.orchestra.run.smithi099.stdout:require_osd_release reef 2023-08-25T08:58:47.946 INFO:teuthology.orchestra.run.smithi099.stdout:stretch_mode_enabled false 2023-08-25T08:58:47.947 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-08-25T08:58:47.947 INFO:teuthology.orchestra.run.smithi099.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-08-25T08:58:47.947 INFO:teuthology.orchestra.run.smithi099.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-08-25T08:58:47.947 INFO:teuthology.orchestra.run.smithi099.stdout:max_osd 12 2023-08-25T08:58:47.947 INFO:teuthology.orchestra.run.smithi099.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6802/3269986106,v1:172.21.15.99:6803/3269986106] [v2:172.21.15.99:6804/3269986106,v1:172.21.15.99:6805/3269986106] exists,up fd38b95e-eec4-4af7-91aa-73e6e78584e3 2023-08-25T08:58:47.947 INFO:teuthology.orchestra.run.smithi099.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/2722577134,v1:172.21.15.157:6801/2722577134] [v2:172.21.15.157:6802/2722577134,v1:172.21.15.157:6803/2722577134] exists,up 6b212381-183b-4878-be8a-e088b3e62122 2023-08-25T08:58:47.947 INFO:teuthology.orchestra.run.smithi099.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/4155797556,v1:172.21.15.181:6801/4155797556] [v2:172.21.15.181:6802/4155797556,v1:172.21.15.181:6803/4155797556] exists,up 2577a270-c667-48bf-9382-d1d5d2ec37df 2023-08-25T08:58:47.948 INFO:teuthology.orchestra.run.smithi099.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6810/2656518905,v1:172.21.15.99:6811/2656518905] [v2:172.21.15.99:6812/2656518905,v1:172.21.15.99:6813/2656518905] exists,up ae5e672d-2f3e-4afd-8207-bcc4f2aa6530 2023-08-25T08:58:47.948 INFO:teuthology.orchestra.run.smithi099.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/488498023,v1:172.21.15.157:6809/488498023] [v2:172.21.15.157:6810/488498023,v1:172.21.15.157:6811/488498023] exists,up 91518e8f-f259-4741-9f5b-17b1e1d1b662 2023-08-25T08:58:47.948 INFO:teuthology.orchestra.run.smithi099.stdout:osd.5 up in weight 1 up_from 38 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/1806112824,v1:172.21.15.181:6809/1806112824] [v2:172.21.15.181:6810/1806112824,v1:172.21.15.181:6811/1806112824] exists,up c17bcccb-4491-4d2c-8864-f2e744438d5e 2023-08-25T08:58:47.948 INFO:teuthology.orchestra.run.smithi099.stdout:osd.6 up in weight 1 up_from 46 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6818/2954530435,v1:172.21.15.99:6819/2954530435] [v2:172.21.15.99:6820/2954530435,v1:172.21.15.99:6821/2954530435] exists,up 8629c9d9-a959-4532-94c1-9b923a0760a2 2023-08-25T08:58:47.948 INFO:teuthology.orchestra.run.smithi099.stdout:osd.7 up in weight 1 up_from 51 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/3957644340,v1:172.21.15.157:6817/3957644340] [v2:172.21.15.157:6818/3957644340,v1:172.21.15.157:6819/3957644340] exists,up c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5 2023-08-25T08:58:47.948 INFO:teuthology.orchestra.run.smithi099.stdout:osd.8 up in weight 1 up_from 58 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/411715472,v1:172.21.15.181:6817/411715472] [v2:172.21.15.181:6818/411715472,v1:172.21.15.181:6819/411715472] exists,up 91174f3e-a7c9-496c-a2c7-bdf90a0d59f6 2023-08-25T08:58:47.948 INFO:teuthology.orchestra.run.smithi099.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6826/797626209,v1:172.21.15.99:6827/797626209] [v2:172.21.15.99:6828/797626209,v1:172.21.15.99:6829/797626209] exists,up 7256a919-8afa-4675-867e-880a61ae8a4f 2023-08-25T08:58:47.949 INFO:teuthology.orchestra.run.smithi099.stdout:osd.10 up in weight 1 up_from 71 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/3880870693,v1:172.21.15.157:6825/3880870693] [v2:172.21.15.157:6826/3880870693,v1:172.21.15.157:6827/3880870693] exists,up 8274d02b-096a-4240-ac33-738e885e7451 2023-08-25T08:58:47.949 INFO:teuthology.orchestra.run.smithi099.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/1448660331,v1:172.21.15.181:6825/1448660331] [v2:172.21.15.181:6826/1448660331,v1:172.21.15.181:6827/1448660331] exists,up 4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4 2023-08-25T08:58:47.949 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.1c [8,11] 2023-08-25T08:58:47.949 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.1d [2,4] 2023-08-25T08:58:47.949 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.0 [4,6] 2023-08-25T08:58:47.949 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.e [4,7] 2023-08-25T08:58:47.950 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.18 [3,9] 2023-08-25T08:58:47.950 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.1d [4,6] 2023-08-25T08:58:47.950 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.20 [5,9] 2023-08-25T08:58:47.950 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.2a [1,9] 2023-08-25T08:58:47.950 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.39 [5,8] 2023-08-25T08:58:47.951 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.3a [4,8] 2023-08-25T08:58:47.951 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2769940141 expires 2023-08-26T08:50:54.351659+0000 2023-08-25T08:58:47.951 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/124662375 expires 2023-08-26T08:50:54.351659+0000 2023-08-25T08:58:47.951 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/483386971 expires 2023-08-26T08:50:54.351659+0000 2023-08-25T08:58:47.951 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6801/2336486699 expires 2023-08-26T08:50:54.351659+0000 2023-08-25T08:58:47.951 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/1268974739 expires 2023-08-26T08:50:28.397071+0000 2023-08-25T08:58:47.952 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/1871470354 expires 2023-08-26T08:50:28.397071+0000 2023-08-25T08:58:47.952 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6801/3349539225 expires 2023-08-26T08:50:28.397071+0000 2023-08-25T08:58:47.952 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6800/3349539225 expires 2023-08-26T08:50:28.397071+0000 2023-08-25T08:58:47.952 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2636132577 expires 2023-08-26T08:50:28.397071+0000 2023-08-25T08:58:47.953 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6800/2336486699 expires 2023-08-26T08:50:54.351659+0000 2023-08-25T08:58:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.947+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6c09ffe0 msgr2=0x7f8e6c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.947+0000 7f8e89d38700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6c09ffe0 0x7f8e6c0a24a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400b680 tx=0x7f8e74005fd0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.947+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 msgr2=0x7f8e84198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.947+0000 7f8e89d38700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 0x7f8e84198dc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800e9f0 tx=0x7f8e7800ed00 comp rx=0 tx=0).stop 2023-08-25T08:58:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.948+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 shutdown_connections 2023-08-25T08:58:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.948+0000 7f8e89d38700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f8e6c09ffe0 0x7f8e6c0a24a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.948+0000 7f8e89d38700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e84108880 0x7f8e8419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.948+0000 7f8e89d38700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e84073380 0x7f8e84198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.948+0000 7f8e89d38700 1 --2- 172.21.15.99:0/3306736838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e84074950 0x7f8e84198dc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:47.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.948+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 >> 172.21.15.99:0/3306736838 conn(0x7f8e840fbf10 msgr2=0x7f8e840fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:47.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.948+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 shutdown_connections 2023-08-25T08:58:47.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:47.948+0000 7f8e89d38700 1 -- 172.21.15.99:0/3306736838 wait complete. 2023-08-25T08:58:47.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:47 smithi181 ceph-mon[131296]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:48.425 INFO:teuthology.run_tasks:Running task fs.ready... 2023-08-25T08:58:48.438 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:58:48.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.642+0000 7f07419ff700 1 -- 172.21.15.99:0/1294046744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c100930 msgr2=0x7f073c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:48.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.642+0000 7f07419ff700 1 --2- 172.21.15.99:0/1294046744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c100930 0x7f073c100d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f072c0099c0 tx=0x7f072c009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.643+0000 7f07419ff700 1 -- 172.21.15.99:0/1294046744 shutdown_connections 2023-08-25T08:58:48.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.643+0000 7f07419ff700 1 --2- 172.21.15.99:0/1294046744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073c1024c0 0x7f073c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.643+0000 7f07419ff700 1 --2- 172.21.15.99:0/1294046744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c100930 0x7f073c100d50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.643+0000 7f07419ff700 1 --2- 172.21.15.99:0/1294046744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c101b20 0x7f073c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.643+0000 7f07419ff700 1 -- 172.21.15.99:0/1294046744 >> 172.21.15.99:0/1294046744 conn(0x7f073c0fbe60 msgr2=0x7f073c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:48.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.643+0000 7f07419ff700 1 -- 172.21.15.99:0/1294046744 shutdown_connections 2023-08-25T08:58:48.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.643+0000 7f07419ff700 1 -- 172.21.15.99:0/1294046744 wait complete. 2023-08-25T08:58:48.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f07419ff700 1 Processor -- start 2023-08-25T08:58:48.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f07419ff700 1 -- start start 2023-08-25T08:58:48.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f07419ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c100930 0x7f073c196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:48.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f07419ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073c101b20 0x7f073c196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:48.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f07419ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c1024c0 0x7f073c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:48.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f07419ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c19e1a0 con 0x7f073c1024c0 2023-08-25T08:58:48.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f07419ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c19e2e0 con 0x7f073c101b20 2023-08-25T08:58:48.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f07419ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c19e470 con 0x7f073c100930 2023-08-25T08:58:48.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.644+0000 7f073a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073c101b20 0x7f073c196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:48.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c100930 0x7f073c196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:48.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c1024c0 0x7f073c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:48.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073c101b20 0x7f073c196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47962/0 (socket says 172.21.15.99:47962) 2023-08-25T08:58:48.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073a7fc700 1 -- 172.21.15.99:0/2437621110 learned_addr learned my addr 172.21.15.99:0/2437621110 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:48.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073b7fe700 1 -- 172.21.15.99:0/2437621110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c100930 msgr2=0x7f073c196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:48.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073b7fe700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c100930 0x7f073c196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073b7fe700 1 -- 172.21.15.99:0/2437621110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073c101b20 msgr2=0x7f073c196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:48.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073b7fe700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073c101b20 0x7f073c196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073b7fe700 1 -- 172.21.15.99:0/2437621110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0095f0 con 0x7f073c1024c0 2023-08-25T08:58:48.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073affd700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c100930 0x7f073c196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:48.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.645+0000 7f073b7fe700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c1024c0 0x7f073c19dbd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0734007f10 tx=0x7f073400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:48.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.646+0000 7f07409fd700 1 -- 172.21.15.99:0/2437621110 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734009ea0 con 0x7f073c1024c0 2023-08-25T08:58:48.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.646+0000 7f07409fd700 1 -- 172.21.15.99:0/2437621110 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f073400f040 con 0x7f073c1024c0 2023-08-25T08:58:48.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.646+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c19e750 con 0x7f073c1024c0 2023-08-25T08:58:48.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.646+0000 7f07409fd700 1 -- 172.21.15.99:0/2437621110 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734004350 con 0x7f073c1024c0 2023-08-25T08:58:48.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.646+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c19ec10 con 0x7f073c1024c0 2023-08-25T08:58:48.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.647+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c107ec0 con 0x7f073c1024c0 2023-08-25T08:58:48.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.649+0000 7f07409fd700 1 -- 172.21.15.99:0/2437621110 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f073400db20 con 0x7f073c1024c0 2023-08-25T08:58:48.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.650+0000 7f07409fd700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07240a0020 0x7f07240a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:48.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.650+0000 7f07409fd700 1 -- 172.21.15.99:0/2437621110 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f07340cad00 con 0x7f073c1024c0 2023-08-25T08:58:48.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.651+0000 7f073affd700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07240a0020 0x7f07240a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:48.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.651+0000 7f073affd700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07240a0020 0x7f07240a24e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f072c009f50 tx=0x7f072c005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:48.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.654+0000 7f07409fd700 1 -- 172.21.15.99:0/2437621110 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f07340948b0 con 0x7f073c1024c0 2023-08-25T08:58:48.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:48 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3306736838' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-25T08:58:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.855+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f073c10c3e0 con 0x7f073c1024c0 2023-08-25T08:58:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.857+0000 7f07409fd700 1 -- 172.21.15.99:0/2437621110 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22646 (secure 0 0 0) 0x7f0734019090 con 0x7f073c1024c0 2023-08-25T08:58:48.858 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:48.858 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:56.041917+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:58:48.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.859+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07240a0020 msgr2=0x7f07240a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:48.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.859+0000 7f07419ff700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07240a0020 0x7f07240a24e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f072c009f50 tx=0x7f072c005bc0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.859+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c1024c0 msgr2=0x7f073c19dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:48.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.859+0000 7f07419ff700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c1024c0 0x7f073c19dbd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0734007f10 tx=0x7f073400d3b0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.860+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 shutdown_connections 2023-08-25T08:58:48.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.860+0000 7f07419ff700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c1024c0 0x7f073c19dbd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.860+0000 7f07419ff700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c100930 0x7f073c196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.860+0000 7f07419ff700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07240a0020 0x7f07240a24e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.860+0000 7f07419ff700 1 --2- 172.21.15.99:0/2437621110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073c101b20 0x7f073c196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:48.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.860+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 >> 172.21.15.99:0/2437621110 conn(0x7f073c0fbe60 msgr2=0x7f073c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:48.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.860+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 shutdown_connections 2023-08-25T08:58:48.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:48.860+0000 7f07419ff700 1 -- 172.21.15.99:0/2437621110 wait complete. 2023-08-25T08:58:48.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T08:57:17.389400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T08:57:18.695578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T08:58:48.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T08:58:48.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:48 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3306736838' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-25T08:58:48.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:48 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3306736838' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-25T08:58:49.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 -- 172.21.15.99:0/557824888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c101b20 msgr2=0x7fc37c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 --2- 172.21.15.99:0/557824888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c101b20 0x7fc37c101f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0099c0 tx=0x7fc36c009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 -- 172.21.15.99:0/557824888 shutdown_connections 2023-08-25T08:58:49.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 --2- 172.21.15.99:0/557824888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37c1024c0 0x7fc37c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 --2- 172.21.15.99:0/557824888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37c100930 0x7fc37c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 --2- 172.21.15.99:0/557824888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c101b20 0x7fc37c101f80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 -- 172.21.15.99:0/557824888 >> 172.21.15.99:0/557824888 conn(0x7fc37c0fbea0 msgr2=0x7fc37c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:49.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 -- 172.21.15.99:0/557824888 shutdown_connections 2023-08-25T08:58:49.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.081+0000 7fc383bff700 1 -- 172.21.15.99:0/557824888 wait complete. 2023-08-25T08:58:49.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.082+0000 7fc383bff700 1 Processor -- start 2023-08-25T08:58:49.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.082+0000 7fc383bff700 1 -- start start 2023-08-25T08:58:49.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc383bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37c100930 0x7fc37c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc383bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37c101b20 0x7fc37c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc383bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c1024c0 0x7fc37c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc383bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1a0340 con 0x7fc37c1024c0 2023-08-25T08:58:49.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc383bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1a0480 con 0x7fc37c101b20 2023-08-25T08:58:49.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc383bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1a0610 con 0x7fc37c100930 2023-08-25T08:58:49.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38219c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c1024c0 0x7fc37c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38219c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c1024c0 0x7fc37c19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41894/0 (socket says 172.21.15.99:41894) 2023-08-25T08:58:49.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38219c700 1 -- 172.21.15.99:0/3156893314 learned_addr learned my addr 172.21.15.99:0/3156893314 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:49.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38199b700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37c100930 0x7fc37c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38119a700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37c101b20 0x7fc37c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38219c700 1 -- 172.21.15.99:0/3156893314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37c100930 msgr2=0x7fc37c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38219c700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37c100930 0x7fc37c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38219c700 1 -- 172.21.15.99:0/3156893314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37c101b20 msgr2=0x7fc37c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38219c700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37c101b20 0x7fc37c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.083+0000 7fc38219c700 1 -- 172.21.15.99:0/3156893314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c0095f0 con 0x7fc37c1024c0 2023-08-25T08:58:49.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.084+0000 7fc38219c700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c1024c0 0x7fc37c19fd70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc37400ef30 tx=0x7fc37400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:49.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.084+0000 7fc372ffd700 1 -- 172.21.15.99:0/3156893314 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374018070 con 0x7fc37c1024c0 2023-08-25T08:58:49.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.084+0000 7fc372ffd700 1 -- 172.21.15.99:0/3156893314 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc37400f040 con 0x7fc37c1024c0 2023-08-25T08:58:49.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.084+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c1a0890 con 0x7fc37c1024c0 2023-08-25T08:58:49.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.084+0000 7fc372ffd700 1 -- 172.21.15.99:0/3156893314 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37400cd00 con 0x7fc37c1024c0 2023-08-25T08:58:49.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.084+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c1a0da0 con 0x7fc37c1024c0 2023-08-25T08:58:49.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.085+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c192f80 con 0x7fc37c1024c0 2023-08-25T08:58:49.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.087+0000 7fc372ffd700 1 -- 172.21.15.99:0/3156893314 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc374003a00 con 0x7fc37c1024c0 2023-08-25T08:58:49.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.088+0000 7fc372ffd700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3680a0060 0x7fc3680a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.088+0000 7fc372ffd700 1 -- 172.21.15.99:0/3156893314 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc3740caf00 con 0x7fc37c1024c0 2023-08-25T08:58:49.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.088+0000 7fc38199b700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3680a0060 0x7fc3680a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.089+0000 7fc38199b700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3680a0060 0x7fc3680a2520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc37c101980 tx=0x7fc378009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.092+0000 7fc372ffd700 1 -- 172.21.15.99:0/3156893314 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc374094b80 con 0x7fc37c1024c0 2023-08-25T08:58:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.292+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc37c10c3b0 con 0x7fc37c1024c0 2023-08-25T08:58:49.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.293+0000 7fc372ffd700 1 -- 172.21.15.99:0/3156893314 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3740981a0 con 0x7fc37c1024c0 2023-08-25T08:58:49.293 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T08:58:49.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3680a0060 msgr2=0x7fc3680a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3680a0060 0x7fc3680a2520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc37c101980 tx=0x7fc378009260 comp rx=0 tx=0).stop 2023-08-25T08:58:49.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c1024c0 msgr2=0x7fc37c19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c1024c0 0x7fc37c19fd70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc37400ef30 tx=0x7fc37400c3a0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 shutdown_connections 2023-08-25T08:58:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc37c1024c0 0x7fc37c19fd70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37c100930 0x7fc37c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fc3680a0060 0x7fc3680a2520 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 --2- 172.21.15.99:0/3156893314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37c101b20 0x7fc37c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 >> 172.21.15.99:0/3156893314 conn(0x7fc37c0fbea0 msgr2=0x7fc37c1073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 shutdown_connections 2023-08-25T08:58:49.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.295+0000 7fc383bff700 1 -- 172.21.15.99:0/3156893314 wait complete. 2023-08-25T08:58:49.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T08:58:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 -- 172.21.15.99:0/3014082275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f8100930 msgr2=0x7f74f8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/3014082275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f8100930 0x7f74f8100d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009990 tx=0x7f74ec009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 -- 172.21.15.99:0/3014082275 shutdown_connections 2023-08-25T08:58:49.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/3014082275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f81024c0 0x7f74f8108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/3014082275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f8100930 0x7f74f8100d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/3014082275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f8101b20 0x7f74f8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 -- 172.21.15.99:0/3014082275 >> 172.21.15.99:0/3014082275 conn(0x7f74f80fbea0 msgr2=0x7f74f80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:49.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 -- 172.21.15.99:0/3014082275 shutdown_connections 2023-08-25T08:58:49.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.514+0000 7f74ff1ff700 1 -- 172.21.15.99:0/3014082275 wait complete. 2023-08-25T08:58:49.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.515+0000 7f74ff1ff700 1 Processor -- start 2023-08-25T08:58:49.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74ff1ff700 1 -- start start 2023-08-25T08:58:49.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74ff1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f8100930 0x7f74f81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74ff1ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f8101b20 0x7f74f8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74ff1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f81024c0 0x7f74f819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74ff1ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f81a0420 con 0x7f74f8101b20 2023-08-25T08:58:49.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74ff1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f81a0560 con 0x7f74f8100930 2023-08-25T08:58:49.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74ff1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f81a06f0 con 0x7f74f81024c0 2023-08-25T08:58:49.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74f7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f8101b20 0x7f74f8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74fcf9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f8100930 0x7f74f81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74fcf9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f8100930 0x7f74f81988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47994/0 (socket says 172.21.15.99:47994) 2023-08-25T08:58:49.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74fcf9b700 1 -- 172.21.15.99:0/2602868727 learned_addr learned my addr 172.21.15.99:0/2602868727 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:49.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.516+0000 7f74fd79c700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f81024c0 0x7f74f819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.517+0000 7f74f7fff700 1 -- 172.21.15.99:0/2602868727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f81024c0 msgr2=0x7f74f819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.517+0000 7f74f7fff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f81024c0 0x7f74f819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.517+0000 7f74f7fff700 1 -- 172.21.15.99:0/2602868727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f8100930 msgr2=0x7f74f81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.517+0000 7f74f7fff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f8100930 0x7f74f81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.517+0000 7f74f7fff700 1 -- 172.21.15.99:0/2602868727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0009c40 con 0x7f74f8101b20 2023-08-25T08:58:49.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.517+0000 7f74fcf9b700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f8100930 0x7f74f81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:49.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.517+0000 7f74f7fff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f8101b20 0x7f74f8198de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f74e400e9f0 tx=0x7f74e400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.520+0000 7f74f5ffb700 1 -- 172.21.15.99:0/2602868727 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e400c9c0 con 0x7f74f8101b20 2023-08-25T08:58:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.520+0000 7f74f5ffb700 1 -- 172.21.15.99:0/2602868727 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74e4010430 con 0x7f74f8101b20 2023-08-25T08:58:49.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.520+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec0095f0 con 0x7f74f8101b20 2023-08-25T08:58:49.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.520+0000 7f74f5ffb700 1 -- 172.21.15.99:0/2602868727 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4018680 con 0x7f74f8101b20 2023-08-25T08:58:49.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.520+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f81a0d50 con 0x7f74f8101b20 2023-08-25T08:58:49.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.521+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f810c190 con 0x7f74f8101b20 2023-08-25T08:58:49.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.523+0000 7f74f5ffb700 1 -- 172.21.15.99:0/2602868727 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f74e4010860 con 0x7f74f8101b20 2023-08-25T08:58:49.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.524+0000 7f74f5ffb700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74e809ffe0 0x7f74e80a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.524+0000 7f74f5ffb700 1 -- 172.21.15.99:0/2602868727 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f74e4014070 con 0x7f74f8101b20 2023-08-25T08:58:49.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.525+0000 7f74fcf9b700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74e809ffe0 0x7f74e80a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.525+0000 7f74fcf9b700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74e809ffe0 0x7f74e80a24a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00b2e0 tx=0x7f74ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:49.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.528+0000 7f74f5ffb700 1 -- 172.21.15.99:0/2602868727 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f74e4051e00 con 0x7f74f8101b20 2023-08-25T08:58:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:49 smithi099 ceph-mon[97641]: pgmap v312: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:49 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2437621110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:49 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3156893314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.729+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74f81a0f70 con 0x7f74f8101b20 2023-08-25T08:58:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.729+0000 7f74f5ffb700 1 -- 172.21.15.99:0/2602868727 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74e4095370 con 0x7f74f8101b20 2023-08-25T08:58:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-08-25T08:58:49.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74e809ffe0 msgr2=0x7f74e80a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74e809ffe0 0x7f74e80a24a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00b2e0 tx=0x7f74ec005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f8101b20 msgr2=0x7f74f8198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f8101b20 0x7f74f8198de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f74e400e9f0 tx=0x7f74e400ed00 comp rx=0 tx=0).stop 2023-08-25T08:58:49.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 shutdown_connections 2023-08-25T08:58:49.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f81024c0 0x7f74f819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f74e809ffe0 0x7f74e80a24a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f8100930 0x7f74f81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 --2- 172.21.15.99:0/2602868727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f8101b20 0x7f74f8198de0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 >> 172.21.15.99:0/2602868727 conn(0x7f74f80fbea0 msgr2=0x7f74f80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:49.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 shutdown_connections 2023-08-25T08:58:49.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.732+0000 7f74ff1ff700 1 -- 172.21.15.99:0/2602868727 wait complete. 2023-08-25T08:58:49.752 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T08:58:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:49 smithi157 ceph-mon[130879]: pgmap v312: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:49 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2437621110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:49 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3156893314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:49.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 -- 172.21.15.99:0/1798177323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 msgr2=0x7f6ffc073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 --2- 172.21.15.99:0/1798177323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 0x7f6ffc073800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009990 tx=0x7f6fe4009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 -- 172.21.15.99:0/1798177323 shutdown_connections 2023-08-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 --2- 172.21.15.99:0/1798177323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ffc1087d0 0x7f6ffc10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 --2- 172.21.15.99:0/1798177323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ffc074960 0x7f6ffc072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 --2- 172.21.15.99:0/1798177323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 0x7f6ffc073800 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 -- 172.21.15.99:0/1798177323 >> 172.21.15.99:0/1798177323 conn(0x7f6ffc0fbe60 msgr2=0x7f6ffc0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 -- 172.21.15.99:0/1798177323 shutdown_connections 2023-08-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.952+0000 7f7000fff700 1 -- 172.21.15.99:0/1798177323 wait complete. 2023-08-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.953+0000 7f7000fff700 1 Processor -- start 2023-08-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.953+0000 7f7000fff700 1 -- start start 2023-08-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f7000fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 0x7f6ffc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f7000fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ffc074960 0x7f6ffc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f7000fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ffc1087d0 0x7f6ffc19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f7000fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1a03f0 con 0x7f6ffc073380 2023-08-25T08:58:49.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f7000fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1a0530 con 0x7f6ffc074960 2023-08-25T08:58:49.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f7000fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1a06c0 con 0x7f6ffc1087d0 2023-08-25T08:58:49.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 0x7f6ffc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 0x7f6ffc198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41936/0 (socket says 172.21.15.99:41936) 2023-08-25T08:58:49.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 -- 172.21.15.99:0/2095053693 learned_addr learned my addr 172.21.15.99:0/2095053693 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:49.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ff3fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ffc074960 0x7f6ffc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffad9d700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ffc1087d0 0x7f6ffc19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 -- 172.21.15.99:0/2095053693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ffc1087d0 msgr2=0x7f6ffc19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ffc1087d0 0x7f6ffc19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 -- 172.21.15.99:0/2095053693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ffc074960 msgr2=0x7f6ffc198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:49.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ffc074960 0x7f6ffc198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:49.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 -- 172.21.15.99:0/2095053693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40095f0 con 0x7f6ffc073380 2023-08-25T08:58:49.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.954+0000 7f6ffa59c700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 0x7f6ffc198870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00d980 tx=0x7f6fec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:49.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.955+0000 7f6ff37fe700 1 -- 172.21.15.99:0/2095053693 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec004220 con 0x7f6ffc073380 2023-08-25T08:58:49.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.955+0000 7f6ff37fe700 1 -- 172.21.15.99:0/2095053693 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fec007640 con 0x7f6ffc073380 2023-08-25T08:58:49.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.955+0000 7f6ff37fe700 1 -- 172.21.15.99:0/2095053693 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec005610 con 0x7f6ffc073380 2023-08-25T08:58:49.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.955+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc1a09a0 con 0x7f6ffc073380 2023-08-25T08:58:49.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.955+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc1a0eb0 con 0x7f6ffc073380 2023-08-25T08:58:49.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.956+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc106e80 con 0x7f6ffc073380 2023-08-25T08:58:49.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.958+0000 7f6ff37fe700 1 -- 172.21.15.99:0/2095053693 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f6fec004750 con 0x7f6ffc073380 2023-08-25T08:58:49.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.959+0000 7f6ff37fe700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fe80a0060 0x7f6fe80a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:49.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.959+0000 7f6ff37fe700 1 -- 172.21.15.99:0/2095053693 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6fec0cb2d0 con 0x7f6ffc073380 2023-08-25T08:58:49.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.960+0000 7f6ff3fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fe80a0060 0x7f6fe80a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:49.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.960+0000 7f6ff3fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fe80a0060 0x7f6fe80a2520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400b5c0 tx=0x7f6fe4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:49.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:49.963+0000 7f6ff37fe700 1 -- 172.21.15.99:0/2095053693 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f6fec094e80 con 0x7f6ffc073380 2023-08-25T08:58:49.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:49 smithi181 ceph-mon[131296]: pgmap v312: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T08:58:49.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:49 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2437621110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:49.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:49 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3156893314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:50.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.166+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ffc10e350 con 0x7f6ffc073380 2023-08-25T08:58:50.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.166+0000 7f6ff37fe700 1 -- 172.21.15.99:0/2095053693 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ffc10e350 con 0x7f6ffc073380 2023-08-25T08:58:50.166 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-08-25T08:58:50.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fe80a0060 msgr2=0x7f6fe80a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fe80a0060 0x7f6fe80a2520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400b5c0 tx=0x7f6fe4005fd0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 msgr2=0x7f6ffc198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 0x7f6ffc198870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00d980 tx=0x7f6fec00dc90 comp rx=0 tx=0).stop 2023-08-25T08:58:50.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 shutdown_connections 2023-08-25T08:58:50.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ffc1087d0 0x7f6ffc19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ffc073380 0x7f6ffc198870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f6fe80a0060 0x7f6fe80a2520 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.168+0000 7f7000fff700 1 --2- 172.21.15.99:0/2095053693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ffc074960 0x7f6ffc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.169+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 >> 172.21.15.99:0/2095053693 conn(0x7f6ffc0fbe60 msgr2=0x7f6ffc0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:50.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.169+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 shutdown_connections 2023-08-25T08:58:50.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.169+0000 7f7000fff700 1 -- 172.21.15.99:0/2095053693 wait complete. 2023-08-25T08:58:50.188 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:58:50.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.385+0000 7f4c409ff700 1 -- 172.21.15.99:0/2106885843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c074960 msgr2=0x7f4c3c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.385+0000 7f4c409ff700 1 --2- 172.21.15.99:0/2106885843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c074960 0x7f4c3c072db0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0098f0 tx=0x7f4c2c009c00 comp rx=0 tx=0).stop 2023-08-25T08:58:50.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.385+0000 7f4c409ff700 1 -- 172.21.15.99:0/2106885843 shutdown_connections 2023-08-25T08:58:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.385+0000 7f4c409ff700 1 --2- 172.21.15.99:0/2106885843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c1087f0 0x7f4c3c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.385+0000 7f4c409ff700 1 --2- 172.21.15.99:0/2106885843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c074960 0x7f4c3c072db0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.385+0000 7f4c409ff700 1 --2- 172.21.15.99:0/2106885843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c073380 0x7f4c3c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.385+0000 7f4c409ff700 1 -- 172.21.15.99:0/2106885843 >> 172.21.15.99:0/2106885843 conn(0x7f4c3c0fbe60 msgr2=0x7f4c3c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:50.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.386+0000 7f4c409ff700 1 -- 172.21.15.99:0/2106885843 shutdown_connections 2023-08-25T08:58:50.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.386+0000 7f4c409ff700 1 -- 172.21.15.99:0/2106885843 wait complete. 2023-08-25T08:58:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c409ff700 1 Processor -- start 2023-08-25T08:58:50.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c409ff700 1 -- start start 2023-08-25T08:58:50.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c409ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c073380 0x7f4c3c103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:50.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c409ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c074960 0x7f4c3c104190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:50.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c409ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c1087f0 0x7f4c3c1007e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:50.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c409ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c100f80 con 0x7f4c3c074960 2023-08-25T08:58:50.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c409ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c1010f0 con 0x7f4c3c073380 2023-08-25T08:58:50.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c409ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c1012b0 con 0x7f4c3c1087f0 2023-08-25T08:58:50.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c39d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c074960 0x7f4c3c104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:50.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c3a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c073380 0x7f4c3c103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:50.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c39d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c074960 0x7f4c3c104190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41940/0 (socket says 172.21.15.99:41940) 2023-08-25T08:58:50.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c3ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c1087f0 0x7f4c3c1007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:50.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.387+0000 7f4c39d9b700 1 -- 172.21.15.99:0/1712789669 learned_addr learned my addr 172.21.15.99:0/1712789669 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:50.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c39d9b700 1 -- 172.21.15.99:0/1712789669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c1087f0 msgr2=0x7f4c3c1007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c39d9b700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c1087f0 0x7f4c3c1007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c39d9b700 1 -- 172.21.15.99:0/1712789669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c073380 msgr2=0x7f4c3c103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c39d9b700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c073380 0x7f4c3c103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c39d9b700 1 -- 172.21.15.99:0/1712789669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300095f0 con 0x7f4c3c074960 2023-08-25T08:58:50.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c39d9b700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c074960 0x7f4c3c104190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000ebf0 tx=0x7f4c3000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c2b7fe700 1 -- 172.21.15.99:0/1712789669 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30009a10 con 0x7f4c3c074960 2023-08-25T08:58:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c009520 con 0x7f4c3c074960 2023-08-25T08:58:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c2b7fe700 1 -- 172.21.15.99:0/1712789669 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c3000cc90 con 0x7f4c3c074960 2023-08-25T08:58:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c2b7fe700 1 -- 172.21.15.99:0/1712789669 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30018750 con 0x7f4c3c074960 2023-08-25T08:58:50.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.388+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c1a5110 con 0x7f4c3c074960 2023-08-25T08:58:50.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.389+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c106ea0 con 0x7f4c3c074960 2023-08-25T08:58:50.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.391+0000 7f4c2b7fe700 1 -- 172.21.15.99:0/1712789669 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f4c300188b0 con 0x7f4c3c074960 2023-08-25T08:58:50.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.392+0000 7f4c2b7fe700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c240a0020 0x7f4c240a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:50.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.393+0000 7f4c2b7fe700 1 -- 172.21.15.99:0/1712789669 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c30014070 con 0x7f4c3c074960 2023-08-25T08:58:50.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.393+0000 7f4c3a59c700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c240a0020 0x7f4c240a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:50.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.393+0000 7f4c3a59c700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c240a0020 0x7f4c240a24e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005e90 tx=0x7f4c2c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:50.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.396+0000 7f4c2b7fe700 1 -- 172.21.15.99:0/1712789669 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4c300955d0 con 0x7f4c3c074960 2023-08-25T08:58:50.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.643+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c3c10e370 con 0x7f4c3c074960 2023-08-25T08:58:50.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.644+0000 7f4c2b7fe700 1 -- 172.21.15.99:0/1712789669 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11283 (secure 0 0 0) 0x7f4c30098bf0 con 0x7f4c3c074960 2023-08-25T08:58:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:50.644 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T08:57:19.244738+0000","modified":"2023-08-25T08:57:25.030649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24547,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T08:58:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c240a0020 msgr2=0x7f4c240a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c240a0020 0x7f4c240a24e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005e90 tx=0x7f4c2c019040 comp rx=0 tx=0).stop 2023-08-25T08:58:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c074960 msgr2=0x7f4c3c104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c074960 0x7f4c3c104190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000ebf0 tx=0x7f4c3000ef00 comp rx=0 tx=0).stop 2023-08-25T08:58:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 shutdown_connections 2023-08-25T08:58:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c1087f0 0x7f4c3c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4c240a0020 0x7f4c240a24e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3c073380 0x7f4c3c103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 --2- 172.21.15.99:0/1712789669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c074960 0x7f4c3c104190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 >> 172.21.15.99:0/1712789669 conn(0x7f4c3c0fbe60 msgr2=0x7f4c3c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 shutdown_connections 2023-08-25T08:58:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.646+0000 7f4c409ff700 1 -- 172.21.15.99:0/1712789669 wait complete. 2023-08-25T08:58:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2023-08-25T08:58:50.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:58:50.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:50 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2602868727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:50.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:50 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2095053693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:50.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 -- 172.21.15.99:0/1004034208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c00100930 msgr2=0x7f0c00100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 --2- 172.21.15.99:0/1004034208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c00100930 0x7f0c00100d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0099c0 tx=0x7f0bfc009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 -- 172.21.15.99:0/1004034208 shutdown_connections 2023-08-25T08:58:50.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 --2- 172.21.15.99:0/1004034208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c001024c0 0x7f0c00108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 --2- 172.21.15.99:0/1004034208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c00100930 0x7f0c00100d50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 --2- 172.21.15.99:0/1004034208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c00101b20 0x7f0c00101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 -- 172.21.15.99:0/1004034208 >> 172.21.15.99:0/1004034208 conn(0x7f0c000fbe60 msgr2=0x7f0c000fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:50.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 -- 172.21.15.99:0/1004034208 shutdown_connections 2023-08-25T08:58:50.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.863+0000 7f0c07bff700 1 -- 172.21.15.99:0/1004034208 wait complete. 2023-08-25T08:58:50.867 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.864+0000 7f0c07bff700 1 Processor -- start 2023-08-25T08:58:50.867 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.864+0000 7f0c07bff700 1 -- start start 2023-08-25T08:58:50.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c07bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c00100930 0x7f0c001987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:50.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c07bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c00101b20 0x7f0c00198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:50.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c07bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c001024c0 0x7f0c0019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:50.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c07bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a0360 con 0x7f0c001024c0 2023-08-25T08:58:50.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c07bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a04a0 con 0x7f0c00100930 2023-08-25T08:58:50.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c07bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a0630 con 0x7f0c00101b20 2023-08-25T08:58:50.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0619c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c001024c0 0x7f0c0019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:50.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0619c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c001024c0 0x7f0c0019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41958/0 (socket says 172.21.15.99:41958) 2023-08-25T08:58:50.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0619c700 1 -- 172.21.15.99:0/3548073559 learned_addr learned my addr 172.21.15.99:0/3548073559 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:50.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0599b700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c00100930 0x7f0c001987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0519a700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c00101b20 0x7f0c00198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0619c700 1 -- 172.21.15.99:0/3548073559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c00101b20 msgr2=0x7f0c00198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0619c700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c00101b20 0x7f0c00198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0619c700 1 -- 172.21.15.99:0/3548073559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c00100930 msgr2=0x7f0c001987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0619c700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c00100930 0x7f0c001987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0619c700 1 -- 172.21.15.99:0/3548073559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc0095f0 con 0x7f0c001024c0 2023-08-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.865+0000 7f0c0519a700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c00101b20 0x7f0c00198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.866+0000 7f0c0619c700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c001024c0 0x7f0c0019fd90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800ef30 tx=0x7f0bf800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:50.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.866+0000 7f0bf6ffd700 1 -- 172.21.15.99:0/3548073559 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8018070 con 0x7f0c001024c0 2023-08-25T08:58:50.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.866+0000 7f0bf6ffd700 1 -- 172.21.15.99:0/3548073559 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf800f040 con 0x7f0c001024c0 2023-08-25T08:58:50.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.866+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c001a08b0 con 0x7f0c001024c0 2023-08-25T08:58:50.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.866+0000 7f0bf6ffd700 1 -- 172.21.15.99:0/3548073559 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800cdb0 con 0x7f0c001024c0 2023-08-25T08:58:50.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.866+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c001a0d70 con 0x7f0c001024c0 2023-08-25T08:58:50.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.867+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0010c190 con 0x7f0c001024c0 2023-08-25T08:58:50.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.869+0000 7f0bf6ffd700 1 -- 172.21.15.99:0/3548073559 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f0bf8003a00 con 0x7f0c001024c0 2023-08-25T08:58:50.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.870+0000 7f0bf6ffd700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0bec0a0020 0x7f0bec0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:50.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.870+0000 7f0bf6ffd700 1 -- 172.21.15.99:0/3548073559 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0bf80cb090 con 0x7f0c001024c0 2023-08-25T08:58:50.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.870+0000 7f0c0599b700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0bec0a0020 0x7f0bec0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:50.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.871+0000 7f0c0599b700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0bec0a0020 0x7f0bec0a24e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009990 tx=0x7f0bfc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:50.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:50.874+0000 7f0bf6ffd700 1 -- 172.21.15.99:0/3548073559 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0bf8094cd0 con 0x7f0c001024c0 2023-08-25T08:58:50.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:50 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2602868727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:50.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:50 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2095053693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:50.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:50 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2602868727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:50.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:50 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2095053693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.075+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c0010c3a0 con 0x7f0c001024c0 2023-08-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.077+0000 7f0bf6ffd700 1 -- 172.21.15.99:0/3548073559 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22646 (secure 0 0 0) 0x7f0bf80982f0 con 0x7f0c001024c0 2023-08-25T08:58:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:51.077 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:56.041917+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:58:51.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0bec0a0020 msgr2=0x7f0bec0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0bec0a0020 0x7f0bec0a24e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009990 tx=0x7f0bfc005fb0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c001024c0 msgr2=0x7f0c0019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c001024c0 0x7f0c0019fd90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800ef30 tx=0x7f0bf800c3a0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 shutdown_connections 2023-08-25T08:58:51.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c001024c0 0x7f0c0019fd90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f0bec0a0020 0x7f0bec0a24e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c00100930 0x7f0c001987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 --2- 172.21.15.99:0/3548073559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c00101b20 0x7f0c00198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.079+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 >> 172.21.15.99:0/3548073559 conn(0x7f0c000fbe60 msgr2=0x7f0c001073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:51.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.080+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 shutdown_connections 2023-08-25T08:58:51.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.080+0000 7f0c07bff700 1 -- 172.21.15.99:0/3548073559 wait complete. 2023-08-25T08:58:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T08:57:17.389400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T08:57:18.695578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T08:58:51.099 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T08:58:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 -- 172.21.15.99:0/2396561706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6381008d0 msgr2=0x7fd638100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/2396561706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6381008d0 0x7fd638100cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd6340099c0 tx=0x7fd634009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 -- 172.21.15.99:0/2396561706 shutdown_connections 2023-08-25T08:58:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/2396561706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd638102460 0x7fd638108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/2396561706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6381008d0 0x7fd638100cf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/2396561706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd638101ac0 0x7fd638101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 -- 172.21.15.99:0/2396561706 >> 172.21.15.99:0/2396561706 conn(0x7fd6380fbe40 msgr2=0x7fd6380fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 -- 172.21.15.99:0/2396561706 shutdown_connections 2023-08-25T08:58:51.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.299+0000 7fd63f5ff700 1 -- 172.21.15.99:0/2396561706 wait complete. 2023-08-25T08:58:51.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.300+0000 7fd63f5ff700 1 Processor -- start 2023-08-25T08:58:51.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.300+0000 7fd63f5ff700 1 -- start start 2023-08-25T08:58:51.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63f5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6381008d0 0x7fd6381987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:51.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63f5ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd638101ac0 0x7fd638198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:51.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63f5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd638102460 0x7fd63819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:51.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63f5ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381a0340 con 0x7fd638101ac0 2023-08-25T08:58:51.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63f5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381a0480 con 0x7fd6381008d0 2023-08-25T08:58:51.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63f5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381a0610 con 0x7fd638102460 2023-08-25T08:58:51.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63cb9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd638101ac0 0x7fd638198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:51.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63cb9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd638101ac0 0x7fd638198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41974/0 (socket says 172.21.15.99:41974) 2023-08-25T08:58:51.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63cb9a700 1 -- 172.21.15.99:0/863088139 learned_addr learned my addr 172.21.15.99:0/863088139 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:51.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63d39b700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6381008d0 0x7fd6381987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:51.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63db9c700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd638102460 0x7fd63819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:51.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63cb9a700 1 -- 172.21.15.99:0/863088139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd638102460 msgr2=0x7fd63819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63cb9a700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd638102460 0x7fd63819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63cb9a700 1 -- 172.21.15.99:0/863088139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6381008d0 msgr2=0x7fd6381987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63cb9a700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6381008d0 0x7fd6381987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63cb9a700 1 -- 172.21.15.99:0/863088139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6340095f0 con 0x7fd638101ac0 2023-08-25T08:58:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.301+0000 7fd63d39b700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6381008d0 0x7fd6381987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.302+0000 7fd63cb9a700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd638101ac0 0x7fd638198d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd62800e9a0 tx=0x7fd62800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.302+0000 7fd62e7fc700 1 -- 172.21.15.99:0/863088139 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628009790 con 0x7fd638101ac0 2023-08-25T08:58:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.302+0000 7fd62e7fc700 1 -- 172.21.15.99:0/863088139 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62800cbd0 con 0x7fd638101ac0 2023-08-25T08:58:51.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.302+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6381a08f0 con 0x7fd638101ac0 2023-08-25T08:58:51.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.302+0000 7fd62e7fc700 1 -- 172.21.15.99:0/863088139 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628018680 con 0x7fd638101ac0 2023-08-25T08:58:51.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.302+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6381a0e00 con 0x7fd638101ac0 2023-08-25T08:58:51.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.303+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd638192f20 con 0x7fd638101ac0 2023-08-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.305+0000 7fd62e7fc700 1 -- 172.21.15.99:0/863088139 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fd62800cd40 con 0x7fd638101ac0 2023-08-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.306+0000 7fd62e7fc700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd62409ffe0 0x7fd6240a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.307+0000 7fd62e7fc700 1 -- 172.21.15.99:0/863088139 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd628014070 con 0x7fd638101ac0 2023-08-25T08:58:51.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.307+0000 7fd63d39b700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd62409ffe0 0x7fd6240a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:51.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.307+0000 7fd63d39b700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd62409ffe0 0x7fd6240a24a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd634009990 tx=0x7fd63401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:51.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.310+0000 7fd62e7fc700 1 -- 172.21.15.99:0/863088139 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd628051e00 con 0x7fd638101ac0 2023-08-25T08:58:51.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.507+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd63810c310 con 0x7fd638101ac0 2023-08-25T08:58:51.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.507+0000 7fd62e7fc700 1 -- 172.21.15.99:0/863088139 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6280952b0 con 0x7fd638101ac0 2023-08-25T08:58:51.507 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T08:58:51.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd62409ffe0 msgr2=0x7fd6240a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd62409ffe0 0x7fd6240a24a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd634009990 tx=0x7fd63401a040 comp rx=0 tx=0).stop 2023-08-25T08:58:51.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd638101ac0 msgr2=0x7fd638198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd638101ac0 0x7fd638198d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd62800e9a0 tx=0x7fd62800ecb0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 shutdown_connections 2023-08-25T08:58:51.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd638102460 0x7fd63819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd62409ffe0 0x7fd6240a24a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6381008d0 0x7fd6381987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 --2- 172.21.15.99:0/863088139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd638101ac0 0x7fd638198d00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 >> 172.21.15.99:0/863088139 conn(0x7fd6380fbe40 msgr2=0x7fd6381073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:51.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 shutdown_connections 2023-08-25T08:58:51.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.510+0000 7fd63f5ff700 1 -- 172.21.15.99:0/863088139 wait complete. 2023-08-25T08:58:51.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T08:58:51.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.728+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2693213567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 msgr2=0x7ff5a4108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.728+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2693213567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 0x7ff5a4108d70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00b0d0 tx=0x7ff59c00b3e0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.729+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2693213567 shutdown_connections 2023-08-25T08:58:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.729+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2693213567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 0x7ff5a4108d70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.729+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2693213567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a4100930 0x7ff5a4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.729+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2693213567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5a4101b20 0x7ff5a4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.729+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2693213567 >> 172.21.15.99:0/2693213567 conn(0x7ff5a40fbea0 msgr2=0x7ff5a40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.729+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2693213567 shutdown_connections 2023-08-25T08:58:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.729+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2693213567 wait complete. 2023-08-25T08:58:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5aa3ff700 1 Processor -- start 2023-08-25T08:58:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5aa3ff700 1 -- start start 2023-08-25T08:58:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5aa3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5a4100930 0x7ff5a4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5aa3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a4101b20 0x7ff5a4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5aa3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 0x7ff5a419ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5aa3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41a04e0 con 0x7ff5a41024c0 2023-08-25T08:58:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5aa3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41a0620 con 0x7ff5a4101b20 2023-08-25T08:58:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5aa3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41a07b0 con 0x7ff5a4100930 2023-08-25T08:58:51.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.730+0000 7ff5a899c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 0x7ff5a419ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:51.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a899c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 0x7ff5a419ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41990/0 (socket says 172.21.15.99:41990) 2023-08-25T08:58:51.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a899c700 1 -- 172.21.15.99:0/2991064185 learned_addr learned my addr 172.21.15.99:0/2991064185 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a3fff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5a4100930 0x7ff5a4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a37fe700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a4101b20 0x7ff5a4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a899c700 1 -- 172.21.15.99:0/2991064185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5a4100930 msgr2=0x7ff5a4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a899c700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5a4100930 0x7ff5a4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a899c700 1 -- 172.21.15.99:0/2991064185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a4101b20 msgr2=0x7ff5a4198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a899c700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a4101b20 0x7ff5a4198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a899c700 1 -- 172.21.15.99:0/2991064185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c009d00 con 0x7ff5a41024c0 2023-08-25T08:58:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a899c700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 0x7ff5a419ff10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff59c007a00 tx=0x7ff59c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a17fa700 1 -- 172.21.15.99:0/2991064185 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00e040 con 0x7ff5a41024c0 2023-08-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a17fa700 1 -- 172.21.15.99:0/2991064185 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff59c01dd50 con 0x7ff5a41024c0 2023-08-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a41a0a30 con 0x7ff5a41024c0 2023-08-25T08:58:51.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.731+0000 7ff5a17fa700 1 -- 172.21.15.99:0/2991064185 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c01b450 con 0x7ff5a41024c0 2023-08-25T08:58:51.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.732+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a41a0f00 con 0x7ff5a41024c0 2023-08-25T08:58:51.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.732+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a410c1a0 con 0x7ff5a41024c0 2023-08-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.734+0000 7ff5a17fa700 1 -- 172.21.15.99:0/2991064185 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ff59c012570 con 0x7ff5a41024c0 2023-08-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.736+0000 7ff5a17fa700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff58c0a0060 0x7ff58c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.736+0000 7ff5a17fa700 1 -- 172.21.15.99:0/2991064185 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff59c0cd170 con 0x7ff5a41024c0 2023-08-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.736+0000 7ff5a3fff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff58c0a0060 0x7ff58c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:51.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.736+0000 7ff5a3fff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff58c0a0060 0x7ff58c0a2520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4101980 tx=0x7ff59400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:51.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.739+0000 7ff5a17fa700 1 -- 172.21.15.99:0/2991064185 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff59c096db0 con 0x7ff5a41024c0 2023-08-25T08:58:51.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:51 smithi099 ceph-mon[97641]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:51 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1712789669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:51.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:51 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3548073559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:51.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:51 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/863088139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:51 smithi157 ceph-mon[130879]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:51 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1712789669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:51.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:51 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3548073559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:51.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:51 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/863088139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:51.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.938+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5a41a10f0 con 0x7ff5a41024c0 2023-08-25T08:58:51.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.938+0000 7ff5a17fa700 1 -- 172.21.15.99:0/2991064185 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff59c09a3d0 con 0x7ff5a41024c0 2023-08-25T08:58:51.939 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-08-25T08:58:51.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff58c0a0060 msgr2=0x7ff58c0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff58c0a0060 0x7ff58c0a2520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4101980 tx=0x7ff59400b410 comp rx=0 tx=0).stop 2023-08-25T08:58:51.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 msgr2=0x7ff5a419ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:51.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 0x7ff5a419ff10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff59c007a00 tx=0x7ff59c009440 comp rx=0 tx=0).stop 2023-08-25T08:58:51.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 shutdown_connections 2023-08-25T08:58:51.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a41024c0 0x7ff5a419ff10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5a4100930 0x7ff5a4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7ff58c0a0060 0x7ff58c0a2520 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 --2- 172.21.15.99:0/2991064185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a4101b20 0x7ff5a4198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:51.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 >> 172.21.15.99:0/2991064185 conn(0x7ff5a40fbea0 msgr2=0x7ff5a40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:51.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 shutdown_connections 2023-08-25T08:58:51.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:51.941+0000 7ff5aa3ff700 1 -- 172.21.15.99:0/2991064185 wait complete. 2023-08-25T08:58:51.962 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T08:58:51.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:51 smithi181 ceph-mon[131296]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:51.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:51 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1712789669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:51.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:51 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3548073559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:51.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:51 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/863088139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T08:58:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 -- 172.21.15.99:0/3732053418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194101b00 msgr2=0x7fd194101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 --2- 172.21.15.99:0/3732053418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194101b00 0x7fd194101f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd1840099c0 tx=0x7fd184009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 -- 172.21.15.99:0/3732053418 shutdown_connections 2023-08-25T08:58:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 --2- 172.21.15.99:0/3732053418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1941024a0 0x7fd194108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 --2- 172.21.15.99:0/3732053418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd194100910 0x7fd194100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 --2- 172.21.15.99:0/3732053418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194101b00 0x7fd194101f60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 -- 172.21.15.99:0/3732053418 >> 172.21.15.99:0/3732053418 conn(0x7fd1940fbe60 msgr2=0x7fd1940fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 -- 172.21.15.99:0/3732053418 shutdown_connections 2023-08-25T08:58:52.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.162+0000 7fd19bdff700 1 -- 172.21.15.99:0/3732053418 wait complete. 2023-08-25T08:58:52.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.163+0000 7fd19bdff700 1 Processor -- start 2023-08-25T08:58:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.163+0000 7fd19bdff700 1 -- start start 2023-08-25T08:58:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19bdff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194100910 0x7fd194196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19bdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd194101b00 0x7fd194196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:52.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19bdff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1941024a0 0x7fd19419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19bdff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19419e1e0 con 0x7fd194100910 2023-08-25T08:58:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19bdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19419e320 con 0x7fd194101b00 2023-08-25T08:58:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19bdff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19419e4b0 con 0x7fd1941024a0 2023-08-25T08:58:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194100910 0x7fd194196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194100910 0x7fd194196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42006/0 (socket says 172.21.15.99:42006) 2023-08-25T08:58:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 -- 172.21.15.99:0/2341077276 learned_addr learned my addr 172.21.15.99:0/2341077276 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:52.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19939a700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd194101b00 0x7fd194196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19a39c700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1941024a0 0x7fd19419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 -- 172.21.15.99:0/2341077276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1941024a0 msgr2=0x7fd19419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1941024a0 0x7fd19419dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 -- 172.21.15.99:0/2341077276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd194101b00 msgr2=0x7fd194196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd194101b00 0x7fd194196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 -- 172.21.15.99:0/2341077276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1840095f0 con 0x7fd194100910 2023-08-25T08:58:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd19939a700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd194101b00 0x7fd194196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:52.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.164+0000 7fd199b9b700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194100910 0x7fd194196660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e970 tx=0x7fd19000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.165+0000 7fd18affd700 1 -- 172.21.15.99:0/2341077276 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190009790 con 0x7fd194100910 2023-08-25T08:58:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.165+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19419e790 con 0x7fd194100910 2023-08-25T08:58:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.165+0000 7fd18affd700 1 -- 172.21.15.99:0/2341077276 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1900044e0 con 0x7fd194100910 2023-08-25T08:58:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.165+0000 7fd18affd700 1 -- 172.21.15.99:0/2341077276 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190003e30 con 0x7fd194100910 2023-08-25T08:58:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.165+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19419ecf0 con 0x7fd194100910 2023-08-25T08:58:52.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.166+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd194107890 con 0x7fd194100910 2023-08-25T08:58:52.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.168+0000 7fd18affd700 1 -- 172.21.15.99:0/2341077276 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fd190004000 con 0x7fd194100910 2023-08-25T08:58:52.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.169+0000 7fd18affd700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd18009ffe0 0x7fd1800a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:52.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.169+0000 7fd18affd700 1 -- 172.21.15.99:0/2341077276 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd190014070 con 0x7fd194100910 2023-08-25T08:58:52.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.169+0000 7fd19939a700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd18009ffe0 0x7fd1800a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:52.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.170+0000 7fd19939a700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd18009ffe0 0x7fd1800a24a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b580 tx=0x7fd18401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:52.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.171+0000 7fd18affd700 1 -- 172.21.15.99:0/2341077276 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd190095470 con 0x7fd194100910 2023-08-25T08:58:52.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.372+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd19402cd00 con 0x7fd194100910 2023-08-25T08:58:52.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.372+0000 7fd18affd700 1 -- 172.21.15.99:0/2341077276 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd190098a90 con 0x7fd194100910 2023-08-25T08:58:52.374 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-08-25T08:58:52.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd18009ffe0 msgr2=0x7fd1800a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd18009ffe0 0x7fd1800a24a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b580 tx=0x7fd18401a040 comp rx=0 tx=0).stop 2023-08-25T08:58:52.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194100910 msgr2=0x7fd194196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194100910 0x7fd194196660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e970 tx=0x7fd19000ed30 comp rx=0 tx=0).stop 2023-08-25T08:58:52.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 shutdown_connections 2023-08-25T08:58:52.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1941024a0 0x7fd19419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd194100910 0x7fd194196660 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7fd18009ffe0 0x7fd1800a24a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 --2- 172.21.15.99:0/2341077276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd194101b00 0x7fd194196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 >> 172.21.15.99:0/2341077276 conn(0x7fd1940fbe60 msgr2=0x7fd1940fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:52.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 shutdown_connections 2023-08-25T08:58:52.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.375+0000 7fd19bdff700 1 -- 172.21.15.99:0/2341077276 wait complete. 2023-08-25T08:58:52.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T08:58:52.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.592+0000 7f308d1ff700 1 -- 172.21.15.99:0/4248306131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3088073320 msgr2=0x7f30880737a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.592+0000 7f308d1ff700 1 --2- 172.21.15.99:0/4248306131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3088073320 0x7f30880737a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f307c009990 tx=0x7f307c009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.592+0000 7f308d1ff700 1 -- 172.21.15.99:0/4248306131 shutdown_connections 2023-08-25T08:58:52.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.592+0000 7f308d1ff700 1 --2- 172.21.15.99:0/4248306131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30881087f0 0x7f308810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.592+0000 7f308d1ff700 1 --2- 172.21.15.99:0/4248306131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3088074900 0x7f3088072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.592+0000 7f308d1ff700 1 --2- 172.21.15.99:0/4248306131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3088073320 0x7f30880737a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.592+0000 7f308d1ff700 1 -- 172.21.15.99:0/4248306131 >> 172.21.15.99:0/4248306131 conn(0x7f30880fbe70 msgr2=0x7f30880fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:52.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.592+0000 7f308d1ff700 1 -- 172.21.15.99:0/4248306131 shutdown_connections 2023-08-25T08:58:52.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.593+0000 7f308d1ff700 1 -- 172.21.15.99:0/4248306131 wait complete. 2023-08-25T08:58:52.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.593+0000 7f308d1ff700 1 Processor -- start 2023-08-25T08:58:52.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308d1ff700 1 -- start start 2023-08-25T08:58:52.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308d1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3088073320 0x7f30881987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:52.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308d1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3088074900 0x7f3088198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:52.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308d1ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30881087f0 0x7f308819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:52.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308d1ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881a0350 con 0x7f30881087f0 2023-08-25T08:58:52.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308d1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881a0490 con 0x7f3088073320 2023-08-25T08:58:52.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308d1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881a0620 con 0x7f3088074900 2023-08-25T08:58:52.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30881087f0 0x7f308819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:52.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30881087f0 0x7f308819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42026/0 (socket says 172.21.15.99:42026) 2023-08-25T08:58:52.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308759e700 1 -- 172.21.15.99:0/3978315386 learned_addr learned my addr 172.21.15.99:0/3978315386 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:52.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f3086d9d700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3088073320 0x7f30881987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:52.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.594+0000 7f308659c700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3088074900 0x7f3088198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:52.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f308759e700 1 -- 172.21.15.99:0/3978315386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3088074900 msgr2=0x7f3088198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f308759e700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3088074900 0x7f3088198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f308759e700 1 -- 172.21.15.99:0/3978315386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3088073320 msgr2=0x7f30881987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f308759e700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3088073320 0x7f30881987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f308759e700 1 -- 172.21.15.99:0/3978315386 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0095f0 con 0x7f30881087f0 2023-08-25T08:58:52.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f3086d9d700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3088073320 0x7f30881987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:52.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f308759e700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30881087f0 0x7f308819fd80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f308000ef30 tx=0x7f308000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:52.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f306ffff700 1 -- 172.21.15.99:0/3978315386 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080018070 con 0x7f30881087f0 2023-08-25T08:58:52.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f306ffff700 1 -- 172.21.15.99:0/3978315386 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f308000f040 con 0x7f30881087f0 2023-08-25T08:58:52.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30881a08a0 con 0x7f30881087f0 2023-08-25T08:58:52.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f306ffff700 1 -- 172.21.15.99:0/3978315386 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000cdb0 con 0x7f30881087f0 2023-08-25T08:58:52.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.595+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30881a0db0 con 0x7f30881087f0 2023-08-25T08:58:52.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.596+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3088106ef0 con 0x7f30881087f0 2023-08-25T08:58:52.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.598+0000 7f306ffff700 1 -- 172.21.15.99:0/3978315386 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f3080003a00 con 0x7f30881087f0 2023-08-25T08:58:52.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.599+0000 7f306ffff700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f30700a00a0 0x7f30700a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:52.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.600+0000 7f306ffff700 1 -- 172.21.15.99:0/3978315386 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f30800cafb0 con 0x7f30881087f0 2023-08-25T08:58:52.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.600+0000 7f3086d9d700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f30700a00a0 0x7f30700a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:52.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.600+0000 7f3086d9d700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f30700a00a0 0x7f30700a2560 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3078009cb0 tx=0x7f3078009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:52.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.603+0000 7f306ffff700 1 -- 172.21.15.99:0/3978315386 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f30800fde00 con 0x7f30881087f0 2023-08-25T08:58:52.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:52 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2991064185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:52.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:52 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2341077276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:52.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.849+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f308810e360 con 0x7f30881087f0 2023-08-25T08:58:52.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.849+0000 7f306ffff700 1 -- 172.21.15.99:0/3978315386 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11283 (secure 0 0 0) 0x7f3080094c50 con 0x7f30881087f0 2023-08-25T08:58:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:52.850 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T08:57:19.244738+0000","modified":"2023-08-25T08:57:25.030649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24547,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/135740173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":135740173},{"type":"v1","addr":"172.21.15.157:6837","nonce":135740173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T08:58:52.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f30700a00a0 msgr2=0x7f30700a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f30700a00a0 0x7f30700a2560 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3078009cb0 tx=0x7f3078009260 comp rx=0 tx=0).stop 2023-08-25T08:58:52.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30881087f0 msgr2=0x7f308819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:52.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30881087f0 0x7f308819fd80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f308000ef30 tx=0x7f308000c3a0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 shutdown_connections 2023-08-25T08:58:52.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30881087f0 0x7f308819fd80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f30700a00a0 0x7f30700a2560 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3088073320 0x7f30881987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 --2- 172.21.15.99:0/3978315386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3088074900 0x7f3088198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:52.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 >> 172.21.15.99:0/3978315386 conn(0x7f30880fbe70 msgr2=0x7f30880fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:52.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 shutdown_connections 2023-08-25T08:58:52.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:52.852+0000 7f308d1ff700 1 -- 172.21.15.99:0/3978315386 wait complete. 2023-08-25T08:58:52.857 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2023-08-25T08:58:52.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T08:58:52.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:52 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2991064185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:52.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:52 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2341077276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:52.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:52 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2991064185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T08:58:52.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:52 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2341077276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T08:58:53.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 -- 172.21.15.99:0/2869513700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 msgr2=0x7f81e0101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 --2- 172.21.15.99:0/2869513700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 0x7f81e0101f80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f81d40099c0 tx=0x7f81d4009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 -- 172.21.15.99:0/2869513700 shutdown_connections 2023-08-25T08:58:53.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 --2- 172.21.15.99:0/2869513700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e01024c0 0x7f81e0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 --2- 172.21.15.99:0/2869513700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e0100930 0x7f81e0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 --2- 172.21.15.99:0/2869513700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 0x7f81e0101f80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 -- 172.21.15.99:0/2869513700 >> 172.21.15.99:0/2869513700 conn(0x7f81e00fbe60 msgr2=0x7f81e00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 -- 172.21.15.99:0/2869513700 shutdown_connections 2023-08-25T08:58:53.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.069+0000 7f81e4dff700 1 -- 172.21.15.99:0/2869513700 wait complete. 2023-08-25T08:58:53.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.070+0000 7f81e4dff700 1 Processor -- start 2023-08-25T08:58:53.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.070+0000 7f81e4dff700 1 -- start start 2023-08-25T08:58:53.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81e4dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e0100930 0x7f81e00746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81e4dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 0x7f81e0072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81e4dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e01024c0 0x7f81e0073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81e4dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00739f0 con 0x7f81e0101b20 2023-08-25T08:58:53.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81e4dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e0073b60 con 0x7f81e0100930 2023-08-25T08:58:53.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81e4dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e01a05e0 con 0x7f81e01024c0 2023-08-25T08:58:53.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 0x7f81e0072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 0x7f81e0072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42044/0 (socket says 172.21.15.99:42044) 2023-08-25T08:58:53.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ddd9b700 1 -- 172.21.15.99:0/1068334643 learned_addr learned my addr 172.21.15.99:0/1068334643 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81de59c700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e0100930 0x7f81e00746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ded9d700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e01024c0 0x7f81e0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ddd9b700 1 -- 172.21.15.99:0/1068334643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e01024c0 msgr2=0x7f81e0073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ddd9b700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e01024c0 0x7f81e0073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ddd9b700 1 -- 172.21.15.99:0/1068334643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e0100930 msgr2=0x7f81e00746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ddd9b700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e0100930 0x7f81e00746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81ddd9b700 1 -- 172.21.15.99:0/1068334643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d40095f0 con 0x7f81e0101b20 2023-08-25T08:58:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.071+0000 7f81de59c700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e0100930 0x7f81e00746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.072+0000 7f81ddd9b700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 0x7f81e0072d10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009fd0 tx=0x7f81d400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.072+0000 7f81cf7fe700 1 -- 172.21.15.99:0/1068334643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d401d070 con 0x7f81e0101b20 2023-08-25T08:58:53.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.072+0000 7f81cf7fe700 1 -- 172.21.15.99:0/1068334643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81d4005030 con 0x7f81e0101b20 2023-08-25T08:58:53.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.072+0000 7f81cf7fe700 1 -- 172.21.15.99:0/1068334643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4004b30 con 0x7f81e0101b20 2023-08-25T08:58:53.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.072+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e01a0860 con 0x7f81e0101b20 2023-08-25T08:58:53.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.072+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e01a0c30 con 0x7f81e0101b20 2023-08-25T08:58:53.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.073+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e010c140 con 0x7f81e0101b20 2023-08-25T08:58:53.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.075+0000 7f81cf7fe700 1 -- 172.21.15.99:0/1068334643 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f81d4004cd0 con 0x7f81e0101b20 2023-08-25T08:58:53.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.076+0000 7f81cf7fe700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f81c80a00a0 0x7f81c80a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.076+0000 7f81cf7fe700 1 -- 172.21.15.99:0/1068334643 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f81d40cc180 con 0x7f81e0101b20 2023-08-25T08:58:53.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.076+0000 7f81de59c700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f81c80a00a0 0x7f81c80a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.077+0000 7f81de59c700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f81c80a00a0 0x7f81c80a2560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f81e0101980 tx=0x7f81d0007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.080+0000 7f81cf7fe700 1 -- 172.21.15.99:0/1068334643 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f81d4095e20 con 0x7f81e0101b20 2023-08-25T08:58:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.280+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81e01077c0 con 0x7f81e0101b20 2023-08-25T08:58:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.282+0000 7f81cf7fe700 1 -- 172.21.15.99:0/1068334643 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22646 (secure 0 0 0) 0x7f81d4099440 con 0x7f81e0101b20 2023-08-25T08:58:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T08:58:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T08:57:56.041917+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T08:58:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f81c80a00a0 msgr2=0x7f81c80a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f81c80a00a0 0x7f81c80a2560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f81e0101980 tx=0x7f81d0007400 comp rx=0 tx=0).stop 2023-08-25T08:58:53.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 msgr2=0x7f81e0072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 0x7f81e0072d10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009fd0 tx=0x7f81d400f690 comp rx=0 tx=0).stop 2023-08-25T08:58:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 shutdown_connections 2023-08-25T08:58:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e01024c0 0x7f81e0073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f81c80a00a0 0x7f81c80a2560 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e0100930 0x7f81e00746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 --2- 172.21.15.99:0/1068334643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81e0101b20 0x7f81e0072d10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 >> 172.21.15.99:0/1068334643 conn(0x7f81e00fbe60 msgr2=0x7f81e00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 shutdown_connections 2023-08-25T08:58:53.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.284+0000 7f81e4dff700 1 -- 172.21.15.99:0/1068334643 wait complete. 2023-08-25T08:58:53.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-08-25T08:57:19.244738+0000', 'modified': '2023-08-25T08:57:25.030649+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24593, 'mds_1': 24547, 'mds_2': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/1963370106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 1963370106}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 1963370106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.181:6835/3995966648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3995966648}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 3995966648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6837/2291370984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2291370984}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2291370984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24545': {'gid': 24545, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/135740173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 135740173}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 135740173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/2903255438', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 2903255438}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 2903255438}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6839/1062424301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1062424301}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1062424301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T08:58:53.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-08-25T08:58:53.306 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-08-25T08:58:53.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 -- 172.21.15.99:0/728716115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8101b20 msgr2=0x7f60d8101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 --2- 172.21.15.99:0/728716115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8101b20 0x7f60d8101f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f60c00099c0 tx=0x7f60c0009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 -- 172.21.15.99:0/728716115 shutdown_connections 2023-08-25T08:58:53.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 --2- 172.21.15.99:0/728716115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d81024c0 0x7f60d8108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 --2- 172.21.15.99:0/728716115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d8100930 0x7f60d8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 --2- 172.21.15.99:0/728716115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8101b20 0x7f60d8101f80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 -- 172.21.15.99:0/728716115 >> 172.21.15.99:0/728716115 conn(0x7f60d80fbea0 msgr2=0x7f60d80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 -- 172.21.15.99:0/728716115 shutdown_connections 2023-08-25T08:58:53.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.503+0000 7f60dcbff700 1 -- 172.21.15.99:0/728716115 wait complete. 2023-08-25T08:58:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.504+0000 7f60dcbff700 1 Processor -- start 2023-08-25T08:58:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.504+0000 7f60dcbff700 1 -- start start 2023-08-25T08:58:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.504+0000 7f60dcbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8100930 0x7f60d81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60dcbff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d8101b20 0x7f60d8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60dcbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d81024c0 0x7f60d819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60dcbff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81a0340 con 0x7f60d8101b20 2023-08-25T08:58:53.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60dcbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81a0480 con 0x7f60d8100930 2023-08-25T08:58:53.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60dcbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81a0610 con 0x7f60d81024c0 2023-08-25T08:58:53.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d8101b20 0x7f60d8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60d659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8100930 0x7f60d81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d8101b20 0x7f60d8198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42060/0 (socket says 172.21.15.99:42060) 2023-08-25T08:58:53.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 -- 172.21.15.99:0/2965125450 learned_addr learned my addr 172.21.15.99:0/2965125450 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d81024c0 0x7f60d819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 -- 172.21.15.99:0/2965125450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d81024c0 msgr2=0x7f60d819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d81024c0 0x7f60d819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 -- 172.21.15.99:0/2965125450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8100930 msgr2=0x7f60d81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8100930 0x7f60d81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 -- 172.21.15.99:0/2965125450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c00095f0 con 0x7f60d8101b20 2023-08-25T08:58:53.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60d659c700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8100930 0x7f60d81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:53.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.505+0000 7f60cffff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d8101b20 0x7f60d8198d00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009fd0 tx=0x7f60c000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.506+0000 7f60cf7fe700 1 -- 172.21.15.99:0/2965125450 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c001d070 con 0x7f60d8101b20 2023-08-25T08:58:53.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.506+0000 7f60dcbff700 1 -- 172.21.15.99:0/2965125450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d81a0890 con 0x7f60d8101b20 2023-08-25T08:58:53.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.506+0000 7f60cf7fe700 1 -- 172.21.15.99:0/2965125450 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c0005030 con 0x7f60d8101b20 2023-08-25T08:58:53.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.506+0000 7f60cf7fe700 1 -- 172.21.15.99:0/2965125450 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0004b30 con 0x7f60d8101b20 2023-08-25T08:58:53.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.506+0000 7f60dcbff700 1 -- 172.21.15.99:0/2965125450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d81a0d10 con 0x7f60d8101b20 2023-08-25T08:58:53.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.509+0000 7f60cf7fe700 1 -- 172.21.15.99:0/2965125450 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f60c0004cd0 con 0x7f60d8101b20 2023-08-25T08:58:53.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.510+0000 7f60f056cb80 1 Processor -- start 2023-08-25T08:58:53.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.510+0000 7f60f056cb80 1 -- start start 2023-08-25T08:58:53.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.510+0000 7f60cf7fe700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60c409ffe0 0x7f60c40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.510+0000 7f60cf7fe700 1 -- 172.21.15.99:0/2965125450 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f60c00cc270 con 0x7f60d8101b20 2023-08-25T08:58:53.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.510+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60c409ffe0 0x7f60c40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.510+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9af95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.510+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9af9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9afa510 0x55eef9afa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60f056cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef9a58760 con 0x55eef9af9af0 2023-08-25T08:58:53.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60f056cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef9a49730 con 0x55eef9af9190 2023-08-25T08:58:53.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60f056cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef9a47050 con 0x55eef9afa510 2023-08-25T08:58:53.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9af9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9af9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42062/0 (socket says 172.21.15.99:42062) 2023-08-25T08:58:53.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 -- 172.21.15.99:0/3847949359 learned_addr learned my addr 172.21.15.99:0/3847949359 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60d659c700 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9af95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9afa510 0x55eef9afa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 -- 172.21.15.99:0/3847949359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9afa510 msgr2=0x55eef9afa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9afa510 0x55eef9afa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 -- 172.21.15.99:0/3847949359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 msgr2=0x55eef9af95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9af95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 -- 172.21.15.99:0/3847949359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eef98c25e0 con 0x55eef9af9af0 2023-08-25T08:58:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9afa510 0x55eef9afa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60d659c700 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9af95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60c409ffe0 0x7f60c40a24a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f60d000f870 tx=0x7f60d000fc30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.511+0000 7f60cffff700 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9af9f30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60c00a13c0 tx=0x7f60c009a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa589a9a34bd9317 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60ccff9700 1 -- 172.21.15.99:0/3847949359 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c009d9a0 con 0x55eef9af9af0 2023-08-25T08:58:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60ccff9700 1 -- 172.21.15.99:0/3847949359 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c00a18f0 con 0x55eef9af9af0 2023-08-25T08:58:53.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60ccff9700 1 -- 172.21.15.99:0/3847949359 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c009a0a0 con 0x55eef9af9af0 2023-08-25T08:58:53.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60f056cb80 1 -- 172.21.15.99:0/3847949359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 msgr2=0x55eef9af9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60f056cb80 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9af9f30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60c00a13c0 tx=0x7f60c009a270 comp rx=0 tx=0).stop 2023-08-25T08:58:53.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60f056cb80 1 -- 172.21.15.99:0/3847949359 shutdown_connections 2023-08-25T08:58:53.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60f056cb80 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9afa510 0x55eef9afa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60f056cb80 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9af95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60f056cb80 1 --2- 172.21.15.99:0/3847949359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9af9f30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.512+0000 7f60f056cb80 1 -- 172.21.15.99:0/3847949359 >> 172.21.15.99:0/3847949359 conn(0x55eef9a65090 msgr2=0x55eef9aeb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.513+0000 7f60f056cb80 1 -- 172.21.15.99:0/3847949359 shutdown_connections 2023-08-25T08:58:53.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.513+0000 7f60f056cb80 1 -- 172.21.15.99:0/3847949359 wait complete. 2023-08-25T08:58:53.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.513+0000 7f60f056cb80 1 -- start start 2023-08-25T08:58:53.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.513+0000 7f60f056cb80 1 Processor -- start 2023-08-25T08:58:53.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9190 0x55eef9b02840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 0x55eef9b02d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60f056cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef9a42940 con 0x55eef9af9190 2023-08-25T08:58:53.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60f056cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef99e0b60 con 0x55eef9afa510 2023-08-25T08:58:53.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60f056cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef99e0860 con 0x55eef9af9af0 2023-08-25T08:58:53.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60f056cb80 10 client.0 _ll_register_callbacks cb 0x55eef992aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:53.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9190 0x55eef9b02840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9190 0x55eef9b02840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42066/0 (socket says 172.21.15.99:42066) 2023-08-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d659c700 1 -- 172.21.15.99:0/473626652 learned_addr learned my addr 172.21.15.99:0/473626652 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60cffff700 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 0x55eef9b02d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d659c700 1 -- 172.21.15.99:0/473626652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 msgr2=0x55eef9b02d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d659c700 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 0x55eef9b02d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d659c700 1 -- 172.21.15.99:0/473626652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 msgr2=0x55eef9b09dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d659c700 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.514+0000 7f60d659c700 1 -- 172.21.15.99:0/473626652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c8007c80 con 0x55eef9af9190 2023-08-25T08:58:53.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60d659c700 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9190 0x55eef9b02840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f60c8011920 tx=0x7f60c8011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4c9fa2b7d75152 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:58:53.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60bdffb700 1 -- 172.21.15.99:0/473626652 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8016070 con 0x55eef9af9190 2023-08-25T08:58:53.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60f056cb80 1 -- 172.21.15.99:0/473626652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eef9ac0170 con 0x55eef9af9190 2023-08-25T08:58:53.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60bdffb700 1 -- 172.21.15.99:0/473626652 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f60c8010980 con 0x55eef9af9190 2023-08-25T08:58:53.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60bdffb700 1 -- 172.21.15.99:0/473626652 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60c8010b40 con 0x55eef9af9190 2023-08-25T08:58:53.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60f056cb80 10 client.15084 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:53.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60f056cb80 1 -- 172.21.15.99:0/473626652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f60d81a0d10 con 0x55eef9af9190 2023-08-25T08:58:53.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60bdffb700 1 -- 172.21.15.99:0/473626652 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eef9ac0170 con 0x55eef9af9190 2023-08-25T08:58:53.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.515+0000 7f60bdffb700 1 -- 172.21.15.99:0/473626652 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60c8024050 con 0x55eef9af9190 2023-08-25T08:58:53.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60f056cb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:53.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60f056cb80 10 client.15084 resolve_mds: resolved name 'f' to daemon mds.f 2023-08-25T08:58:53.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60f056cb80 20 client.15084 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:53.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60f056cb80 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55eef9b0e400 0x55eef9b0cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60f056cb80 4 client.15084 mds_command: new command op to 24517 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:58:53.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60f056cb80 1 -- 172.21.15.99:0/473626652 --> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eef99e8b50 con 0x55eef9b0e400 2023-08-25T08:58:53.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60cffff700 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55eef9b0e400 0x55eef9b0cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60cffff700 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55eef9b0e400 0x55eef9b0cf20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f75d6dde5051eb0 server_cookie=142d0bd462137f46 in_seq=0 out_seq=0 2023-08-25T08:58:53.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.516+0000 7f60bdffb700 10 client.15084 ms_handle_connect on v2:172.21.15.181:6834/3995966648 2023-08-25T08:58:53.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60bdffb700 1 -- 172.21.15.99:0/473626652 <== mds.2 v2:172.21.15.181:6834/3995966648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eef99e8b50 con 0x55eef9b0e400 2023-08-25T08:58:53.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60bdffb700 10 client.15084 handle_command_reply: tid=0 2023-08-25T08:58:53.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 client.15084 shutdown 2023-08-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 -- 172.21.15.99:0/473626652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9190 msgr2=0x55eef9b02840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9190 0x55eef9b02840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f60c8011920 tx=0x7f60c8011c30 comp rx=0 tx=0).stop 2023-08-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 -- 172.21.15.99:0/473626652 shutdown_connections 2023-08-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9190 0x55eef9b02840 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55eef9b0e400 0x55eef9b0cf20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 --2- 172.21.15.99:0/473626652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 0x55eef9b02d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60f056cb80 1 -- 172.21.15.99:0/473626652 >> 172.21.15.99:0/473626652 conn(0x55eef9a65090 msgr2=0x55eef9ae5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.517+0000 7f60bdffb700 0 client.15084 ms_handle_reset on v2:172.21.15.181:6834/3995966648 2023-08-25T08:58:53.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.518+0000 7f60f056cb80 1 -- 172.21.15.99:0/473626652 shutdown_connections 2023-08-25T08:58:53.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.518+0000 7f60f056cb80 1 -- 172.21.15.99:0/473626652 wait complete. 2023-08-25T08:58:53.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.518+0000 7f60f056cb80 20 client.15084 trim_cache size 0 max 16384 2023-08-25T08:58:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.536+0000 7f60f056cb80 1 Processor -- start 2023-08-25T08:58:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.536+0000 7f60f056cb80 1 -- start start 2023-08-25T08:58:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.536+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9aebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.536+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 0x55eef9b09d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.536+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9afa510 0x55eef9aeb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60f056cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef98c48d0 con 0x55eef9afa510 2023-08-25T08:58:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60f056cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef98d0e10 con 0x55eef9af9190 2023-08-25T08:58:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60f056cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef99dce00 con 0x55eef9af9af0 2023-08-25T08:58:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9afa510 0x55eef9aeb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9afa510 0x55eef9aeb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42082/0 (socket says 172.21.15.99:42082) 2023-08-25T08:58:53.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 -- 172.21.15.99:0/347346238 learned_addr learned my addr 172.21.15.99:0/347346238 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d659c700 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9aebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60cffff700 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 0x55eef9b09d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 -- 172.21.15.99:0/347346238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 msgr2=0x55eef9b09d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 0x55eef9b09d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 -- 172.21.15.99:0/347346238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 msgr2=0x55eef9aebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9aebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 -- 172.21.15.99:0/347346238 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eef986ed90 con 0x55eef9afa510 2023-08-25T08:58:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d659c700 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9aebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.537+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9afa510 0x55eef9aeb1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009130 tx=0x7f60d001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d2011ba5487e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60cd7fa700 1 -- 172.21.15.99:0/347346238 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d000e040 con 0x55eef9afa510 2023-08-25T08:58:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60cd7fa700 1 -- 172.21.15.99:0/347346238 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60d000f040 con 0x55eef9afa510 2023-08-25T08:58:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60cd7fa700 1 -- 172.21.15.99:0/347346238 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0008510 con 0x55eef9afa510 2023-08-25T08:58:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60f056cb80 1 -- 172.21.15.99:0/347346238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9afa510 msgr2=0x55eef9aeb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60f056cb80 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9afa510 0x55eef9aeb1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009130 tx=0x7f60d001f000 comp rx=0 tx=0).stop 2023-08-25T08:58:53.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60f056cb80 1 -- 172.21.15.99:0/347346238 shutdown_connections 2023-08-25T08:58:53.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60f056cb80 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9afa510 0x55eef9aeb1a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60f056cb80 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9af9190 0x55eef9aebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60f056cb80 1 --2- 172.21.15.99:0/347346238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9af0 0x55eef9b09d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.538+0000 7f60f056cb80 1 -- 172.21.15.99:0/347346238 >> 172.21.15.99:0/347346238 conn(0x55eef9a65090 msgr2=0x55eef9ae4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.539+0000 7f60f056cb80 1 -- 172.21.15.99:0/347346238 shutdown_connections 2023-08-25T08:58:53.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.539+0000 7f60f056cb80 1 -- 172.21.15.99:0/347346238 wait complete. 2023-08-25T08:58:53.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.539+0000 7f60f056cb80 1 -- start start 2023-08-25T08:58:53.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.539+0000 7f60f056cb80 1 Processor -- start 2023-08-25T08:58:53.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9190 0x55eef9b06db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9b09300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60f056cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60f056cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef9b09f20 con 0x55eef9af9af0 2023-08-25T08:58:53.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60f056cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef9b0a060 con 0x55eef9afa510 2023-08-25T08:58:53.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60f056cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef9b61df0 con 0x55eef9af9190 2023-08-25T08:58:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60f056cb80 10 client.0 _ll_register_callbacks cb 0x7f60c8011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9190 0x55eef9b06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60cffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9b09300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9190 0x55eef9b06db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:37046/0 (socket says 172.21.15.99:37046) 2023-08-25T08:58:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 -- 172.21.15.99:0/399493191 learned_addr learned my addr 172.21.15.99:0/399493191 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 -- 172.21.15.99:0/399493191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 msgr2=0x55eef9b09840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 -- 172.21.15.99:0/399493191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 msgr2=0x55eef9b09300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9b09300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 -- 172.21.15.99:0/399493191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d81a0d10 con 0x55eef9af9190 2023-08-25T08:58:53.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60cffff700 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9b09300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:53.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d6d9d700 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:58:53.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60d659c700 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9190 0x55eef9b06db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f60c80118f0 tx=0x7f60c8026ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f799692b134dc8f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.540+0000 7f60bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T08:58:53.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60bdffb700 1 -- 172.21.15.99:0/399493191 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8016040 con 0x55eef9af9190 2023-08-25T08:58:53.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 1 -- 172.21.15.99:0/399493191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eef9ae3860 con 0x55eef9af9190 2023-08-25T08:58:53.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60bdffb700 1 -- 172.21.15.99:0/399493191 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f60c8026610 con 0x55eef9af9190 2023-08-25T08:58:53.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60bdffb700 1 -- 172.21.15.99:0/399493191 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60c8026420 con 0x55eef9af9190 2023-08-25T08:58:53.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 10 client.25129 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:53.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 1 -- 172.21.15.99:0/399493191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f60d000fe50 con 0x55eef9af9190 2023-08-25T08:58:53.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60bdffb700 1 -- 172.21.15.99:0/399493191 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eef9ae3860 con 0x55eef9af9190 2023-08-25T08:58:53.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60bdffb700 1 -- 172.21.15.99:0/399493191 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60c8024070 con 0x55eef9af9190 2023-08-25T08:58:53.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 10 client.25129 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:53.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 10 client.25129 resolve_mds: resolved name 'f' to daemon mds.f 2023-08-25T08:58:53.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 20 client.25129 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:53.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55eef9b67070 0x55eef9b634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 4 client.25129 mds_command: new command op to 24517 tid=0[{"prefix": "status"}] 2023-08-25T08:58:53.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.541+0000 7f60f056cb80 1 -- 172.21.15.99:0/399493191 --> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eef9a3b920 con 0x55eef9b67070 2023-08-25T08:58:53.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60cffff700 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55eef9b67070 0x55eef9b634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60cffff700 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55eef9b67070 0x55eef9b634b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1688b4927825773 server_cookie=c2a3e00c1f083f9c in_seq=0 out_seq=0 2023-08-25T08:58:53.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60bdffb700 10 client.25129 ms_handle_connect on v2:172.21.15.181:6834/3995966648 2023-08-25T08:58:53.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60bdffb700 1 -- 172.21.15.99:0/399493191 <== mds.2 v2:172.21.15.181:6834/3995966648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55eef9a3b920 con 0x55eef9b67070 2023-08-25T08:58:53.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60bdffb700 10 client.25129 handle_command_reply: tid=0 2023-08-25T08:58:53.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60f056cb80 1 client.25129 shutdown 2023-08-25T08:58:53.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60f056cb80 1 -- 172.21.15.99:0/399493191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9190 msgr2=0x55eef9b06db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60f056cb80 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9190 0x55eef9b06db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f60c80118f0 tx=0x7f60c8026ab0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60f056cb80 1 -- 172.21.15.99:0/399493191 shutdown_connections 2023-08-25T08:58:53.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.542+0000 7f60f056cb80 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eef9afa510 0x55eef9b09840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60f056cb80 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.181:6834/3995966648,v1:172.21.15.181:6835/3995966648] conn(0x55eef9b67070 0x55eef9b634b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60f056cb80 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eef9af9190 0x55eef9b06db0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60f056cb80 1 --2- 172.21.15.99:0/399493191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eef9af9af0 0x55eef9b09300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60bdffb700 0 client.25129 ms_handle_reset on v2:172.21.15.181:6834/3995966648 2023-08-25T08:58:53.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60f056cb80 1 -- 172.21.15.99:0/399493191 >> 172.21.15.99:0/399493191 conn(0x55eef9a65090 msgr2=0x55eef9aebb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60f056cb80 1 -- 172.21.15.99:0/399493191 shutdown_connections 2023-08-25T08:58:53.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60f056cb80 1 -- 172.21.15.99:0/399493191 wait complete. 2023-08-25T08:58:53.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60f056cb80 20 client.25129 trim_cache size 0 max 16384 2023-08-25T08:58:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T08:58:53.553 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:58:53.553 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2023-08-25T08:58:53.554 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24517, 2023-08-25T08:58:53.554 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-08-25T08:58:53.554 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-08-25T08:58:53.554 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-08-25T08:58:53.554 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 89.514526618000005, 2023-08-25T08:58:53.554 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2023-08-25T08:58:53.554 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2023-08-25T08:58:53.555 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2023-08-25T08:58:53.555 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 115.74774207900001 2023-08-25T08:58:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T08:58:53.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 -- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60c409ffe0 msgr2=0x7f60c40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60c409ffe0 0x7f60c40a24a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f60d000f870 tx=0x7f60d000fc30 comp rx=0 tx=0).stop 2023-08-25T08:58:53.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 -- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d8101b20 msgr2=0x7f60d8198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d8101b20 0x7f60d8198d00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009fd0 tx=0x7f60c000f690 comp rx=0 tx=0).stop 2023-08-25T08:58:53.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 -- 172.21.15.99:0/2965125450 shutdown_connections 2023-08-25T08:58:53.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d81024c0 0x7f60d819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f60c409ffe0 0x7f60c40a24a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d8100930 0x7f60d81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 --2- 172.21.15.99:0/2965125450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d8101b20 0x7f60d8198d00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.543+0000 7f60dcbff700 1 -- 172.21.15.99:0/2965125450 >> 172.21.15.99:0/2965125450 conn(0x7f60d80fbea0 msgr2=0x7f60d8107420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.544+0000 7f60dcbff700 1 -- 172.21.15.99:0/2965125450 shutdown_connections 2023-08-25T08:58:53.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.544+0000 7f60dcbff700 1 -- 172.21.15.99:0/2965125450 wait complete. 2023-08-25T08:58:53.560 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:53 smithi099 ceph-mon[97641]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:53.560 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:53 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3978315386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:53.560 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:53 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1068334643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:53.562 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-08-25T08:58:53.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 -- 172.21.15.99:0/1457424507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340103d20 msgr2=0x7f9340106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 --2- 172.21.15.99:0/1457424507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340103d20 0x7f9340106110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9334009990 tx=0x7f9334009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 -- 172.21.15.99:0/1457424507 shutdown_connections 2023-08-25T08:58:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 --2- 172.21.15.99:0/1457424507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9340106650 0x7f9340108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 --2- 172.21.15.99:0/1457424507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93401013f0 0x7f93401037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 --2- 172.21.15.99:0/1457424507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340103d20 0x7f9340106110 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 -- 172.21.15.99:0/1457424507 >> 172.21.15.99:0/1457424507 conn(0x7f93400fad80 msgr2=0x7f93400fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 -- 172.21.15.99:0/1457424507 shutdown_connections 2023-08-25T08:58:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.758+0000 7f9345bff700 1 -- 172.21.15.99:0/1457424507 wait complete. 2023-08-25T08:58:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.759+0000 7f9345bff700 1 Processor -- start 2023-08-25T08:58:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.759+0000 7f9345bff700 1 -- start start 2023-08-25T08:58:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.759+0000 7f9345bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93401013f0 0x7f93401987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f9345bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9340103d20 0x7f9340198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f9345bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340106650 0x7f934019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f9345bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a0350 con 0x7f9340106650 2023-08-25T08:58:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f9345bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a0490 con 0x7f9340103d20 2023-08-25T08:58:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f9345bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a0620 con 0x7f93401013f0 2023-08-25T08:58:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340106650 0x7f934019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340106650 0x7f934019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42120/0 (socket says 172.21.15.99:42120) 2023-08-25T08:58:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 -- 172.21.15.99:0/4122860438 learned_addr learned my addr 172.21.15.99:0/4122860438 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933effd700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9340103d20 0x7f9340198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933f7fe700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93401013f0 0x7f93401987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 -- 172.21.15.99:0/4122860438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93401013f0 msgr2=0x7f93401987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93401013f0 0x7f93401987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 -- 172.21.15.99:0/4122860438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9340103d20 msgr2=0x7f9340198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9340103d20 0x7f9340198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 -- 172.21.15.99:0/4122860438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340095f0 con 0x7f9340106650 2023-08-25T08:58:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.760+0000 7f933ffff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340106650 0x7f934019fd80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9338007ee0 tx=0x7f933800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.761+0000 7f933cff9700 1 -- 172.21.15.99:0/4122860438 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338017070 con 0x7f9340106650 2023-08-25T08:58:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.761+0000 7f933cff9700 1 -- 172.21.15.99:0/4122860438 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933800f040 con 0x7f9340106650 2023-08-25T08:58:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.761+0000 7f9345bff700 1 -- 172.21.15.99:0/4122860438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93401a0900 con 0x7f9340106650 2023-08-25T08:58:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.761+0000 7f933cff9700 1 -- 172.21.15.99:0/4122860438 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338005580 con 0x7f9340106650 2023-08-25T08:58:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.761+0000 7f9345bff700 1 -- 172.21.15.99:0/4122860438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93401a0e10 con 0x7f9340106650 2023-08-25T08:58:53.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.764+0000 7f933cff9700 1 -- 172.21.15.99:0/4122860438 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f933800db20 con 0x7f9340106650 2023-08-25T08:58:53.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.766+0000 7f93594d1b80 1 Processor -- start 2023-08-25T08:58:53.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.766+0000 7f93594d1b80 1 -- start start 2023-08-25T08:58:53.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.766+0000 7f933cff9700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f93280a0060 0x7f93280a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.766+0000 7f933cff9700 1 -- 172.21.15.99:0/4122860438 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f93380cab80 con 0x7f9340106650 2023-08-25T08:58:53.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933effd700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f93280a0060 0x7f93280a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cacd980 0x56093cacdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cace2e0 0x56093cace720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093caced00 0x56093cacf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f93594d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093ca35760 con 0x56093cace2e0 2023-08-25T08:58:53.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f93594d1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093ca26730 con 0x56093caced00 2023-08-25T08:58:53.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f93594d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093ca24050 con 0x56093cacd980 2023-08-25T08:58:53.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cace2e0 0x56093cace720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cace2e0 0x56093cace720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42132/0 (socket says 172.21.15.99:42132) 2023-08-25T08:58:53.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933effd700 1 -- 172.21.15.99:0/3544247449 learned_addr learned my addr 172.21.15.99:0/3544247449 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933f7fe700 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cacd980 0x56093cacdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933ffff700 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093caced00 0x56093cacf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933f7fe700 1 -- 172.21.15.99:0/3544247449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093caced00 msgr2=0x56093cacf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933f7fe700 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093caced00 0x56093cacf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933f7fe700 1 -- 172.21.15.99:0/3544247449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cace2e0 msgr2=0x56093cace720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933f7fe700 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cace2e0 0x56093cace720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933effd700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f93280a0060 0x7f93280a2520 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f93340098e0 tx=0x7f933400bca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933f7fe700 1 -- 172.21.15.99:0/3544247449 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56093c89f5e0 con 0x56093cacd980 2023-08-25T08:58:53.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.767+0000 7f933ffff700 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093caced00 0x56093cacf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:53.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.768+0000 7f933f7fe700 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cacd980 0x56093cacdda0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9330009cb0 tx=0x7f9330011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a534516406c8698e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.768+0000 7f9325ffb700 1 -- 172.21.15.99:0/3544247449 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330011ca0 con 0x56093cacd980 2023-08-25T08:58:53.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.768+0000 7f9325ffb700 1 -- 172.21.15.99:0/3544247449 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9330011e00 con 0x56093cacd980 2023-08-25T08:58:53.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.768+0000 7f9325ffb700 1 -- 172.21.15.99:0/3544247449 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330005220 con 0x56093cacd980 2023-08-25T08:58:53.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.768+0000 7f93594d1b80 1 -- 172.21.15.99:0/3544247449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cacd980 msgr2=0x56093cacdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.768+0000 7f93594d1b80 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cacd980 0x56093cacdda0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9330009cb0 tx=0x7f9330011440 comp rx=0 tx=0).stop 2023-08-25T08:58:53.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 -- 172.21.15.99:0/3544247449 shutdown_connections 2023-08-25T08:58:53.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093caced00 0x56093cacf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cacd980 0x56093cacdda0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 --2- 172.21.15.99:0/3544247449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cace2e0 0x56093cace720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 -- 172.21.15.99:0/3544247449 >> 172.21.15.99:0/3544247449 conn(0x56093ca42090 msgr2=0x56093cac8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 -- 172.21.15.99:0/3544247449 shutdown_connections 2023-08-25T08:58:53.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 -- 172.21.15.99:0/3544247449 wait complete. 2023-08-25T08:58:53.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 -- start start 2023-08-25T08:58:53.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.769+0000 7f93594d1b80 1 Processor -- start 2023-08-25T08:58:53.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cad7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cad7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cade8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f93594d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093ca1f940 con 0x56093cacd980 2023-08-25T08:58:53.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f93594d1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093c9bdb60 con 0x56093cace2e0 2023-08-25T08:58:53.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f93594d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093c9bd860 con 0x56093caced00 2023-08-25T08:58:53.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f93594d1b80 10 client.0 _ll_register_callbacks cb 0x56093c907a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:53.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f933f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cad7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f933f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cad7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42142/0 (socket says 172.21.15.99:42142) 2023-08-25T08:58:53.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f933f7fe700 1 -- 172.21.15.99:0/1183015329 learned_addr learned my addr 172.21.15.99:0/1183015329 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f933effd700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cad7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f933ffff700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cade8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.770+0000 7f933f7fe700 1 -- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 msgr2=0x56093cade8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f933f7fe700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cade8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f933f7fe700 1 -- 172.21.15.99:0/1183015329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 msgr2=0x56093cad7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f933f7fe700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cad7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f933f7fe700 1 -- 172.21.15.99:0/1183015329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93401a0e10 con 0x56093cacd980 2023-08-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f933ffff700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cade8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f933effd700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cad7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f933f7fe700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cad7330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9330005c50 tx=0x7f9330011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ace73dfc4fa306f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f931effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f931effd700 1 -- 172.21.15.99:0/1183015329 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330004f10 con 0x56093cacd980 2023-08-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f931effd700 1 -- 172.21.15.99:0/1183015329 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9330005070 con 0x56093cacd980 2023-08-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f93594d1b80 1 -- 172.21.15.99:0/1183015329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56093ca9d170 con 0x56093cacd980 2023-08-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f931effd700 1 -- 172.21.15.99:0/1183015329 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9330005230 con 0x56093cacd980 2023-08-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f93594d1b80 10 client.15102 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.771+0000 7f93594d1b80 1 -- 172.21.15.99:0/1183015329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f93300098e0 con 0x56093cacd980 2023-08-25T08:58:53.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f931effd700 1 -- 172.21.15.99:0/1183015329 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56093ca9d170 con 0x56093cacd980 2023-08-25T08:58:53.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f931effd700 1 -- 172.21.15.99:0/1183015329 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9330022c50 con 0x56093cacd980 2023-08-25T08:58:53.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f93594d1b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:53.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f93594d1b80 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-25T08:58:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f93594d1b80 20 client.15102 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x56093cae2e90 0x56093cae1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f93594d1b80 4 client.15102 mds_command: new command op to 24547 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:58:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f93594d1b80 1 -- 172.21.15.99:0/1183015329 --> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56093c9c5b50 con 0x56093cae2e90 2023-08-25T08:58:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.772+0000 7f933effd700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x56093cae2e90 0x56093cae1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.773+0000 7f933effd700 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x56093cae2e90 0x56093cae1950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22d44d3434e0940e server_cookie=e62f2e3a2e4ec903 in_seq=0 out_seq=0 2023-08-25T08:58:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.773+0000 7f931effd700 10 client.15102 ms_handle_connect on v2:172.21.15.181:6838/2903255438 2023-08-25T08:58:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.773+0000 7f931effd700 1 -- 172.21.15.99:0/1183015329 <== mds.1 v2:172.21.15.181:6838/2903255438 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56093c9c5b50 con 0x56093cae2e90 2023-08-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.773+0000 7f931effd700 10 client.15102 handle_command_reply: tid=0 2023-08-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 client.15102 shutdown 2023-08-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 -- 172.21.15.99:0/1183015329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 msgr2=0x56093cad7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cad7330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9330005c50 tx=0x7f9330011f00 comp rx=0 tx=0).stop 2023-08-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 -- 172.21.15.99:0/1183015329 shutdown_connections 2023-08-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x56093cae2e90 0x56093cae1950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cade8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cad7330 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f931effd700 0 client.15102 ms_handle_reset on v2:172.21.15.181:6838/2903255438 2023-08-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1183015329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cad7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 -- 172.21.15.99:0/1183015329 >> 172.21.15.99:0/1183015329 conn(0x56093ca42090 msgr2=0x56093cac2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 -- 172.21.15.99:0/1183015329 shutdown_connections 2023-08-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 1 -- 172.21.15.99:0/1183015329 wait complete. 2023-08-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.774+0000 7f93594d1b80 20 client.15102 trim_cache size 0 max 16384 2023-08-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.792+0000 7f93594d1b80 1 Processor -- start 2023-08-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.792+0000 7f93594d1b80 1 -- start start 2023-08-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cac8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cade7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cad1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f93594d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093c8a18d0 con 0x56093cacd980 2023-08-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f93594d1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093c9dbf00 con 0x56093cace2e0 2023-08-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f93594d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093c9c7470 con 0x56093caced00 2023-08-25T08:58:53.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cac8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cac8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42144/0 (socket says 172.21.15.99:42144) 2023-08-25T08:58:53.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 -- 172.21.15.99:0/2308765928 learned_addr learned my addr 172.21.15.99:0/2308765928 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933effd700 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cade7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933ffff700 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cad1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 -- 172.21.15.99:0/2308765928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 msgr2=0x56093cad1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cad1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 -- 172.21.15.99:0/2308765928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 msgr2=0x56093cade7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cade7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 -- 172.21.15.99:0/2308765928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56093c84bd90 con 0x56093cacd980 2023-08-25T08:58:53.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.793+0000 7f933f7fe700 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cac8200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9330011480 tx=0x7f9330004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a2ba0c1e3fa846 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93267fc700 1 -- 172.21.15.99:0/2308765928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330019630 con 0x56093cacd980 2023-08-25T08:58:53.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93267fc700 1 -- 172.21.15.99:0/2308765928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9330019790 con 0x56093cacd980 2023-08-25T08:58:53.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93267fc700 1 -- 172.21.15.99:0/2308765928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300050d0 con 0x56093cacd980 2023-08-25T08:58:53.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 -- 172.21.15.99:0/2308765928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 msgr2=0x56093cac8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cac8200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9330011480 tx=0x7f9330004960 comp rx=0 tx=0).stop 2023-08-25T08:58:53.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 -- 172.21.15.99:0/2308765928 shutdown_connections 2023-08-25T08:58:53.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093caced00 0x56093cad1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093cacd980 0x56093cac8200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 --2- 172.21.15.99:0/2308765928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cace2e0 0x56093cade7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 -- 172.21.15.99:0/2308765928 >> 172.21.15.99:0/2308765928 conn(0x56093ca42090 msgr2=0x56093cac1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 -- 172.21.15.99:0/2308765928 shutdown_connections 2023-08-25T08:58:53.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.794+0000 7f93594d1b80 1 -- 172.21.15.99:0/2308765928 wait complete. 2023-08-25T08:58:53.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 1 -- start start 2023-08-25T08:58:53.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 1 Processor -- start 2023-08-25T08:58:53.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cacd980 0x56093cac2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cace2e0 0x56093cade7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093caced00 0x56093cac27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093cac2e90 con 0x56093caced00 2023-08-25T08:58:53.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093cac3000 con 0x56093cacd980 2023-08-25T08:58:53.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093cb36880 con 0x56093cace2e0 2023-08-25T08:58:53.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.795+0000 7f93594d1b80 10 client.0 _ll_register_callbacks cb 0x7f933400bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:53.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093caced00 0x56093cac27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093caced00 0x56093cac27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42154/0 (socket says 172.21.15.99:42154) 2023-08-25T08:58:53.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933ffff700 1 -- 172.21.15.99:0/1489497831 learned_addr learned my addr 172.21.15.99:0/1489497831 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933effd700 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cace2e0 0x56093cade7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933f7fe700 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cacd980 0x56093cac2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933effd700 1 -- 172.21.15.99:0/1489497831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cacd980 msgr2=0x56093cac2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933effd700 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cacd980 0x56093cac2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933effd700 1 -- 172.21.15.99:0/1489497831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093caced00 msgr2=0x56093cac27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933effd700 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093caced00 0x56093cac27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933effd700 1 -- 172.21.15.99:0/1489497831 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93300098e0 con 0x56093cace2e0 2023-08-25T08:58:53.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f933effd700 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cace2e0 0x56093cade7d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9334000c00 tx=0x7f9334009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10cebb08a577bcd9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:53.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f931effd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T08:58:53.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f931effd700 1 -- 172.21.15.99:0/1489497831 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933401d040 con 0x56093cace2e0 2023-08-25T08:58:53.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.796+0000 7f93594d1b80 1 -- 172.21.15.99:0/1489497831 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56093cac0860 con 0x56093cace2e0 2023-08-25T08:58:53.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f931effd700 1 -- 172.21.15.99:0/1489497831 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9334035110 con 0x56093cace2e0 2023-08-25T08:58:53.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f931effd700 1 -- 172.21.15.99:0/1489497831 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93340352d0 con 0x56093cace2e0 2023-08-25T08:58:53.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f93594d1b80 10 client.25147 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:53.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f93594d1b80 1 -- 172.21.15.99:0/1489497831 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56093c834c10 con 0x56093cace2e0 2023-08-25T08:58:53.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f931effd700 1 -- 172.21.15.99:0/1489497831 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56093cac0860 con 0x56093cace2e0 2023-08-25T08:58:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f931effd700 1 -- 172.21.15.99:0/1489497831 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f933400f460 con 0x56093cace2e0 2023-08-25T08:58:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f93594d1b80 10 client.25147 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f93594d1b80 10 client.25147 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-25T08:58:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f93594d1b80 20 client.25147 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x56093cb3b970 0x56093cb37df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f93594d1b80 4 client.25147 mds_command: new command op to 24547 tid=0[{"prefix": "status"}] 2023-08-25T08:58:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.797+0000 7f93594d1b80 1 -- 172.21.15.99:0/1489497831 --> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56093ca18920 con 0x56093cb3b970 2023-08-25T08:58:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f933f7fe700 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x56093cb3b970 0x56093cb37df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f931effd700 10 client.25147 ms_handle_connect on v2:172.21.15.181:6838/2903255438 2023-08-25T08:58:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f933f7fe700 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x56093cb3b970 0x56093cb37df0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=931c47e07ef70ad6 server_cookie=9e630ec97935d0c1 in_seq=0 out_seq=0 2023-08-25T08:58:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f931effd700 1 -- 172.21.15.99:0/1489497831 <== mds.1 v2:172.21.15.181:6838/2903255438 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56093ca18920 con 0x56093cb3b970 2023-08-25T08:58:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f931effd700 10 client.25147 handle_command_reply: tid=0 2023-08-25T08:58:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 client.25147 shutdown 2023-08-25T08:58:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 -- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cace2e0 msgr2=0x56093cade7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cace2e0 0x56093cade7d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9334000c00 tx=0x7f9334009ca0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 -- 172.21.15.99:0/1489497831 shutdown_connections 2023-08-25T08:58:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:6838/2903255438,v1:172.21.15.181:6839/2903255438] conn(0x56093cb3b970 0x56093cb37df0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56093caced00 0x56093cac27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56093cacd980 0x56093cac2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f931effd700 0 client.25147 ms_handle_reset on v2:172.21.15.181:6838/2903255438 2023-08-25T08:58:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 --2- 172.21.15.99:0/1489497831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56093cace2e0 0x56093cade7d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.798+0000 7f93594d1b80 1 -- 172.21.15.99:0/1489497831 >> 172.21.15.99:0/1489497831 conn(0x56093ca42090 msgr2=0x56093cac8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f93594d1b80 1 -- 172.21.15.99:0/1489497831 shutdown_connections 2023-08-25T08:58:53.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f93594d1b80 1 -- 172.21.15.99:0/1489497831 wait complete. 2023-08-25T08:58:53.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f93594d1b80 20 client.25147 trim_cache size 0 max 16384 2023-08-25T08:58:53.837 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T08:58:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:58:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2023-08-25T08:58:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24547, 2023-08-25T08:58:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-08-25T08:58:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-08-25T08:58:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-08-25T08:58:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 90.746752647999998, 2023-08-25T08:58:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2023-08-25T08:58:53.839 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2023-08-25T08:58:53.839 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2023-08-25T08:58:53.839 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 109.354645502 2023-08-25T08:58:53.839 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T08:58:53.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 -- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f93280a0060 msgr2=0x7f93280a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f93280a0060 0x7f93280a2520 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f93340098e0 tx=0x7f933400bca0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 -- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340106650 msgr2=0x7f934019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340106650 0x7f934019fd80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9338007ee0 tx=0x7f933800d3b0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 -- 172.21.15.99:0/4122860438 shutdown_connections 2023-08-25T08:58:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9340106650 0x7f934019fd80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93401013f0 0x7f93401987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f93280a0060 0x7f93280a2520 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 --2- 172.21.15.99:0/4122860438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9340103d20 0x7f9340198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.799+0000 7f9345bff700 1 -- 172.21.15.99:0/4122860438 >> 172.21.15.99:0/4122860438 conn(0x7f93400fad80 msgr2=0x7f93400ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:53.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.800+0000 7f9345bff700 1 -- 172.21.15.99:0/4122860438 shutdown_connections 2023-08-25T08:58:53.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:53.800+0000 7f9345bff700 1 -- 172.21.15.99:0/4122860438 wait complete. 2023-08-25T08:58:53.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-25T08:58:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:53 smithi157 ceph-mon[130879]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:53 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3978315386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:53 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1068334643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:53 smithi181 ceph-mon[131296]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T08:58:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:53 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3978315386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T08:58:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:53 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1068334643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T08:58:54.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 -- 172.21.15.99:0/3757214063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58100930 msgr2=0x7f4d58100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 --2- 172.21.15.99:0/3757214063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58100930 0x7f4d58100d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d540099c0 tx=0x7f4d54009cd0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 -- 172.21.15.99:0/3757214063 shutdown_connections 2023-08-25T08:58:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 --2- 172.21.15.99:0/3757214063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d581024c0 0x7f4d58108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 --2- 172.21.15.99:0/3757214063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58100930 0x7f4d58100d50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 --2- 172.21.15.99:0/3757214063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d58101b20 0x7f4d58101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 -- 172.21.15.99:0/3757214063 >> 172.21.15.99:0/3757214063 conn(0x7f4d580fbea0 msgr2=0x7f4d580fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 -- 172.21.15.99:0/3757214063 shutdown_connections 2023-08-25T08:58:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.039+0000 7f4d605ff700 1 -- 172.21.15.99:0/3757214063 wait complete. 2023-08-25T08:58:54.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.040+0000 7f4d605ff700 1 Processor -- start 2023-08-25T08:58:54.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.040+0000 7f4d605ff700 1 -- start start 2023-08-25T08:58:54.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d605ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d58100930 0x7f4d58198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d605ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58101b20 0x7f4d58198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d605ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d581024c0 0x7f4d5819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d605ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a0390 con 0x7f4d58101b20 2023-08-25T08:58:54.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d605ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a04d0 con 0x7f4d581024c0 2023-08-25T08:58:54.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d605ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a0660 con 0x7f4d58100930 2023-08-25T08:58:54.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5db9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58101b20 0x7f4d58198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5db9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58101b20 0x7f4d58198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42170/0 (socket says 172.21.15.99:42170) 2023-08-25T08:58:54.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5db9a700 1 -- 172.21.15.99:0/2400058275 learned_addr learned my addr 172.21.15.99:0/2400058275 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d58100930 0x7f4d58198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d581024c0 0x7f4d5819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5db9a700 1 -- 172.21.15.99:0/2400058275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d58100930 msgr2=0x7f4d58198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d58100930 0x7f4d58198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5db9a700 1 -- 172.21.15.99:0/2400058275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d581024c0 msgr2=0x7f4d5819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d581024c0 0x7f4d5819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5db9a700 1 -- 172.21.15.99:0/2400058275 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d540095f0 con 0x7f4d58101b20 2023-08-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.041+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d581024c0 0x7f4d5819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:58:54.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.042+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58101b20 0x7f4d58198d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e9a0 tx=0x7f4d4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:54.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.042+0000 7f4d4f7fe700 1 -- 172.21.15.99:0/2400058275 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48009790 con 0x7f4d58101b20 2023-08-25T08:58:54.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.042+0000 7f4d4f7fe700 1 -- 172.21.15.99:0/2400058275 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d4800cbd0 con 0x7f4d58101b20 2023-08-25T08:58:54.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.042+0000 7f4d605ff700 1 -- 172.21.15.99:0/2400058275 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d581a0940 con 0x7f4d58101b20 2023-08-25T08:58:54.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.042+0000 7f4d4f7fe700 1 -- 172.21.15.99:0/2400058275 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48018680 con 0x7f4d58101b20 2023-08-25T08:58:54.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.042+0000 7f4d605ff700 1 -- 172.21.15.99:0/2400058275 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d581a0e50 con 0x7f4d58101b20 2023-08-25T08:58:54.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.045+0000 7f4d4f7fe700 1 -- 172.21.15.99:0/2400058275 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f4d4800cd40 con 0x7f4d58101b20 2023-08-25T08:58:54.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.046+0000 7f4d74020b80 1 Processor -- start 2023-08-25T08:58:54.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.046+0000 7f4d74020b80 1 -- start start 2023-08-25T08:58:54.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03a5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a5d00 0x5560a03a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d74020b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a030c760 con 0x5560a03a52e0 2023-08-25T08:58:54.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d74020b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a02fd730 con 0x5560a03a4980 2023-08-25T08:58:54.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d74020b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a02fb050 con 0x5560a03a5d00 2023-08-25T08:58:54.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d4f7fe700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d4409ffe0 0x7f4d440a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d5e39b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d5eb9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d5db9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a5d00 0x5560a03a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d5eb9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48324/0 (socket says 172.21.15.99:48324) 2023-08-25T08:58:54.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d5eb9c700 1 -- 172.21.15.99:0/1133155678 learned_addr learned my addr 172.21.15.99:0/1133155678 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:54.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d5e39b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42182/0 (socket says 172.21.15.99:42182) 2023-08-25T08:58:54.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d4409ffe0 0x7f4d440a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.047+0000 7f4d4f7fe700 1 -- 172.21.15.99:0/2400058275 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d48014070 con 0x7f4d58101b20 2023-08-25T08:58:54.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5db9a700 1 -- 172.21.15.99:0/1133155678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 msgr2=0x5560a03a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:54.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5db9a700 1 -- 172.21.15.99:0/1133155678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 msgr2=0x5560a03a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:54.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5db9a700 1 -- 172.21.15.99:0/1133155678 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560a01765e0 con 0x5560a03a5d00 2023-08-25T08:58:54.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a5d00 0x5560a03a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d480c8fc0 tx=0x7f4d4801cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b45b3130cc8b115b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.048+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d4409ffe0 0x7f4d440a24a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4d50019570 tx=0x7f4d5000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:54.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.049+0000 7f4d4cbfe700 1 -- 172.21.15.99:0/1133155678 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48098a40 con 0x5560a03a5d00 2023-08-25T08:58:54.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.049+0000 7f4d4cbfe700 1 -- 172.21.15.99:0/1133155678 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d48098860 con 0x5560a03a5d00 2023-08-25T08:58:54.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.049+0000 7f4d4cbfe700 1 -- 172.21.15.99:0/1133155678 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d480a0da0 con 0x5560a03a5d00 2023-08-25T08:58:54.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 -- 172.21.15.99:0/1133155678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a5d00 msgr2=0x5560a03a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a5d00 0x5560a03a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d480c8fc0 tx=0x7f4d4801cdd0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 -- 172.21.15.99:0/1133155678 shutdown_connections 2023-08-25T08:58:54.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a5d00 0x5560a03a6180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1133155678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 -- 172.21.15.99:0/1133155678 >> 172.21.15.99:0/1133155678 conn(0x5560a0319090 msgr2=0x5560a039f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:54.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 -- 172.21.15.99:0/1133155678 shutdown_connections 2023-08-25T08:58:54.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.050+0000 7f4d74020b80 1 -- 172.21.15.99:0/1133155678 wait complete. 2023-08-25T08:58:54.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.051+0000 7f4d74020b80 1 -- start start 2023-08-25T08:58:54.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.051+0000 7f4d74020b80 1 Processor -- start 2023-08-25T08:58:54.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.051+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.051+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a5d00 0x5560a03b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d74020b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a02f6940 con 0x5560a03a5d00 2023-08-25T08:58:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d74020b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a0294b60 con 0x5560a03a4980 2023-08-25T08:58:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d74020b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a0294860 con 0x5560a03a52e0 2023-08-25T08:58:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d74020b80 10 client.0 _ll_register_callbacks cb 0x5560a01dea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5e39b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5eb9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a5d00 0x5560a03b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5e39b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48328/0 (socket says 172.21.15.99:48328) 2023-08-25T08:58:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5eb9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a5d00 0x5560a03b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42188/0 (socket says 172.21.15.99:42188) 2023-08-25T08:58:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5e39b700 1 -- 172.21.15.99:0/1275276335 learned_addr learned my addr 172.21.15.99:0/1275276335 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5eb9c700 1 -- 172.21.15.99:0/1275276335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 msgr2=0x5560a03ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5eb9c700 1 -- 172.21.15.99:0/1275276335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 msgr2=0x5560a03ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5eb9c700 1 -- 172.21.15.99:0/1275276335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d581096c0 con 0x5560a03a5d00 2023-08-25T08:58:54.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.052+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:54.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a5d00 0x5560a03b58d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4d50019c10 tx=0x7f4d50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea62a282b9af83c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:54.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:58:54.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1275276335 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50028070 con 0x5560a03a5d00 2023-08-25T08:58:54.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d74020b80 1 -- 172.21.15.99:0/1275276335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560a0374170 con 0x5560a03a5d00 2023-08-25T08:58:54.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1275276335 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d50009350 con 0x5560a03a5d00 2023-08-25T08:58:54.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1275276335 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d5000e040 con 0x5560a03a5d00 2023-08-25T08:58:54.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d74020b80 10 client.15120 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:54.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d74020b80 1 -- 172.21.15.99:0/1275276335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f4d581a0e50 con 0x5560a03a5d00 2023-08-25T08:58:54.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1275276335 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560a0374170 con 0x5560a03a5d00 2023-08-25T08:58:54.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.053+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1275276335 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d50023070 con 0x5560a03a5d00 2023-08-25T08:58:54.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.054+0000 7f4d74020b80 10 client.15120 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:54.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.054+0000 7f4d74020b80 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T08:58:54.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.054+0000 7f4d74020b80 20 client.15120 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:54.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.054+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5560a03b9fc0 0x5560a03b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.054+0000 7f4d74020b80 4 client.15120 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:58:54.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.054+0000 7f4d74020b80 1 -- 172.21.15.99:0/1275276335 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560a029cb50 con 0x5560a03b9fc0 2023-08-25T08:58:54.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.054+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5560a03b9fc0 0x5560a03b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.055+0000 7f4d41ffb700 10 client.15120 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:58:54.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.055+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5560a03b9fc0 0x5560a03b8a20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704b5ee107584b78 server_cookie=8745554b94b2a09f in_seq=0 out_seq=0 2023-08-25T08:58:54.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.055+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1275276335 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5560a029cb50 con 0x5560a03b9fc0 2023-08-25T08:58:54.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.055+0000 7f4d41ffb700 10 client.15120 handle_command_reply: tid=0 2023-08-25T08:58:54.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.055+0000 7f4d74020b80 1 client.15120 shutdown 2023-08-25T08:58:54.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 -- 172.21.15.99:0/1275276335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a5d00 msgr2=0x5560a03b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a5d00 0x5560a03b58d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4d50019c10 tx=0x7f4d50004ad0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 -- 172.21.15.99:0/1275276335 shutdown_connections 2023-08-25T08:58:54.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a5d00 0x5560a03b58d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5560a03b9fc0 0x5560a03b8a20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a4980 0x5560a03ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d41ffb700 0 client.15120 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:58:54.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1275276335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 -- 172.21.15.99:0/1275276335 >> 172.21.15.99:0/1275276335 conn(0x5560a0319090 msgr2=0x5560a0399af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:54.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 -- 172.21.15.99:0/1275276335 shutdown_connections 2023-08-25T08:58:54.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 1 -- 172.21.15.99:0/1275276335 wait complete. 2023-08-25T08:58:54.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.056+0000 7f4d74020b80 20 client.15120 trim_cache size 0 max 16384 2023-08-25T08:58:54.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d74020b80 1 Processor -- start 2023-08-25T08:58:54.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d74020b80 1 -- start start 2023-08-25T08:58:54.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a4980 0x5560a039f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 0x5560a03a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d74020b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a01788d0 con 0x5560a03a52e0 2023-08-25T08:58:54.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d74020b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a03b75c0 con 0x5560a03a5d00 2023-08-25T08:58:54.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d74020b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a02b6df0 con 0x5560a03a4980 2023-08-25T08:58:54.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d5db9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d5db9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42198/0 (socket says 172.21.15.99:42198) 2023-08-25T08:58:54.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d5db9a700 1 -- 172.21.15.99:0/2517939495 learned_addr learned my addr 172.21.15.99:0/2517939495 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:54.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.074+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 0x5560a03a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a4980 0x5560a039f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d5db9a700 1 -- 172.21.15.99:0/2517939495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a4980 msgr2=0x5560a039f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:54.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a4980 0x5560a039f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d5db9a700 1 -- 172.21.15.99:0/2517939495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 msgr2=0x5560a03a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:54.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 0x5560a03a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d5db9a700 1 -- 172.21.15.99:0/2517939495 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560a0122d90 con 0x5560a03a52e0 2023-08-25T08:58:54.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a4980 0x5560a039f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:58:54.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800c5b0 tx=0x7f4d480c8f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3744c1b6292d7b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:54.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d4d3ff700 1 -- 172.21.15.99:0/2517939495 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48098f00 con 0x5560a03a52e0 2023-08-25T08:58:54.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d4d3ff700 1 -- 172.21.15.99:0/2517939495 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d48099600 con 0x5560a03a52e0 2023-08-25T08:58:54.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.075+0000 7f4d4d3ff700 1 -- 172.21.15.99:0/2517939495 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48099160 con 0x5560a03a52e0 2023-08-25T08:58:54.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 -- 172.21.15.99:0/2517939495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 msgr2=0x5560a03b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800c5b0 tx=0x7f4d480c8f00 comp rx=0 tx=0).stop 2023-08-25T08:58:54.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 -- 172.21.15.99:0/2517939495 shutdown_connections 2023-08-25T08:58:54.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 0x5560a03a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a4980 0x5560a039f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 --2- 172.21.15.99:0/2517939495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 -- 172.21.15.99:0/2517939495 >> 172.21.15.99:0/2517939495 conn(0x5560a0319090 msgr2=0x5560a03ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 -- 172.21.15.99:0/2517939495 shutdown_connections 2023-08-25T08:58:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 -- 172.21.15.99:0/2517939495 wait complete. 2023-08-25T08:58:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 -- start start 2023-08-25T08:58:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.076+0000 7f4d74020b80 1 Processor -- start 2023-08-25T08:58:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a4980 0x5560a0399260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d74020b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 0x5560a03997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d74020b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a03ab9e0 con 0x5560a03a4980 2023-08-25T08:58:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d74020b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a0399ce0 con 0x5560a03a5d00 2023-08-25T08:58:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d74020b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a0399ea0 con 0x5560a03a52e0 2023-08-25T08:58:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d74020b80 10 client.0 _ll_register_callbacks cb 0x5560a01dea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:58:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5e39b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a4980 0x5560a0399260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5db9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5db9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:37108/0 (socket says 172.21.15.99:37108) 2023-08-25T08:58:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5e39b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a4980 0x5560a0399260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42202/0 (socket says 172.21.15.99:42202) 2023-08-25T08:58:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5db9a700 1 -- 172.21.15.99:0/1736454743 learned_addr learned my addr 172.21.15.99:0/1736454743 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5eb9c700 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 0x5560a03997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5db9a700 1 -- 172.21.15.99:0/1736454743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 msgr2=0x5560a03997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 0x5560a03997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5db9a700 1 -- 172.21.15.99:0/1736454743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a4980 msgr2=0x5560a0399260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:58:54.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a4980 0x5560a0399260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5db9a700 1 -- 172.21.15.99:0/1736454743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560a01765e0 con 0x5560a03a52e0 2023-08-25T08:58:54.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.077+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a4980 0x5560a0399260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:58:54.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d5db9a700 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d48098e90 tx=0x7f4d48097410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1161f318290a07 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:54.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T08:58:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1736454743 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48099360 con 0x5560a03a52e0 2023-08-25T08:58:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1736454743 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d48098f00 con 0x5560a03a52e0 2023-08-25T08:58:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 1 -- 172.21.15.99:0/1736454743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560a0397860 con 0x5560a03a52e0 2023-08-25T08:58:54.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1736454743 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d480a0880 con 0x5560a03a52e0 2023-08-25T08:58:54.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 10 client.25162 fetch_fsmap learned FSMap version 23 2023-08-25T08:58:54.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 1 -- 172.21.15.99:0/1736454743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5560a010bc10 con 0x5560a03a52e0 2023-08-25T08:58:54.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1736454743 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560a0397860 con 0x5560a03a52e0 2023-08-25T08:58:54.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1736454743 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d480c8080 con 0x5560a03a52e0 2023-08-25T08:58:54.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 10 client.25162 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:58:54.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 10 client.25162 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T08:58:54.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 20 client.25162 populate_metadata read hostname 'smithi099' 2023-08-25T08:58:54.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5560a04122e0 0x5560a040e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:54.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 4 client.25162 mds_command: new command op to 24593 tid=0[{"prefix": "status"}] 2023-08-25T08:58:54.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.078+0000 7f4d74020b80 1 -- 172.21.15.99:0/1736454743 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5560a02ef920 con 0x5560a04122e0 2023-08-25T08:58:54.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.079+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5560a04122e0 0x5560a040e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:54.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.079+0000 7f4d5e39b700 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5560a04122e0 0x5560a040e780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdb3894db534a5b server_cookie=80c57fb94974e61b in_seq=0 out_seq=0 2023-08-25T08:58:54.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.079+0000 7f4d41ffb700 10 client.25162 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:58:54.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d41ffb700 1 -- 172.21.15.99:0/1736454743 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5560a02ef920 con 0x5560a04122e0 2023-08-25T08:58:54.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d41ffb700 10 client.25162 handle_command_reply: tid=0 2023-08-25T08:58:54.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 client.25162 shutdown 2023-08-25T08:58:54.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 -- 172.21.15.99:0/1736454743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 msgr2=0x5560a03b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d48098e90 tx=0x7f4d48097410 comp rx=0 tx=0).stop 2023-08-25T08:58:54.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 -- 172.21.15.99:0/1736454743 shutdown_connections 2023-08-25T08:58:54.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5560a04122e0 0x5560a040e780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a03a5d00 0x5560a03997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d41ffb700 0 client.25162 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:58:54.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560a03a4980 0x5560a0399260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 --2- 172.21.15.99:0/1736454743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560a03a52e0 0x5560a03b57f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 -- 172.21.15.99:0/1736454743 >> 172.21.15.99:0/1736454743 conn(0x5560a0319090 msgr2=0x5560a03ac0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:54.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 -- 172.21.15.99:0/1736454743 shutdown_connections 2023-08-25T08:58:54.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 1 -- 172.21.15.99:0/1736454743 wait complete. 2023-08-25T08:58:54.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.080+0000 7f4d74020b80 20 client.25162 trim_cache size 0 max 16384 2023-08-25T08:58:54.091 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T08:58:54.091 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "3d4188ec-4324-11ee-9b3d-001a4aab830c", 2023-08-25T08:58:54.091 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-08-25T08:58:54.091 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24593, 2023-08-25T08:58:54.091 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-08-25T08:58:54.091 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-08-25T08:58:54.092 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-08-25T08:58:54.092 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 94.056266268000002, 2023-08-25T08:58:54.092 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2023-08-25T08:58:54.092 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2023-08-25T08:58:54.092 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2023-08-25T08:58:54.092 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 101.310479359 2023-08-25T08:58:54.093 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T08:58:54.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 -- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d4409ffe0 msgr2=0x7f4d440a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d4409ffe0 0x7f4d440a24a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4d50019570 tx=0x7f4d5000b040 comp rx=0 tx=0).stop 2023-08-25T08:58:54.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 -- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58101b20 msgr2=0x7f4d58198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:54.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58101b20 0x7f4d58198d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e9a0 tx=0x7f4d4800ecb0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 -- 172.21.15.99:0/2400058275 shutdown_connections 2023-08-25T08:58:54.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f4d4409ffe0 0x7f4d440a24a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d581024c0 0x7f4d5819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d58100930 0x7f4d58198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 --2- 172.21.15.99:0/2400058275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58101b20 0x7f4d58198d50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:54.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 -- 172.21.15.99:0/2400058275 >> 172.21.15.99:0/2400058275 conn(0x7f4d580fbea0 msgr2=0x7f4d58107450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:54.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 -- 172.21.15.99:0/2400058275 shutdown_connections 2023-08-25T08:58:54.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:54.081+0000 7f4d605ff700 1 -- 172.21.15.99:0/2400058275 wait complete. 2023-08-25T08:58:54.099 INFO:teuthology.run_tasks:Running task kclient... 2023-08-25T08:58:54.123 INFO:tasks.kclient:Mounting kernel clients... 2023-08-25T08:58:54.123 INFO:tasks.kclient:config is {'client.0': {}} 2023-08-25T08:58:54.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:54.124 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-08-25T08:58:54.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:54.168 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2023-08-25T08:58:54.280 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2023-08-25T08:58:54.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T08:58:54.283 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-08-25T08:58:54.284 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T08:58:54.284 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T08:58:54.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph fs ls 2023-08-25T08:58:55.033 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T08:58:55.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:55 smithi099 ceph-mon[97641]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:55 smithi157 ceph-mon[130879]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:55 smithi181 ceph-mon[131296]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-08-25T08:58:56.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-08-25T08:58:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-08-25T08:58:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:56 smithi099 ceph-mon[97641]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-25T08:58:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-08-25T08:58:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:56 smithi099 ceph-mon[97641]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-08-25T08:58:56.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:56 smithi099 ceph-mon[97641]: osdmap e86: 12 total, 12 up, 12 in 2023-08-25T08:58:56.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-08-25T08:58:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-08-25T08:58:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:56 smithi157 ceph-mon[130879]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-25T08:58:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-08-25T08:58:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:56 smithi157 ceph-mon[130879]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-08-25T08:58:56.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:56 smithi157 ceph-mon[130879]: osdmap e86: 12 total, 12 up, 12 in 2023-08-25T08:58:56.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-08-25T08:58:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-08-25T08:58:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:56 smithi181 ceph-mon[131296]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-25T08:58:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-08-25T08:58:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:56 smithi181 ceph-mon[131296]: from='mgr.14150 172.21.15.99:0/2437094307' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-08-25T08:58:56.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:56 smithi181 ceph-mon[131296]: osdmap e86: 12 total, 12 up, 12 in 2023-08-25T08:58:56.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 -- 172.21.15.99:0/1827458781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028c101300 msgr2=0x7f028c10d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:56.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 --2- 172.21.15.99:0/1827458781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028c101300 0x7f028c10d810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f028400b0a0 tx=0x7f028400b3b0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 -- 172.21.15.99:0/1827458781 shutdown_connections 2023-08-25T08:58:56.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 --2- 172.21.15.99:0/1827458781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028c101300 0x7f028c10d810 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 --2- 172.21.15.99:0/1827458781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c106940 0x7f028c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 --2- 172.21.15.99:0/1827458781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f028c100940 0x7f028c100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 -- 172.21.15.99:0/1827458781 >> 172.21.15.99:0/1827458781 conn(0x7f028c0fc410 msgr2=0x7f028c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:56.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 -- 172.21.15.99:0/1827458781 shutdown_connections 2023-08-25T08:58:56.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.607+0000 7f02920d7700 1 -- 172.21.15.99:0/1827458781 wait complete. 2023-08-25T08:58:56.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.608+0000 7f02920d7700 1 Processor -- start 2023-08-25T08:58:56.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.608+0000 7f02920d7700 1 -- start start 2023-08-25T08:58:56.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f02920d7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f028c100940 0x7f028c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:56.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f02920d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c101300 0x7f028c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:56.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f02920d7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028c106940 0x7f028c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:56.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f02920d7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c19dd20 con 0x7f028c106940 2023-08-25T08:58:56.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f02920d7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c19de90 con 0x7f028c100940 2023-08-25T08:58:56.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f02920d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c19e050 con 0x7f028c101300 2023-08-25T08:58:56.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f028b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f028c100940 0x7f028c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:56.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f028b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f028c100940 0x7f028c198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48364/0 (socket says 172.21.15.99:48364) 2023-08-25T08:58:56.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f028b7fe700 1 -- 172.21.15.99:0/1608103831 learned_addr learned my addr 172.21.15.99:0/1608103831 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:58:56.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f028bfff700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028c106940 0x7f028c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:56.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.609+0000 7f028affd700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c101300 0x7f028c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:56.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f028b7fe700 1 -- 172.21.15.99:0/1608103831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c101300 msgr2=0x7f028c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:56.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f028b7fe700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c101300 0x7f028c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f028b7fe700 1 -- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028c106940 msgr2=0x7f028c19d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:56.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f028b7fe700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028c106940 0x7f028c19d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f028b7fe700 1 -- 172.21.15.99:0/1608103831 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284009d00 con 0x7f028c100940 2023-08-25T08:58:56.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f028b7fe700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f028c100940 0x7f028c198d60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9a0 tx=0x7f027c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:56.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f0288ff9700 1 -- 172.21.15.99:0/1608103831 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c009790 con 0x7f028c100940 2023-08-25T08:58:56.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c19e330 con 0x7f028c100940 2023-08-25T08:58:56.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f0288ff9700 1 -- 172.21.15.99:0/1608103831 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027c010430 con 0x7f028c100940 2023-08-25T08:58:56.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f0288ff9700 1 -- 172.21.15.99:0/1608103831 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c018680 con 0x7f028c100940 2023-08-25T08:58:56.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.610+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c1a5b90 con 0x7f028c100940 2023-08-25T08:58:56.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.611+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c04e620 con 0x7f028c100940 2023-08-25T08:58:56.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.613+0000 7f0288ff9700 1 -- 172.21.15.99:0/1608103831 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f027c0105a0 con 0x7f028c100940 2023-08-25T08:58:56.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.615+0000 7f0288ff9700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f027409ff50 0x7f02740a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:58:56.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.615+0000 7f0288ff9700 1 -- 172.21.15.99:0/1608103831 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8327+0+0 (secure 0 0 0) 0x7f027c014070 con 0x7f028c100940 2023-08-25T08:58:56.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.615+0000 7f028affd700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f027409ff50 0x7f02740a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:58:56.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.615+0000 7f028affd700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f027409ff50 0x7f02740a2410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f028c19a380 tx=0x7f0280009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:58:56.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.617+0000 7f0288ff9700 1 -- 172.21.15.99:0/1608103831 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f027c095330 con 0x7f028c100940 2023-08-25T08:58:56.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.858+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f028c110a80 con 0x7f028c100940 2023-08-25T08:58:56.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.859+0000 7f0288ff9700 1 -- 172.21.15.99:0/1608103831 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f027c026060 con 0x7f028c100940 2023-08-25T08:58:56.860 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T08:58:56.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f027409ff50 msgr2=0x7f02740a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:56.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f027409ff50 0x7f02740a2410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f028c19a380 tx=0x7f0280009260 comp rx=0 tx=0).stop 2023-08-25T08:58:56.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f028c100940 msgr2=0x7f028c198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:58:56.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f028c100940 0x7f028c198d60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9a0 tx=0x7f027c00ecb0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 shutdown_connections 2023-08-25T08:58:56.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028c106940 0x7f028c19d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f027409ff50 0x7f02740a2410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f028c100940 0x7f028c198d60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 --2- 172.21.15.99:0/1608103831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c101300 0x7f028c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:58:56.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 >> 172.21.15.99:0/1608103831 conn(0x7f028c0fc410 msgr2=0x7f028c10bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:58:56.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 shutdown_connections 2023-08-25T08:58:56.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:58:56.862+0000 7f02920d7700 1 -- 172.21.15.99:0/1608103831 wait complete. 2023-08-25T08:58:57.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:57 smithi099 ceph-mon[97641]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2023-08-25T08:58:57.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:57 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1608103831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T08:58:57.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:57 smithi099 ceph-mon[97641]: osdmap e87: 12 total, 12 up, 12 in 2023-08-25T08:58:57.317 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T08:58:57.318 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T08:58:57.318 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-08-25T08:58:57.318 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T08:58:57.319 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T08:58:57.319 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T08:58:57.320 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T08:58:57.320 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T08:58:57.320 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T08:58:57.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:57.322 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-08-25T08:58:57.366 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T08:58:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T08:58:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T08:58:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-08-25T08:58:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-08-25T08:58:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-08-25T08:58:57.369 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T08:58:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-08-25T08:58:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-08-25T08:58:57.369 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T08:58:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-08-25T08:58:57.370 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-08-25T08:58:57.370 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T08:58:57.370 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-08-25T08:58:57.370 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T08:58:57.370 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-08-25T08:58:57.370 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T08:58:57.371 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 41655sec preferred_lft 41655sec 2023-08-25T08:58:57.371 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::4045:82b9:bb79:8b92/64 scope link noprefixroute 2023-08-25T08:58:57.371 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-08-25T08:58:57.372 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-25T08:58:57.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:57.373 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-08-25T08:58:57.373 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-08-25T08:58:57.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2023-08-25T08:58:57.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2023-08-25T08:58:57.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2023-08-25T08:58:57.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-25T08:58:57.374 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-08-25T08:58:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:57 smithi157 ceph-mon[130879]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2023-08-25T08:58:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:57 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1608103831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T08:58:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:57 smithi157 ceph-mon[130879]: osdmap e87: 12 total, 12 up, 12 in 2023-08-25T08:58:57.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:57 smithi181 ceph-mon[131296]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2023-08-25T08:58:57.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:57 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1608103831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T08:58:57.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:57 smithi181 ceph-mon[131296]: osdmap e87: 12 total, 12 up, 12 in 2023-08-25T08:58:57.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:57.839 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T08:58:57.910 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-08-25T08:58:57.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:57.914 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-08-25T08:58:57.979 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-08-25T08:58:57.980 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T08:58:57.980 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T08:58:57.980 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T08:58:57.981 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T08:58:57.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:57.983 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-08-25T08:58:57.984 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-08-25T08:58:57.984 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T08:58:57.984 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T08:58:57.984 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T08:58:57.985 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-08-25T08:58:58.733 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:58 smithi099 ceph-mon[97641]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-08-25T08:58:58.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:58.736 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-08-25T08:58:58.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:58.782 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2023-08-25T08:58:58.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:58.869 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-08-25T08:58:58.869 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-08-25T08:58:58.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T08:58:58.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-08-25T08:58:58.871 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-08-25T08:58:58.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:58 smithi157 ceph-mon[130879]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-08-25T08:58:58.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:58 smithi181 ceph-mon[131296]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-08-25T08:58:59.060 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-08-25T08:58:59.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:59.065 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-08-25T08:58:59.066 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-08-25T08:58:59.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-08-25T08:58:59.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-25T08:58:59.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-08-25T08:58:59.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-08-25T08:58:59.068 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-08-25T08:58:59.068 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-08-25T08:58:59.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:59.351 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-08-25T08:58:59.352 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-08-25T08:58:59.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-08-25T08:58:59.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2023-08-25T08:58:59.352 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-08-25T08:58:59.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T08:58:59.526 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T08:58:59.575 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T08:58:59.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T08:58:59.576 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T08:58:59.662 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-08-25T08:58:59.663 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-08-25T08:58:59.663 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/a9ce6aa6-bf74-4eec-a0a8-e527a306088d 2023-08-25T08:58:59.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:58:59.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/a9ce6aa6-bf74-4eec-a0a8-e527a306088d /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2023-08-25T08:58:59.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:58:59 smithi099 ceph-mon[97641]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 7 op/s 2023-08-25T08:58:59.820 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2023-08-25T08:58:59.820 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2023-08-25T08:58:59.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:58:59 smithi157 ceph-mon[130879]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 7 op/s 2023-08-25T08:58:59.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:58:59 smithi181 ceph-mon[131296]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 7 op/s 2023-08-25T08:59:00.274 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2023-08-25T08:59:00.275 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2023-08-25T08:59:00.275 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.157:3300,172.21.15.181:3300" 2023-08-25T08:59:00.275 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@3d4188ec-4324-11ee-9b3d-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/a9ce6aa6-bf74-4eec-a0a8-e527a306088d 2023-08-25T08:59:00.275 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.99:3300/172.21.15.157:3300/172.21.15.181:3300" will pass to kernel 2023-08-25T08:59:00.280 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-25T08:59:00.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:59:00.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T08:59:00.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:59:00.357 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T08:59:00.465 INFO:teuthology.orchestra.run.smithi099.stdout:client25171 2023-08-25T08:59:00.466 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:59:00.466 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T08:59:00.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:59:00.511 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T08:59:00.637 INFO:teuthology.orchestra.run.smithi099.stdout:client25171 2023-08-25T08:59:00.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:59:00.638 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T08:59:00.724 INFO:teuthology.orchestra.run.smithi099.stdout:client25171 2023-08-25T08:59:00.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T08:59:00.726 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c.client25171/status') 2023-08-25T08:59:00.832 INFO:teuthology.orchestra.run.smithi099.stdout:instance: client.25171 (3)172.21.15.99:0/1696630717 2023-08-25T08:59:00.833 INFO:teuthology.orchestra.run.smithi099.stdout:blocklisted: false 2023-08-25T08:59:00.833 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records in 2023-08-25T08:59:00.833 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records out 2023-08-25T08:59:00.833 INFO:teuthology.orchestra.run.smithi099.stderr:71 bytes copied, 5.4628e-05 s, 1.3 MB/s 2023-08-25T08:59:00.835 INFO:teuthology.run_tasks:Running task check-counter... 2023-08-25T08:59:00.847 INFO:teuthology.run_tasks:Running task exec... 2023-08-25T08:59:00.858 INFO:teuthology.task.exec:Executing custom commands... 2023-08-25T08:59:00.858 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T08:59:00.858 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-08-25T08:59:01.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.050+0000 7fec08dff700 1 -- 172.21.15.99:0/2803416271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec041024c0 msgr2=0x7fec04108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:01.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.050+0000 7fec08dff700 1 --2- 172.21.15.99:0/2803416271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec041024c0 0x7fec04108d70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febfc00b0a0 tx=0x7febfc00b3b0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.051+0000 7fec08dff700 1 -- 172.21.15.99:0/2803416271 shutdown_connections 2023-08-25T08:59:01.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.051+0000 7fec08dff700 1 --2- 172.21.15.99:0/2803416271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec041024c0 0x7fec04108d70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.051+0000 7fec08dff700 1 --2- 172.21.15.99:0/2803416271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec04100930 0x7fec04100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.051+0000 7fec08dff700 1 --2- 172.21.15.99:0/2803416271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec04101b20 0x7fec04101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.051+0000 7fec08dff700 1 -- 172.21.15.99:0/2803416271 >> 172.21.15.99:0/2803416271 conn(0x7fec040fbe60 msgr2=0x7fec040fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:01.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.051+0000 7fec08dff700 1 -- 172.21.15.99:0/2803416271 shutdown_connections 2023-08-25T08:59:01.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.051+0000 7fec08dff700 1 -- 172.21.15.99:0/2803416271 wait complete. 2023-08-25T08:59:01.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.052+0000 7fec08dff700 1 Processor -- start 2023-08-25T08:59:01.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.052+0000 7fec08dff700 1 -- start start 2023-08-25T08:59:01.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.052+0000 7fec08dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec04100930 0x7fec04198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:01.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.052+0000 7fec08dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04101b20 0x7fec04198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:01.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.052+0000 7fec08dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec041024c0 0x7fec04199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:01.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.052+0000 7fec08dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0419a460 con 0x7fec041024c0 2023-08-25T08:59:01.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.052+0000 7fec08dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0419a5d0 con 0x7fec04100930 2023-08-25T08:59:01.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.052+0000 7fec08dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0419a790 con 0x7fec04101b20 2023-08-25T08:59:01.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04101b20 0x7fec04198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:01.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04101b20 0x7fec04198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34124/0 (socket says 172.21.15.99:34124) 2023-08-25T08:59:01.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 -- 172.21.15.99:0/872987174 learned_addr learned my addr 172.21.15.99:0/872987174 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:01.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec02d9d700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec041024c0 0x7fec04199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:01.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec0259c700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec04100930 0x7fec04198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:01.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 -- 172.21.15.99:0/872987174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec04100930 msgr2=0x7fec04198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:01.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec04100930 0x7fec04198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 -- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec041024c0 msgr2=0x7fec04199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:01.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec041024c0 0x7fec04199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 -- 172.21.15.99:0/872987174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc009d00 con 0x7fec04101b20 2023-08-25T08:59:01.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec01d9b700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04101b20 0x7fec04198f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7febf800e9a0 tx=0x7febf800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:01.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7febf37fe700 1 -- 172.21.15.99:0/872987174 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8009790 con 0x7fec04101b20 2023-08-25T08:59:01.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7febf37fe700 1 -- 172.21.15.99:0/872987174 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf8010430 con 0x7fec04101b20 2023-08-25T08:59:01.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0419aa70 con 0x7fec04101b20 2023-08-25T08:59:01.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.053+0000 7febf37fe700 1 -- 172.21.15.99:0/872987174 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8018680 con 0x7fec04101b20 2023-08-25T08:59:01.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.054+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec041a1230 con 0x7fec04101b20 2023-08-25T08:59:01.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.055+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec04192f90 con 0x7fec04101b20 2023-08-25T08:59:01.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.057+0000 7febf37fe700 1 -- 172.21.15.99:0/872987174 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 134671+0+0 (secure 0 0 0) 0x7febf80105a0 con 0x7fec04101b20 2023-08-25T08:59:01.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.058+0000 7febf37fe700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7febec09ffd0 0x7febec0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:01.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.058+0000 7febf37fe700 1 -- 172.21.15.99:0/872987174 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8327+0+0 (secure 0 0 0) 0x7febf8014070 con 0x7fec04101b20 2023-08-25T08:59:01.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.058+0000 7fec0259c700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7febec09ffd0 0x7febec0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:01.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.059+0000 7fec0259c700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7febec09ffd0 0x7febec0a2490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec04101980 tx=0x7febf4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:01.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.060+0000 7febf37fe700 1 -- 172.21.15.99:0/872987174 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7febf8095130 con 0x7fec04101b20 2023-08-25T08:59:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.318+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fec0410c1a0 con 0x7fec04101b20 2023-08-25T08:59:01.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.586+0000 7febf37fe700 1 -- 172.21.15.99:0/872987174 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 19) v1 ==== 134688+0+0 (secure 0 0 0) 0x7febf8010880 con 0x7fec04101b20 2023-08-25T08:59:01.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.587+0000 7febf37fe700 1 -- 172.21.15.99:0/872987174 <== mon.1 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7febf8097190 con 0x7fec04101b20 2023-08-25T08:59:01.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7febec09ffd0 msgr2=0x7febec0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:01.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7febec09ffd0 0x7febec0a2490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec04101980 tx=0x7febf4009260 comp rx=0 tx=0).stop 2023-08-25T08:59:01.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04101b20 msgr2=0x7fec04198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:01.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04101b20 0x7fec04198f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7febf800e9a0 tx=0x7febf800ecb0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 shutdown_connections 2023-08-25T08:59:01.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec041024c0 0x7fec04199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7febec09ffd0 0x7febec0a2490 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec04100930 0x7fec04198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 --2- 172.21.15.99:0/872987174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04101b20 0x7fec04198f80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 >> 172.21.15.99:0/872987174 conn(0x7fec040fbe60 msgr2=0x7fec041073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 shutdown_connections 2023-08-25T08:59:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.590+0000 7fec08dff700 1 -- 172.21.15.99:0/872987174 wait complete. 2023-08-25T08:59:01.609 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-08-25T08:59:01.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 -- 172.21.15.99:0/887064527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 msgr2=0x7f5770100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:01.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 --2- 172.21.15.99:0/887064527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 0x7f5770100d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f576c009990 tx=0x7f576c009ca0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 -- 172.21.15.99:0/887064527 shutdown_connections 2023-08-25T08:59:01.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 --2- 172.21.15.99:0/887064527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57701024e0 0x7f5770108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 --2- 172.21.15.99:0/887064527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 0x7f5770100d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 --2- 172.21.15.99:0/887064527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5770101b40 0x7f5770101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 -- 172.21.15.99:0/887064527 >> 172.21.15.99:0/887064527 conn(0x7f57700fbea0 msgr2=0x7f57700fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:01.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 -- 172.21.15.99:0/887064527 shutdown_connections 2023-08-25T08:59:01.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.814+0000 7f5777bff700 1 -- 172.21.15.99:0/887064527 wait complete. 2023-08-25T08:59:01.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.815+0000 7f5777bff700 1 Processor -- start 2023-08-25T08:59:01.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.815+0000 7f5777bff700 1 -- start start 2023-08-25T08:59:01.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f5777bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 0x7f5770072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:01.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f5777bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5770101b40 0x7f5770073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:01.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f5777bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57701024e0 0x7f5770073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:01.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f5777bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701a06d0 con 0x7f57701024e0 2023-08-25T08:59:01.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f5777bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701a0810 con 0x7f5770101b40 2023-08-25T08:59:01.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f5777bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701a0950 con 0x7f5770100950 2023-08-25T08:59:01.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577599b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 0x7f5770072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:01.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577599b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 0x7f5770072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34142/0 (socket says 172.21.15.99:34142) 2023-08-25T08:59:01.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577599b700 1 -- 172.21.15.99:0/2423328396 learned_addr learned my addr 172.21.15.99:0/2423328396 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:01.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577519a700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5770101b40 0x7f5770073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:01.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577619c700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57701024e0 0x7f5770073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:01.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577599b700 1 -- 172.21.15.99:0/2423328396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5770101b40 msgr2=0x7f5770073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:01.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577599b700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5770101b40 0x7f5770073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577599b700 1 -- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57701024e0 msgr2=0x7f5770073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:01.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577599b700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57701024e0 0x7f5770073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:01.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.816+0000 7f577599b700 1 -- 172.21.15.99:0/2423328396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0095f0 con 0x7f5770100950 2023-08-25T08:59:01.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.817+0000 7f577599b700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 0x7f5770072d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f576c005720 tx=0x7f576c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:01.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.817+0000 7f5766ffd700 1 -- 172.21.15.99:0/2423328396 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c01d070 con 0x7f5770100950 2023-08-25T08:59:01.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.817+0000 7f5766ffd700 1 -- 172.21.15.99:0/2423328396 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f576c021410 con 0x7f5770100950 2023-08-25T08:59:01.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.817+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57701a0b80 con 0x7f5770100950 2023-08-25T08:59:01.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.817+0000 7f5766ffd700 1 -- 172.21.15.99:0/2423328396 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c017650 con 0x7f5770100950 2023-08-25T08:59:01.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.817+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57701a1000 con 0x7f5770100950 2023-08-25T08:59:01.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.818+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577004e700 con 0x7f5770100950 2023-08-25T08:59:01.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.820+0000 7f5766ffd700 1 -- 172.21.15.99:0/2423328396 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 134688+0+0 (secure 0 0 0) 0x7f576c021580 con 0x7f5770100950 2023-08-25T08:59:01.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.821+0000 7f5766ffd700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f575c0a0020 0x7f575c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:01.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.821+0000 7f577519a700 1 -- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f575c0a0020 msgr2=0x7f575c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:01.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.821+0000 7f5766ffd700 1 -- 172.21.15.99:0/2423328396 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8327+0+0 (secure 0 0 0) 0x7f576c0cc3f0 con 0x7f5770100950 2023-08-25T08:59:01.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.821+0000 7f577519a700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f575c0a0020 0x7f575c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:59:01.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:01.823+0000 7f5766ffd700 1 -- 172.21.15.99:0/2423328396 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f576c096200 con 0x7f5770100950 2023-08-25T08:59:01.836 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-mon[130879]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T08:59:01.837 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/872987174' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T08:59:01.837 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T08:59:01.837 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: ignoring --setuser ceph since I am not root 2023-08-25T08:59:01.837 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: ignoring --setgroup ceph since I am not root 2023-08-25T08:59:01.838 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-mgr[132358]: -- 172.21.15.157:0/3032251223 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d51e63d1e0 con 0x55d51e612c00 2023-08-25T08:59:01.838 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:01.756+0000 7f9a6eeb6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T08:59:01.838 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:01.836+0000 7f9a6eeb6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T08:59:01.842 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: ignoring --setuser ceph since I am not root 2023-08-25T08:59:01.842 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: ignoring --setgroup ceph since I am not root 2023-08-25T08:59:01.842 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-mgr[133436]: -- 172.21.15.181:0/343051059 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dfa1ed91e0 con 0x55dfa1eaec00 2023-08-25T08:59:01.842 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:01.760+0000 7fbf29292200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T08:59:01.843 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:01.841+0000 7fbf29292200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T08:59:01.843 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-mon[131296]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T08:59:01.843 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/872987174' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T08:59:01.843 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T08:59:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:01 smithi099 ceph-mon[97641]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T08:59:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:01 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/872987174' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T08:59:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:01 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T08:59:01.863 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: ignoring --setuser ceph since I am not root 2023-08-25T08:59:01.864 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: ignoring --setgroup ceph since I am not root 2023-08-25T08:59:02.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.022+0000 7f577519a700 1 -- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f575c0a0020 msgr2=0x7f575c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:02.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.022+0000 7f577519a700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f575c0a0020 0x7f575c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T08:59:02.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.030+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f5770107830 con 0x7f5770100950 2023-08-25T08:59:02.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.037+0000 7f5766ffd700 1 -- 172.21.15.99:0/2423328396 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f576c099820 con 0x7f5770100950 2023-08-25T08:59:02.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f575c0a0020 msgr2=0x7f575c0a24e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:59:02.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f575c0a0020 0x7f575c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 msgr2=0x7f5770072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:02.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 0x7f5770072d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f576c005720 tx=0x7f576c009580 comp rx=0 tx=0).stop 2023-08-25T08:59:02.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 shutdown_connections 2023-08-25T08:59:02.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57701024e0 0x7f5770073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100950 0x7f5770072d10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f575c0a0020 0x7f575c0a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 --2- 172.21.15.99:0/2423328396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5770101b40 0x7f5770073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 >> 172.21.15.99:0/2423328396 conn(0x7f57700fbea0 msgr2=0x7f57700fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:02.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 shutdown_connections 2023-08-25T08:59:02.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.040+0000 7f5777bff700 1 -- 172.21.15.99:0/2423328396 wait complete. 2023-08-25T08:59:02.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-08-25T08:59:02.121 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:01.861+0000 7fedb2b00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T08:59:02.122 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:01.967+0000 7fedb2b00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T08:59:02.122 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:02.085+0000 7fedb2b00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T08:59:02.129 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:01.927+0000 7f9a6eeb6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T08:59:02.129 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:02.019+0000 7f9a6eeb6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T08:59:02.196 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:01.932+0000 7fbf29292200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T08:59:02.197 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:02.025+0000 7fbf29292200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T08:59:02.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 -- 172.21.15.99:0/956146910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40041024e0 msgr2=0x7f4004108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:02.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 --2- 172.21.15.99:0/956146910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40041024e0 0x7f4004108d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009fc0 tx=0x7f3ffc00b330 comp rx=0 tx=0).stop 2023-08-25T08:59:02.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 -- 172.21.15.99:0/956146910 shutdown_connections 2023-08-25T08:59:02.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 --2- 172.21.15.99:0/956146910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40041024e0 0x7f4004108d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 --2- 172.21.15.99:0/956146910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4004100950 0x7f4004100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 --2- 172.21.15.99:0/956146910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4004101b40 0x7f4004101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 -- 172.21.15.99:0/956146910 >> 172.21.15.99:0/956146910 conn(0x7f40040fbea0 msgr2=0x7f40040fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:02.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 -- 172.21.15.99:0/956146910 shutdown_connections 2023-08-25T08:59:02.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.260+0000 7f400a7ff700 1 -- 172.21.15.99:0/956146910 wait complete. 2023-08-25T08:59:02.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.261+0000 7f400a7ff700 1 Processor -- start 2023-08-25T08:59:02.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.261+0000 7f400a7ff700 1 -- start start 2023-08-25T08:59:02.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.261+0000 7f400a7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4004100950 0x7f4004072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.261+0000 7f400a7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4004101b40 0x7f4004073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.261+0000 7f400a7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40041024e0 0x7f4004073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.261+0000 7f400a7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40040748b0 con 0x7f4004101b40 2023-08-25T08:59:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.261+0000 7f400a7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4004074a20 con 0x7f40041024e0 2023-08-25T08:59:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.261+0000 7f400a7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4004074be0 con 0x7f4004100950 2023-08-25T08:59:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4004100950 0x7f4004072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f40037fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4004101b40 0x7f4004073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:02.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4004100950 0x7f4004072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34146/0 (socket says 172.21.15.99:34146) 2023-08-25T08:59:02.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 -- 172.21.15.99:0/2568148934 learned_addr learned my addr 172.21.15.99:0/2568148934 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:02.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4008d9c700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40041024e0 0x7f4004073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:02.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 -- 172.21.15.99:0/2568148934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40041024e0 msgr2=0x7f4004073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:02.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40041024e0 0x7f4004073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 -- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4004101b40 msgr2=0x7f4004073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4004101b40 0x7f4004073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 -- 172.21.15.99:0/2568148934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff40095f0 con 0x7f4004100950 2023-08-25T08:59:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f4003fff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4004100950 0x7f4004072d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ebf0 tx=0x7f3ff400ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.262+0000 7f40017fa700 1 -- 172.21.15.99:0/2568148934 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4009a10 con 0x7f4004100950 2023-08-25T08:59:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.263+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc009c40 con 0x7f4004100950 2023-08-25T08:59:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.263+0000 7f40017fa700 1 -- 172.21.15.99:0/2568148934 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff400cc90 con 0x7f4004100950 2023-08-25T08:59:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.263+0000 7f40017fa700 1 -- 172.21.15.99:0/2568148934 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4018710 con 0x7f4004100950 2023-08-25T08:59:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.263+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40041a54a0 con 0x7f4004100950 2023-08-25T08:59:02.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.264+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4004107e40 con 0x7f4004100950 2023-08-25T08:59:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.265+0000 7f40017fa700 1 -- 172.21.15.99:0/2568148934 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 134688+0+0 (secure 0 0 0) 0x7f3ff40188b0 con 0x7f4004100950 2023-08-25T08:59:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.267+0000 7f40017fa700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fec09ffa0 0x7f3fec0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.267+0000 7f40037fe700 1 -- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fec09ffa0 msgr2=0x7f3fec0a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.267+0000 7f40017fa700 1 -- 172.21.15.99:0/2568148934 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8327+0+0 (secure 0 0 0) 0x7f3ff4014070 con 0x7f4004100950 2023-08-25T08:59:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.267+0000 7f40037fe700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fec09ffa0 0x7f3fec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:59:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.270+0000 7f40017fa700 1 -- 172.21.15.99:0/2568148934 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3ff4095490 con 0x7f4004100950 2023-08-25T08:59:02.446 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:02.212+0000 7fedb2b00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T08:59:02.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.468+0000 7f40037fe700 1 -- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fec09ffa0 msgr2=0x7f3fec0a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:02.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.468+0000 7f40037fe700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fec09ffa0 0x7f3fec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T08:59:02.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.479+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f400410c400 con 0x7f4004100950 2023-08-25T08:59:02.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.487+0000 7f40017fa700 1 -- 172.21.15.99:0/2568148934 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3ff4098ab0 con 0x7f4004100950 2023-08-25T08:59:02.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fec09ffa0 msgr2=0x7f3fec0a2460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:59:02.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fec09ffa0 0x7f3fec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4004100950 msgr2=0x7f4004072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:02.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4004100950 0x7f4004072d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ebf0 tx=0x7f3ff400ef00 comp rx=0 tx=0).stop 2023-08-25T08:59:02.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 shutdown_connections 2023-08-25T08:59:02.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f3fec09ffa0 0x7f3fec0a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40041024e0 0x7f4004073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4004100950 0x7f4004072d10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 --2- 172.21.15.99:0/2568148934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4004101b40 0x7f4004073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 >> 172.21.15.99:0/2568148934 conn(0x7f40040fbea0 msgr2=0x7f4004104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:02.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 shutdown_connections 2023-08-25T08:59:02.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.489+0000 7f400a7ff700 1 -- 172.21.15.99:0/2568148934 wait complete. 2023-08-25T08:59:02.492 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:02.195+0000 7fbf29292200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T08:59:02.510 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-08-25T08:59:02.554 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:02.191+0000 7f9a6eeb6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T08:59:02.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 -- 172.21.15.99:0/835644931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0810100940 msgr2=0x7f0810100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:02.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 --2- 172.21.15.99:0/835644931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0810100940 0x7f0810100d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f08000099c0 tx=0x7f0800009cd0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 -- 172.21.15.99:0/835644931 shutdown_connections 2023-08-25T08:59:02.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 --2- 172.21.15.99:0/835644931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08101024d0 0x7f0810108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 --2- 172.21.15.99:0/835644931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0810100940 0x7f0810100d60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 --2- 172.21.15.99:0/835644931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0810101b30 0x7f0810101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 -- 172.21.15.99:0/835644931 >> 172.21.15.99:0/835644931 conn(0x7f08100fbeb0 msgr2=0x7f08100fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:02.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 -- 172.21.15.99:0/835644931 shutdown_connections 2023-08-25T08:59:02.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.710+0000 7f08159ff700 1 -- 172.21.15.99:0/835644931 wait complete. 2023-08-25T08:59:02.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.712+0000 7f08159ff700 1 Processor -- start 2023-08-25T08:59:02.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.712+0000 7f08159ff700 1 -- start start 2023-08-25T08:59:02.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.712+0000 7f08159ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0810100940 0x7f0810198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:02.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.712+0000 7f08159ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0810101b30 0x7f0810198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:02.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f08159ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08101024d0 0x7f0810199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:02.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f08159ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081019a350 con 0x7f0810101b30 2023-08-25T08:59:02.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f08159ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081019a4c0 con 0x7f08101024d0 2023-08-25T08:59:02.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f08159ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081019a680 con 0x7f0810100940 2023-08-25T08:59:02.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0810100940 0x7f0810198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:02.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0810101b30 0x7f0810198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:02.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0810100940 0x7f0810198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34166/0 (socket says 172.21.15.99:34166) 2023-08-25T08:59:02.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 -- 172.21.15.99:0/37028566 learned_addr learned my addr 172.21.15.99:0/37028566 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:02.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080f7fe700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08101024d0 0x7f0810199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:02.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08101024d0 msgr2=0x7f0810199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:02.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08101024d0 0x7f0810199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0810101b30 msgr2=0x7f0810198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:02.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0810101b30 0x7f0810198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:02.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 -- 172.21.15.99:0/37028566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000095f0 con 0x7f0810100940 2023-08-25T08:59:02.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.713+0000 7f080effd700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0810100940 0x7f0810198930 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0800009fd0 tx=0x7f080000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:02.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.714+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080001d070 con 0x7f0810100940 2023-08-25T08:59:02.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.714+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081019a900 con 0x7f0810100940 2023-08-25T08:59:02.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.714+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0800005050 con 0x7f0810100940 2023-08-25T08:59:02.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.714+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800004a70 con 0x7f0810100940 2023-08-25T08:59:02.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.714+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08101a1120 con 0x7f0810100940 2023-08-25T08:59:02.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.715+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0810192f90 con 0x7f0810100940 2023-08-25T08:59:02.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.717+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 134688+0+0 (secure 0 0 0) 0x7f0800005450 con 0x7f0810100940 2023-08-25T08:59:02.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.720+0000 7f08149fd700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 0x7f07f80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:02.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.720+0000 7f080e7fc700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 msgr2=0x7f07f80a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:02.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.720+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8327+0+0 (secure 0 0 0) 0x7f08000cc0c0 con 0x7f0810100940 2023-08-25T08:59:02.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.720+0000 7f080e7fc700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 0x7f07f80a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T08:59:02.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.723+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0800095e20 con 0x7f0810100940 2023-08-25T08:59:02.812 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:02.444+0000 7fedb2b00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T08:59:02.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:02 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-25T08:59:02.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:02 smithi099 ceph-mon[97641]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-08-25T08:59:02.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:02 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' 2023-08-25T08:59:02.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:02 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' 2023-08-25T08:59:02.876 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-25T08:59:02.876 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-mon[130879]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-08-25T08:59:02.877 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' 2023-08-25T08:59:02.877 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' 2023-08-25T08:59:02.878 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:02.554+0000 7f9a6eeb6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T08:59:02.878 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:02.688+0000 7f9a6eeb6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T08:59:02.879 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:02.780+0000 7f9a6eeb6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T08:59:02.879 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:02.875+0000 7f9a6eeb6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T08:59:02.882 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:02.561+0000 7fbf29292200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T08:59:02.882 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:02.693+0000 7fbf29292200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T08:59:02.882 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:02.785+0000 7fbf29292200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T08:59:02.882 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-25T08:59:02.884 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-mon[131296]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-08-25T08:59:02.884 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' 2023-08-25T08:59:02.884 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' 2023-08-25T08:59:02.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.921+0000 7f080e7fc700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 msgr2=0x7f07f80a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:02.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.921+0000 7f080e7fc700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 0x7f07f80a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T08:59:02.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:02.954+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 --> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0810060d60 con 0x7f07f809ffa0 2023-08-25T08:59:03.199 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:02.918+0000 7fedb2b00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T08:59:03.199 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:03.083+0000 7fedb2b00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T08:59:03.243 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:02.881+0000 7fbf29292200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T08:59:03.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:03.322+0000 7f080e7fc700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 msgr2=0x7f07f80a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:03.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:03.322+0000 7f080e7fc700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 0x7f07f80a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-25T08:59:03.562 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:03.198+0000 7fedb2b00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T08:59:03.563 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:03.320+0000 7fedb2b00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T08:59:03.629 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:03.297+0000 7f9a6eeb6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T08:59:03.742 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:03.303+0000 7fbf29292200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T08:59:04.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:04.123+0000 7f080e7fc700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 msgr2=0x7f07f80a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:04.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:04.123+0000 7f080e7fc700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 0x7f07f80a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-25T08:59:04.242 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:03.960+0000 7fbf29292200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T08:59:04.243 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:04.041+0000 7fbf29292200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T08:59:04.312 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:03.862+0000 7fedb2b00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T08:59:04.379 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:03.955+0000 7f9a6eeb6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T08:59:04.379 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:04.037+0000 7f9a6eeb6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T08:59:05.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:04 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:04.703+0000 7fedb2b00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T08:59:05.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:04 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:04.809+0000 7fedb2b00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T08:59:05.076 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:04.813+0000 7f9a6eeb6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:59:05.076 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:04.903+0000 7f9a6eeb6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T08:59:05.081 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:04.816+0000 7fbf29292200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:59:05.081 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:04.907+0000 7fbf29292200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T08:59:05.357 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:05.075+0000 7f9a6eeb6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:59:05.361 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:05.081+0000 7fbf29292200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:59:05.629 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:05.356+0000 7f9a6eeb6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T08:59:05.629 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:05.584+0000 7f9a6eeb6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T08:59:05.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:05.725+0000 7f080e7fc700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 msgr2=0x7f07f80a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:05.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:05.725+0000 7f080e7fc700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 0x7f07f80a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-25T08:59:05.737 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:05.360+0000 7fbf29292200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T08:59:05.738 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:05.589+0000 7fbf29292200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T08:59:05.992 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:05.737+0000 7fbf29292200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T08:59:05.996 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:05.732+0000 7f9a6eeb6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T08:59:06.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:05.760+0000 7fedb2b00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:59:06.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:05.880+0000 7fedb2b00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T08:59:06.379 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:05.996+0000 7f9a6eeb6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T08:59:06.473 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:06.107+0000 7fedb2b00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T08:59:06.492 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:06.002+0000 7fbf29292200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T08:59:06.751 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:06.472+0000 7fedb2b00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T08:59:06.992 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:06.705+0000 7fbf29292200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T08:59:06.993 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:06.836+0000 7fbf29292200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T08:59:07.023 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:06 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:06.699+0000 7f9a6eeb6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T08:59:07.024 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:06 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:06.831+0000 7f9a6eeb6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T08:59:07.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:06.750+0000 7fedb2b00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T08:59:07.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:06.941+0000 7fedb2b00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T08:59:07.352 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:07.023+0000 7f9a6eeb6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T08:59:07.352 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:07.136+0000 7f9a6eeb6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T08:59:07.361 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:07.030+0000 7fbf29292200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T08:59:07.361 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:07.142+0000 7fbf29292200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T08:59:07.562 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:07 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:07.287+0000 7fedb2b00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T08:59:07.629 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:07.351+0000 7f9a6eeb6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T08:59:07.742 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:07.359+0000 7fbf29292200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T08:59:08.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:08.287+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 20) v1 ==== 134688+0+0 (secure 0 0 0) 0x7f08000961f0 con 0x7f0810100940 2023-08-25T08:59:08.492 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z[133369]: 2023-08-25T08:59:08.220+0000 7fbf29292200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T08:59:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: Standby manager daemon y restarted 2023-08-25T08:59:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: Standby manager daemon y started 2023-08-25T08:59:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T08:59:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:59:08.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T08:59:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:59:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: Standby manager daemon z restarted 2023-08-25T08:59:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: Standby manager daemon z started 2023-08-25T08:59:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T08:59:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:59:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T08:59:08.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:08 smithi181 ceph-mon[131296]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:59:08.562 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:08.216+0000 7fedb2b00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T08:59:08.563 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:08.397+0000 7fedb2b00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T08:59:08.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: Standby manager daemon y restarted 2023-08-25T08:59:08.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: Standby manager daemon y started 2023-08-25T08:59:08.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T08:59:08.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:59:08.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T08:59:08.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:59:08.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: Standby manager daemon z restarted 2023-08-25T08:59:08.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: Standby manager daemon z started 2023-08-25T08:59:08.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T08:59:08.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:59:08.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T08:59:08.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-mon[97641]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:59:08.629 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y[132334]: 2023-08-25T08:59:08.212+0000 7f9a6eeb6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T08:59:08.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: Standby manager daemon y restarted 2023-08-25T08:59:08.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: Standby manager daemon y started 2023-08-25T08:59:08.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T08:59:08.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:59:08.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T08:59:08.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.157:0/138260549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:59:08.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: Standby manager daemon z restarted 2023-08-25T08:59:08.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: Standby manager daemon z started 2023-08-25T08:59:08.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T08:59:08.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T08:59:08.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T08:59:08.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:08 smithi157 ceph-mon[130879]: from='mgr.? 172.21.15.181:0/2597246252' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T08:59:08.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:08.928+0000 7f080e7fc700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 msgr2=0x7f07f80a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/4068336524 2023-08-25T08:59:08.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:08.928+0000 7f080e7fc700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 0x7f07f80a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-25T08:59:09.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:08.643+0000 7fedb2b00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T08:59:09.063 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:08.789+0000 7fedb2b00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T08:59:09.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:09 smithi099 ceph-mon[97641]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-08-25T08:59:09.562 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:09 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:09.070+0000 7fedb2b00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T08:59:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:09 smithi157 ceph-mon[130879]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-08-25T08:59:09.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:09 smithi181 ceph-mon[131296]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-08-25T08:59:10.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:10.161+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 21) v1 ==== 134356+0+0 (secure 0 0 0) 0x7f08000c9b80 con 0x7f0810100940 2023-08-25T08:59:10.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:10.162+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 msgr2=0x7f07f80a2460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T08:59:10.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:10.162+0000 7f08149fd700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/4068336524,v1:172.21.15.99:6801/4068336524] conn(0x7f07f809ffa0 0x7f07f80a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:10.445 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.141+0000 7fedb2b00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T08:59:10.445 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed14530700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.445 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed14530700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.446 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed14530700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.446 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed14530700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.446 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed14530700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.446 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed1152a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.447 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed1152a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.447 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed1152a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.447 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed1152a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.447 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T08:59:10.360+0000 7fed1152a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T08:59:10.448 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: Active manager daemon x restarted 2023-08-25T08:59:10.448 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: Activating manager daemon x 2023-08-25T08:59:10.449 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: osdmap e88: 12 total, 12 up, 12 in 2023-08-25T08:59:10.449 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: mgrmap e21: x(active, starting, since 0.0122746s), standbys: y, z 2023-08-25T08:59:10.449 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:59:10.449 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:59:10.450 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:59:10.450 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T08:59:10.450 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T08:59:10.451 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T08:59:10.451 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T08:59:10.451 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T08:59:10.451 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T08:59:10.452 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T08:59:10.452 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T08:59:10.452 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T08:59:10.452 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T08:59:10.453 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T08:59:10.453 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T08:59:10.453 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T08:59:10.453 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T08:59:10.454 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T08:59:10.454 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:59:10.454 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:59:10.454 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:59:10.455 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:59:10.455 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:59:10.455 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:59:10.456 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:59:10.456 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:59:10.456 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:59:10.456 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:59:10.457 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:59:10.457 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:59:10.457 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T08:59:10.457 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T08:59:10.458 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T08:59:10.458 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: Manager daemon x is now available 2023-08-25T08:59:10.458 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:10.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:10.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T08:59:10.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:10 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T08:59:10.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: Active manager daemon x restarted 2023-08-25T08:59:10.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: Activating manager daemon x 2023-08-25T08:59:10.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: osdmap e88: 12 total, 12 up, 12 in 2023-08-25T08:59:10.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: mgrmap e21: x(active, starting, since 0.0122746s), standbys: y, z 2023-08-25T08:59:10.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:59:10.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:59:10.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:59:10.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T08:59:10.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T08:59:10.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T08:59:10.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T08:59:10.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T08:59:10.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T08:59:10.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T08:59:10.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T08:59:10.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T08:59:10.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T08:59:10.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T08:59:10.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T08:59:10.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T08:59:10.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T08:59:10.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T08:59:10.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:59:10.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:59:10.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:59:10.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:59:10.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:59:10.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:59:10.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:59:10.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:59:10.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:59:10.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:59:10.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:59:10.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:59:10.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T08:59:10.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T08:59:10.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T08:59:10.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: Manager daemon x is now available 2023-08-25T08:59:10.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:10.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:10.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T08:59:10.635 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:10 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T08:59:10.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: Active manager daemon x restarted 2023-08-25T08:59:10.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: Activating manager daemon x 2023-08-25T08:59:10.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: osdmap e88: 12 total, 12 up, 12 in 2023-08-25T08:59:10.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: mgrmap e21: x(active, starting, since 0.0122746s), standbys: y, z 2023-08-25T08:59:10.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T08:59:10.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T08:59:10.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T08:59:10.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T08:59:10.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T08:59:10.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T08:59:10.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T08:59:10.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T08:59:10.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T08:59:10.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T08:59:10.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T08:59:10.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T08:59:10.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T08:59:10.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T08:59:10.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T08:59:10.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T08:59:10.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T08:59:10.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T08:59:10.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T08:59:10.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T08:59:10.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T08:59:10.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T08:59:10.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T08:59:10.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T08:59:10.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T08:59:10.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T08:59:10.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T08:59:10.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T08:59:10.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T08:59:10.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T08:59:10.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T08:59:10.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T08:59:10.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T08:59:10.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: Manager daemon x is now available 2023-08-25T08:59:10.752 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:10.752 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:10.752 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T08:59:10.752 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:10 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T08:59:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.163+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mon.1 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 22) v1 ==== 134483+0+0 (secure 0 0 0) 0x7f080000fad0 con 0x7f0810100940 2023-08-25T08:59:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.163+0000 7f08149fd700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f07f801b0d0 0x7f07f801b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:11.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.163+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 --> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0810060d60 con 0x7f07f801b0d0 2023-08-25T08:59:11.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.166+0000 7f080e7fc700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f07f801b0d0 0x7f07f801b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.170+0000 7f080e7fc700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f07f801b0d0 0x7f07f801b4b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0804009880 tx=0x7f0804009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:11.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.232+0000 7f08149fd700 1 -- 172.21.15.99:0/37028566 <== mgr.25202 v2:172.21.15.99:6800/2884087217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0810060d60 con 0x7f07f801b0d0 2023-08-25T08:59:11.234 INFO:teuthology.orchestra.run.smithi099.stdout:Schedule set for path / 2023-08-25T08:59:11.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f07f801b0d0 msgr2=0x7f07f801b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:11.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f08159ff700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f07f801b0d0 0x7f07f801b4b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0804009880 tx=0x7f0804009260 comp rx=0 tx=0).stop 2023-08-25T08:59:11.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0810100940 msgr2=0x7f0810198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:11.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f08159ff700 1 --2- 172.21.15.99:0/37028566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0810100940 0x7f0810198930 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0800009fd0 tx=0x7f080000f690 comp rx=0 tx=0).stop 2023-08-25T08:59:11.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f080f7fe700 1 -- 172.21.15.99:0/37028566 reap_dead start 2023-08-25T08:59:11.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 shutdown_connections 2023-08-25T08:59:11.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 >> 172.21.15.99:0/37028566 conn(0x7f08100fbeb0 msgr2=0x7f0810107470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:11.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 shutdown_connections 2023-08-25T08:59:11.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.235+0000 7f08159ff700 1 -- 172.21.15.99:0/37028566 wait complete. 2023-08-25T08:59:11.256 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-08-25T08:59:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 -- 172.21.15.99:0/1140815560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634101b20 msgr2=0x7ff634101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 --2- 172.21.15.99:0/1140815560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634101b20 0x7ff634101f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff6240099c0 tx=0x7ff624009cd0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 -- 172.21.15.99:0/1140815560 shutdown_connections 2023-08-25T08:59:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 --2- 172.21.15.99:0/1140815560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6341024c0 0x7ff634108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 --2- 172.21.15.99:0/1140815560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634100930 0x7ff634100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 --2- 172.21.15.99:0/1140815560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634101b20 0x7ff634101f80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 -- 172.21.15.99:0/1140815560 >> 172.21.15.99:0/1140815560 conn(0x7ff6340fbe80 msgr2=0x7ff6340fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:11.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 -- 172.21.15.99:0/1140815560 shutdown_connections 2023-08-25T08:59:11.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.462+0000 7ff63bdff700 1 -- 172.21.15.99:0/1140815560 wait complete. 2023-08-25T08:59:11.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.463+0000 7ff63bdff700 1 Processor -- start 2023-08-25T08:59:11.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.463+0000 7ff63bdff700 1 -- start start 2023-08-25T08:59:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63bdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634100930 0x7ff634196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63bdff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff634101b20 0x7ff634196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63bdff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6341024c0 0x7ff634197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63bdff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff634198180 con 0x7ff6341024c0 2023-08-25T08:59:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63bdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341982f0 con 0x7ff634100930 2023-08-25T08:59:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63bdff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341984b0 con 0x7ff634101b20 2023-08-25T08:59:11.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff639b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634100930 0x7ff634196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff639b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634100930 0x7ff634196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:56926/0 (socket says 172.21.15.99:56926) 2023-08-25T08:59:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff639b9b700 1 -- 172.21.15.99:0/3292284520 learned_addr learned my addr 172.21.15.99:0/3292284520 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63939a700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff634101b20 0x7ff634196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63a39c700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6341024c0 0x7ff634197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63939a700 1 -- 172.21.15.99:0/3292284520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634100930 msgr2=0x7ff634196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63939a700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634100930 0x7ff634196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63939a700 1 -- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6341024c0 msgr2=0x7ff634197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63939a700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6341024c0 0x7ff634197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff63939a700 1 -- 172.21.15.99:0/3292284520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6240095f0 con 0x7ff634101b20 2023-08-25T08:59:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.464+0000 7ff639b9b700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634100930 0x7ff634196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.465+0000 7ff63939a700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff634101b20 0x7ff634196ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6240098e0 tx=0x7ff62400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:11.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.465+0000 7ff62affd700 1 -- 172.21.15.99:0/3292284520 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62401d070 con 0x7ff634101b20 2023-08-25T08:59:11.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.465+0000 7ff62affd700 1 -- 172.21.15.99:0/3292284520 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff624005050 con 0x7ff634101b20 2023-08-25T08:59:11.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.465+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff634198730 con 0x7ff634101b20 2023-08-25T08:59:11.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.465+0000 7ff62affd700 1 -- 172.21.15.99:0/3292284520 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624004ae0 con 0x7ff634101b20 2023-08-25T08:59:11.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.465+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63419ef50 con 0x7ff634101b20 2023-08-25T08:59:11.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.466+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63404e700 con 0x7ff634101b20 2023-08-25T08:59:11.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.468+0000 7ff62affd700 1 -- 172.21.15.99:0/3292284520 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 22) v1 ==== 134483+0+0 (secure 0 0 0) 0x7ff624004c80 con 0x7ff634101b20 2023-08-25T08:59:11.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.470+0000 7ff62affd700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ff62009fd50 0x7ff6200a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:11.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.470+0000 7ff62affd700 1 -- 172.21.15.99:0/3292284520 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff6240cbea0 con 0x7ff634101b20 2023-08-25T08:59:11.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.470+0000 7ff639b9b700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ff62009fd50 0x7ff6200a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:11.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.471+0000 7ff639b9b700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ff62009fd50 0x7ff6200a2210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff6300105c0 tx=0x7ff63000b4a0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:11.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.472+0000 7ff62affd700 1 -- 172.21.15.99:0/3292284520 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff624095e10 con 0x7ff634101b20 2023-08-25T08:59:11.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:11 smithi099 ceph-mon[97641]: [25/Aug/2023:08:59:10] ENGINE Bus STARTING 2023-08-25T08:59:11.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:11 smithi099 ceph-mon[97641]: mgrmap e22: x(active, since 1.01787s), standbys: y, z 2023-08-25T08:59:11.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:11 smithi157 ceph-mon[130879]: [25/Aug/2023:08:59:10] ENGINE Bus STARTING 2023-08-25T08:59:11.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:11 smithi157 ceph-mon[130879]: mgrmap e22: x(active, since 1.01787s), standbys: y, z 2023-08-25T08:59:11.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.727+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 --> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7ff634060d60 con 0x7ff62009fd50 2023-08-25T08:59:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:11 smithi181 ceph-mon[131296]: [25/Aug/2023:08:59:10] ENGINE Bus STARTING 2023-08-25T08:59:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:11 smithi181 ceph-mon[131296]: mgrmap e22: x(active, since 1.01787s), standbys: y, z 2023-08-25T08:59:11.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.782+0000 7ff62affd700 1 -- 172.21.15.99:0/3292284520 <== mgr.25202 v2:172.21.15.99:6800/2884087217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7ff634060d60 con 0x7ff62009fd50 2023-08-25T08:59:11.783 INFO:teuthology.orchestra.run.smithi099.stdout:Retention added to path / 2023-08-25T08:59:11.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ff62009fd50 msgr2=0x7ff6200a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:11.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ff62009fd50 0x7ff6200a2210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff6300105c0 tx=0x7ff63000b4a0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff634101b20 msgr2=0x7ff634196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:11.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff634101b20 0x7ff634196ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6240098e0 tx=0x7ff62400f690 comp rx=0 tx=0).stop 2023-08-25T08:59:11.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 shutdown_connections 2023-08-25T08:59:11.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ff62009fd50 0x7ff6200a2210 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6341024c0 0x7ff634197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff634100930 0x7ff634196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 --2- 172.21.15.99:0/3292284520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff634101b20 0x7ff634196ca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:11.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 >> 172.21.15.99:0/3292284520 conn(0x7ff6340fbe80 msgr2=0x7ff6340fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:11.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 shutdown_connections 2023-08-25T08:59:11.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:11.785+0000 7ff63bdff700 1 -- 172.21.15.99:0/3292284520 wait complete. 2023-08-25T08:59:11.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-08-25T08:59:12.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 -- 172.21.15.99:0/1799580482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 msgr2=0x7f699410aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1799580482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 0x7f699410aca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b0a0 tx=0x7f698c00b3b0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 -- 172.21.15.99:0/1799580482 shutdown_connections 2023-08-25T08:59:12.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1799580482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 0x7f699410aca0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1799580482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6994074960 0x7f6994072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1799580482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6994073380 0x7f6994073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 -- 172.21.15.99:0/1799580482 >> 172.21.15.99:0/1799580482 conn(0x7f69940fbea0 msgr2=0x7f69940fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:12.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 -- 172.21.15.99:0/1799580482 shutdown_connections 2023-08-25T08:59:12.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.010+0000 7f699b1ff700 1 -- 172.21.15.99:0/1799580482 wait complete. 2023-08-25T08:59:12.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.011+0000 7f699b1ff700 1 Processor -- start 2023-08-25T08:59:12.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.011+0000 7f699b1ff700 1 -- start start 2023-08-25T08:59:12.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.011+0000 7f699b1ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6994073380 0x7f6994198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:12.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699b1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6994074960 0x7f6994198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:12.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699b1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 0x7f6994199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:12.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699b1ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699419a460 con 0x7f6994073380 2023-08-25T08:59:12.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699b1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699419a5d0 con 0x7f6994074960 2023-08-25T08:59:12.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699b1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699419a790 con 0x7f6994108810 2023-08-25T08:59:12.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 0x7f6994199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:12.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f6998f9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6994073380 0x7f6994198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:12.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 0x7f6994199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:53572/0 (socket says 172.21.15.99:53572) 2023-08-25T08:59:12.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 -- 172.21.15.99:0/1783410971 learned_addr learned my addr 172.21.15.99:0/1783410971 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:12.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f6993fff700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6994074960 0x7f6994198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:12.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 -- 172.21.15.99:0/1783410971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6994074960 msgr2=0x7f6994198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6994074960 0x7f6994198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 -- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6994073380 msgr2=0x7f6994198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6994073380 0x7f6994198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 -- 172.21.15.99:0/1783410971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c009d00 con 0x7f6994108810 2023-08-25T08:59:12.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.012+0000 7f699979c700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 0x7f6994199e00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f698c009fd0 tx=0x7f698c00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:12.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.013+0000 7f6991ffb700 1 -- 172.21.15.99:0/1783410971 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c00e040 con 0x7f6994108810 2023-08-25T08:59:12.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.013+0000 7f6991ffb700 1 -- 172.21.15.99:0/1783410971 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f698c0094a0 con 0x7f6994108810 2023-08-25T08:59:12.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.013+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699419aa10 con 0x7f6994108810 2023-08-25T08:59:12.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.013+0000 7f6991ffb700 1 -- 172.21.15.99:0/1783410971 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c01b6a0 con 0x7f6994108810 2023-08-25T08:59:12.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.013+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69941a1230 con 0x7f6994108810 2023-08-25T08:59:12.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.014+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6994106ec0 con 0x7f6994108810 2023-08-25T08:59:12.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.016+0000 7f6991ffb700 1 -- 172.21.15.99:0/1783410971 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 22) v1 ==== 134483+0+0 (secure 0 0 0) 0x7f698c01d470 con 0x7f6994108810 2023-08-25T08:59:12.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.017+0000 7f6991ffb700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f698409fd50 0x7f69840a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:12.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.017+0000 7f6991ffb700 1 -- 172.21.15.99:0/1783410971 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f698c0cc130 con 0x7f6994108810 2023-08-25T08:59:12.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.017+0000 7f6998f9b700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f698409fd50 0x7f69840a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:12.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.018+0000 7f6998f9b700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f698409fd50 0x7f69840a2210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6988009c60 tx=0x7f6988009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:12.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.020+0000 7f6991ffb700 1 -- 172.21.15.99:0/1783410971 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f698c0960a0 con 0x7f6994108810 2023-08-25T08:59:12.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.271+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 --> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f6994060d60 con 0x7f698409fd50 2023-08-25T08:59:12.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.279+0000 7f6991ffb700 1 -- 172.21.15.99:0/1783410971 <== mgr.25202 v2:172.21.15.99:6800/2884087217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f6994060d60 con 0x7f698409fd50 2023-08-25T08:59:12.281 INFO:teuthology.orchestra.run.smithi099.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-08-25T00:00:00", "created": "2023-08-25T08:59:11", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-08-25T08:59:12.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f698409fd50 msgr2=0x7f69840a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f698409fd50 0x7f69840a2210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6988009c60 tx=0x7f6988009260 comp rx=0 tx=0).stop 2023-08-25T08:59:12.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 msgr2=0x7f6994199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 0x7f6994199e00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f698c009fd0 tx=0x7f698c00bce0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 shutdown_connections 2023-08-25T08:59:12.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f698409fd50 0x7f69840a2210 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6994108810 0x7f6994199e00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6994073380 0x7f6994198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 --2- 172.21.15.99:0/1783410971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6994074960 0x7f6994198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 >> 172.21.15.99:0/1783410971 conn(0x7f69940fbea0 msgr2=0x7f69940fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:12.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.282+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 shutdown_connections 2023-08-25T08:59:12.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.283+0000 7f699b1ff700 1 -- 172.21.15.99:0/1783410971 wait complete. 2023-08-25T08:59:12.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-08-25T08:59:12.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.506+0000 7fa3887ff700 1 -- 172.21.15.99:0/3670630767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 msgr2=0x7fa380100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.506+0000 7fa3887ff700 1 --2- 172.21.15.99:0/3670630767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 0x7fa380100da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0099c0 tx=0x7fa37c009cd0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.507+0000 7fa3887ff700 1 -- 172.21.15.99:0/3670630767 shutdown_connections 2023-08-25T08:59:12.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.507+0000 7fa3887ff700 1 --2- 172.21.15.99:0/3670630767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa380102510 0x7fa380108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.507+0000 7fa3887ff700 1 --2- 172.21.15.99:0/3670630767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 0x7fa380100da0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.507+0000 7fa3887ff700 1 --2- 172.21.15.99:0/3670630767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380101b70 0x7fa380101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.507+0000 7fa3887ff700 1 -- 172.21.15.99:0/3670630767 >> 172.21.15.99:0/3670630767 conn(0x7fa3800fbed0 msgr2=0x7fa3800fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:12.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.507+0000 7fa3887ff700 1 -- 172.21.15.99:0/3670630767 shutdown_connections 2023-08-25T08:59:12.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.507+0000 7fa3887ff700 1 -- 172.21.15.99:0/3670630767 wait complete. 2023-08-25T08:59:12.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa3887ff700 1 Processor -- start 2023-08-25T08:59:12.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa3887ff700 1 -- start start 2023-08-25T08:59:12.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa3887ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 0x7fa380072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:12.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa3887ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380101b70 0x7fa380073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:12.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa3887ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa380102510 0x7fa380073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:12.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa3887ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801a0710 con 0x7fa380102510 2023-08-25T08:59:12.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa3887ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801a0850 con 0x7fa380100980 2023-08-25T08:59:12.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa3887ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801a0990 con 0x7fa380101b70 2023-08-25T08:59:12.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.508+0000 7fa38659b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 0x7fa380072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:12.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa38659b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 0x7fa380072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:56986/0 (socket says 172.21.15.99:56986) 2023-08-25T08:59:12.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa38659b700 1 -- 172.21.15.99:0/1818545629 learned_addr learned my addr 172.21.15.99:0/1818545629 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:12.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa385d9a700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380101b70 0x7fa380073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:12.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa386d9c700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa380102510 0x7fa380073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:12.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa38659b700 1 -- 172.21.15.99:0/1818545629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380101b70 msgr2=0x7fa380073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa38659b700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380101b70 0x7fa380073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa38659b700 1 -- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa380102510 msgr2=0x7fa380073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa38659b700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa380102510 0x7fa380073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa38659b700 1 -- 172.21.15.99:0/1818545629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0095f0 con 0x7fa380100980 2023-08-25T08:59:12.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa385d9a700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380101b70 0x7fa380073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:12.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa386d9c700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa380102510 0x7fa380073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:12.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa38659b700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 0x7fa380072d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0058e0 tx=0x7fa37c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:12.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.509+0000 7fa3777fe700 1 -- 172.21.15.99:0/1818545629 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c01d070 con 0x7fa380100980 2023-08-25T08:59:12.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.510+0000 7fa3777fe700 1 -- 172.21.15.99:0/1818545629 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37c005050 con 0x7fa380100980 2023-08-25T08:59:12.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.510+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3801a0bc0 con 0x7fa380100980 2023-08-25T08:59:12.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.510+0000 7fa3777fe700 1 -- 172.21.15.99:0/1818545629 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c004b10 con 0x7fa380100980 2023-08-25T08:59:12.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.510+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3801a1040 con 0x7fa380100980 2023-08-25T08:59:12.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.511+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa380107f60 con 0x7fa380100980 2023-08-25T08:59:12.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.513+0000 7fa3777fe700 1 -- 172.21.15.99:0/1818545629 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 23) v1 ==== 134791+0+0 (secure 0 0 0) 0x7fa37c0051c0 con 0x7fa380100980 2023-08-25T08:59:12.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.514+0000 7fa3777fe700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa36c0a0140 0x7fa36c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:12.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.514+0000 7fa3777fe700 1 -- 172.21.15.99:0/1818545629 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa37c0cc580 con 0x7fa380100980 2023-08-25T08:59:12.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.514+0000 7fa385d9a700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa36c0a0140 0x7fa36c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:12.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.515+0000 7fa385d9a700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa36c0a0140 0x7fa36c0a2600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa380074010 tx=0x7fa37000b410 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:12.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.516+0000 7fa3777fe700 1 -- 172.21.15.99:0/1818545629 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa37c096040 con 0x7fa380100980 2023-08-25T08:59:12.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:12 smithi099 ceph-mon[97641]: [25/Aug/2023:08:59:10] ENGINE Serving on http://172.21.15.99:8765 2023-08-25T08:59:12.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:12 smithi099 ceph-mon[97641]: [25/Aug/2023:08:59:10] ENGINE Serving on https://172.21.15.99:7150 2023-08-25T08:59:12.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:12 smithi099 ceph-mon[97641]: [25/Aug/2023:08:59:10] ENGINE Bus STARTED 2023-08-25T08:59:12.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:12 smithi099 ceph-mon[97641]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:12.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:12 smithi099 ceph-mon[97641]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:12.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:12 smithi099 ceph-mon[97641]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-08-25T08:59:12.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:12 smithi099 ceph-mon[97641]: Cluster is now healthy 2023-08-25T08:59:12.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:12 smithi157 ceph-mon[130879]: [25/Aug/2023:08:59:10] ENGINE Serving on http://172.21.15.99:8765 2023-08-25T08:59:12.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:12 smithi157 ceph-mon[130879]: [25/Aug/2023:08:59:10] ENGINE Serving on https://172.21.15.99:7150 2023-08-25T08:59:12.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:12 smithi157 ceph-mon[130879]: [25/Aug/2023:08:59:10] ENGINE Bus STARTED 2023-08-25T08:59:12.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:12 smithi157 ceph-mon[130879]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:12.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:12 smithi157 ceph-mon[130879]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:12.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:12 smithi157 ceph-mon[130879]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-08-25T08:59:12.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:12 smithi157 ceph-mon[130879]: Cluster is now healthy 2023-08-25T08:59:12.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:12 smithi181 ceph-mon[131296]: [25/Aug/2023:08:59:10] ENGINE Serving on http://172.21.15.99:8765 2023-08-25T08:59:12.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:12 smithi181 ceph-mon[131296]: [25/Aug/2023:08:59:10] ENGINE Serving on https://172.21.15.99:7150 2023-08-25T08:59:12.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:12 smithi181 ceph-mon[131296]: [25/Aug/2023:08:59:10] ENGINE Bus STARTED 2023-08-25T08:59:12.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:12 smithi181 ceph-mon[131296]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:12.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:12 smithi181 ceph-mon[131296]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:12.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:12 smithi181 ceph-mon[131296]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-08-25T08:59:12.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:12 smithi181 ceph-mon[131296]: Cluster is now healthy 2023-08-25T08:59:12.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.755+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 --> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa38002cbe0 con 0x7fa36c0a0140 2023-08-25T08:59:12.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.762+0000 7fa3777fe700 1 -- 172.21.15.99:0/1818545629 <== mgr.25202 v2:172.21.15.99:6800/2884087217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fa38002cbe0 con 0x7fa36c0a0140 2023-08-25T08:59:12.763 INFO:teuthology.orchestra.run.smithi099.stdout:/ 1M 6M3h 2023-08-25T08:59:12.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa36c0a0140 msgr2=0x7fa36c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa36c0a0140 0x7fa36c0a2600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa380074010 tx=0x7fa37000b410 comp rx=0 tx=0).stop 2023-08-25T08:59:12.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 msgr2=0x7fa380072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:12.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 0x7fa380072d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0058e0 tx=0x7fa37c00f690 comp rx=0 tx=0).stop 2023-08-25T08:59:12.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 shutdown_connections 2023-08-25T08:59:12.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa36c0a0140 0x7fa36c0a2600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa380102510 0x7fa380073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380100980 0x7fa380072d10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 --2- 172.21.15.99:0/1818545629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380101b70 0x7fa380073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 >> 172.21.15.99:0/1818545629 conn(0x7fa3800fbed0 msgr2=0x7fa3801050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 shutdown_connections 2023-08-25T08:59:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T08:59:12.765+0000 7fa3887ff700 1 -- 172.21.15.99:0/1818545629 wait complete. 2023-08-25T08:59:12.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-08-25T08:59:12.869 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-08-25T08:59:12.879 INFO:teuthology.run_tasks:Running task background_exec... 2023-08-25T08:59:12.893 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T08:59:12.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-08-25T08:59:12.899 INFO:teuthology.run_tasks:Running task workunit... 2023-08-25T08:59:12.913 INFO:tasks.workunit:Pulling workunits from ref 4b369cf18ed1391a426ab4ae86da834e9c074f81 2023-08-25T08:59:12.914 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-08-25T08:59:12.914 INFO:tasks.workunit:timeout=3h 2023-08-25T08:59:12.914 INFO:tasks.workunit:cleanup=True 2023-08-25T08:59:12.915 DEBUG:teuthology.orchestra.run.smithi099:> stat -- /home/ubuntu/cephtest/mnt.0 2023-08-25T08:59:12.963 INFO:teuthology.orchestra.run.smithi099.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-08-25T08:59:12.964 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-08-25T08:59:12.964 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-08-25T08:59:12.964 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T08:59:12.964 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:cephfs_t:s0 2023-08-25T08:59:12.964 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-08-25 08:57:26.493981946 +0000 2023-08-25T08:59:12.965 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-08-25 08:57:26.493981946 +0000 2023-08-25T08:59:12.965 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-08-25 08:59:00.347795662 +0000 2023-08-25T08:59:12.965 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-08-25 08:57:26.493981946 +0000 2023-08-25T08:59:12.965 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-08-25T08:59:12.965 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-08-25T08:59:13.049 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/rishabh-d-dave/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4b369cf18ed1391a426ab4ae86da834e9c074f81 2023-08-25T08:59:13.096 INFO:tasks.workunit.client.0.smithi099.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-08-25T08:59:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:13 smithi099 ceph-mon[97641]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:13.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:13 smithi099 ceph-mon[97641]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:13.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:13 smithi099 ceph-mon[97641]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:13.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:13 smithi099 ceph-mon[97641]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-08-25T08:59:13.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:13 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:13 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:13 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:13 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:13 smithi157 ceph-mon[130879]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:13 smithi157 ceph-mon[130879]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:13.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:13 smithi157 ceph-mon[130879]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:13.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:13 smithi157 ceph-mon[130879]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-08-25T08:59:13.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:13 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:13 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:13 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:13 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:13 smithi181 ceph-mon[131296]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:13 smithi181 ceph-mon[131296]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:13 smithi181 ceph-mon[131296]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:13 smithi181 ceph-mon[131296]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-08-25T08:59:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:13 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:13 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:13 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:13.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:13 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:14.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:14 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:14.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.632 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.633 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:14 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T08:59:14.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:14.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:14.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:14 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:15 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-08-25T08:59:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:15 smithi099 ceph-mon[97641]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-08-25T08:59:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:15 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi157 to 196.2M 2023-08-25T08:59:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:15 smithi099 ceph-mon[97641]: Unable to set osd_memory_target on smithi157 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T08:59:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:15 smithi099 ceph-mon[97641]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:15 smithi099 ceph-mon[97641]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-08-25T08:59:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:15 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-08-25T08:59:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:15 smithi157 ceph-mon[130879]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-08-25T08:59:15.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:15 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi157 to 196.2M 2023-08-25T08:59:15.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:15 smithi157 ceph-mon[130879]: Unable to set osd_memory_target on smithi157 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T08:59:15.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:15 smithi157 ceph-mon[130879]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:15.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:15 smithi157 ceph-mon[130879]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-08-25T08:59:15.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:15 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-08-25T08:59:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:15 smithi181 ceph-mon[131296]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-08-25T08:59:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:15 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi157 to 196.2M 2023-08-25T08:59:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:15 smithi181 ceph-mon[131296]: Unable to set osd_memory_target on smithi157 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T08:59:15.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:15 smithi181 ceph-mon[131296]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T08:59:15.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:15 smithi181 ceph-mon[131296]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-08-25T08:59:16.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:16.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:16.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi099 to 220.9M 2023-08-25T08:59:16.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: Unable to set osd_memory_target on smithi099 to 231651840: error parsing value: Value '231651840' is below minimum 939524096 2023-08-25T08:59:16.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:59:16.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:59:16.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: Updating smithi099:/etc/ceph/ceph.conf 2023-08-25T08:59:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: Updating smithi157:/etc/ceph/ceph.conf 2023-08-25T08:59:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: Updating smithi181:/etc/ceph/ceph.conf 2023-08-25T08:59:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:16.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:16 smithi181 ceph-mon[131296]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:16.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:16.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:16.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi099 to 220.9M 2023-08-25T08:59:16.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: Unable to set osd_memory_target on smithi099 to 231651840: error parsing value: Value '231651840' is below minimum 939524096 2023-08-25T08:59:16.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:59:16.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:59:16.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: Updating smithi099:/etc/ceph/ceph.conf 2023-08-25T08:59:16.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: Updating smithi157:/etc/ceph/ceph.conf 2023-08-25T08:59:16.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: Updating smithi181:/etc/ceph/ceph.conf 2023-08-25T08:59:16.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:16.817 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:16 smithi099 ceph-mon[97641]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:16.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:16.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:16.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T08:59:16.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi099 to 220.9M 2023-08-25T08:59:16.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: Unable to set osd_memory_target on smithi099 to 231651840: error parsing value: Value '231651840' is below minimum 939524096 2023-08-25T08:59:16.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:59:16.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:59:16.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: Updating smithi099:/etc/ceph/ceph.conf 2023-08-25T08:59:16.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: Updating smithi157:/etc/ceph/ceph.conf 2023-08-25T08:59:16.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: Updating smithi181:/etc/ceph/ceph.conf 2023-08-25T08:59:16.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:16.884 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:16 smithi157 ceph-mon[130879]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:17.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:17.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:17.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 183 KiB/s wr, 65 op/s 2023-08-25T08:59:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:17 smithi099 ceph-mon[97641]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:17.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:17.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:17.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:17.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 183 KiB/s wr, 65 op/s 2023-08-25T08:59:17.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:17.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:17.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:17 smithi157 ceph-mon[130879]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T08:59:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 183 KiB/s wr, 65 op/s 2023-08-25T08:59:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: Updating smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:17.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: Updating smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:17.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:17.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:17 smithi181 ceph-mon[131296]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-08-25T08:59:18.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:18 smithi157 ceph-mon[130879]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:18.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:18 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:18.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:18 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:18.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:18 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:18.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:18 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:18.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:18 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:59:18.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:18 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:59:18.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:18 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:18.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:18 smithi181 ceph-mon[131296]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:18.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:18 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:18.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:18 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:18.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:18 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:18.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:18 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:18.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:18 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:59:18.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:18 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:59:18.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:18 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:19.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:18 smithi099 ceph-mon[97641]: Updating smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T08:59:19.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:18 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:19.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:18 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:19.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:18 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:19.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:18 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T08:59:19.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:18 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T08:59:19.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:18 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T08:59:19.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:18 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T08:59:19.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:19 smithi157 ceph-mon[130879]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-08-25T08:59:19.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:19 smithi181 ceph-mon[131296]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-08-25T08:59:20.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:19 smithi099 ceph-mon[97641]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-08-25T08:59:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:21 smithi157 ceph-mon[130879]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-08-25T08:59:21.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:21 smithi181 ceph-mon[131296]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-08-25T08:59:22.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:21 smithi099 ceph-mon[97641]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-08-25T08:59:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:23 smithi157 ceph-mon[130879]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 84 KiB/s wr, 46 op/s 2023-08-25T08:59:23.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:23 smithi181 ceph-mon[131296]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 84 KiB/s wr, 46 op/s 2023-08-25T08:59:24.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:23 smithi099 ceph-mon[97641]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 84 KiB/s wr, 46 op/s 2023-08-25T08:59:25.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:25 smithi099 ceph-mon[97641]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 42 op/s 2023-08-25T08:59:25.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:25 smithi157 ceph-mon[130879]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 42 op/s 2023-08-25T08:59:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:25 smithi181 ceph-mon[131296]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 42 op/s 2023-08-25T08:59:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 -- 172.21.15.99:0/2276641190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80108ad0 msgr2=0x7fdb80108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 --2- 172.21.15.99:0/2276641190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80108ad0 0x7fdb80108eb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb700098f0 tx=0x7fdb70009c00 comp rx=0 tx=0).stop 2023-08-25T08:59:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 -- 172.21.15.99:0/2276641190 shutdown_connections 2023-08-25T08:59:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 --2- 172.21.15.99:0/2276641190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb80103480 0x7fdb8010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 --2- 172.21.15.99:0/2276641190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80108ad0 0x7fdb80108eb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 --2- 172.21.15.99:0/2276641190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb80102ac0 0x7fdb80102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 -- 172.21.15.99:0/2276641190 >> 172.21.15.99:0/2276641190 conn(0x7fdb800fe570 msgr2=0x7fdb80100990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 -- 172.21.15.99:0/2276641190 shutdown_connections 2023-08-25T08:59:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.160+0000 7fdb84dff700 1 -- 172.21.15.99:0/2276641190 wait complete. 2023-08-25T08:59:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.162+0000 7fdb84dff700 1 Processor -- start 2023-08-25T08:59:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.162+0000 7fdb84dff700 1 -- start start 2023-08-25T08:59:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.162+0000 7fdb84dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb80102ac0 0x7fdb8019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.162+0000 7fdb84dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80103480 0x7fdb8019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.162+0000 7fdb84dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb80108ad0 0x7fdb80198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.162+0000 7fdb84dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb801990b0 con 0x7fdb80108ad0 2023-08-25T08:59:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.162+0000 7fdb84dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb80199220 con 0x7fdb80102ac0 2023-08-25T08:59:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.162+0000 7fdb84dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb801993e0 con 0x7fdb80103480 2023-08-25T08:59:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb80102ac0 0x7fdb8019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80103480 0x7fdb8019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb80102ac0 0x7fdb8019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:58454/0 (socket says 172.21.15.99:58454) 2023-08-25T08:59:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7e59c700 1 -- 172.21.15.99:0/3442838499 learned_addr learned my addr 172.21.15.99:0/3442838499 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb80108ad0 0x7fdb80198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7dd9b700 1 -- 172.21.15.99:0/3442838499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb80102ac0 msgr2=0x7fdb8019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb80102ac0 0x7fdb8019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7dd9b700 1 -- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb80108ad0 msgr2=0x7fdb80198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb80108ad0 0x7fdb80198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7dd9b700 1 -- 172.21.15.99:0/3442838499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb78009d00 con 0x7fdb80103480 2023-08-25T08:59:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7e59c700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb80102ac0 0x7fdb8019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.163+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80103480 0x7fdb8019f120 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400e9f0 tx=0x7fdb7400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.164+0000 7fdb6f7fe700 1 -- 172.21.15.99:0/3442838499 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400c9c0 con 0x7fdb80103480 2023-08-25T08:59:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.164+0000 7fdb6f7fe700 1 -- 172.21.15.99:0/3442838499 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb74010430 con 0x7fdb80103480 2023-08-25T08:59:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.164+0000 7fdb84dff700 1 -- 172.21.15.99:0/3442838499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb70009520 con 0x7fdb80103480 2023-08-25T08:59:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.164+0000 7fdb6f7fe700 1 -- 172.21.15.99:0/3442838499 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74018680 con 0x7fdb80103480 2023-08-25T08:59:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.164+0000 7fdb84dff700 1 -- 172.21.15.99:0/3442838499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb801a59f0 con 0x7fdb80103480 2023-08-25T08:59:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.167+0000 7fdb6f7fe700 1 -- 172.21.15.99:0/3442838499 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) v1 ==== 134847+0+0 (secure 0 0 0) 0x7fdb74010860 con 0x7fdb80103480 2023-08-25T08:59:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.168+0000 7fdb98683b80 1 Processor -- start 2023-08-25T08:59:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.169+0000 7fdb98683b80 1 -- start start 2023-08-25T08:59:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.169+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36af40 0x5649cb36b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.169+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb365450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.169+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb365990 0x5649cb365dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.169+0000 7fdb98683b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb2cd750 con 0x5649cb365990 2023-08-25T08:59:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.169+0000 7fdb98683b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb2be720 con 0x5649cb36b860 2023-08-25T08:59:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.169+0000 7fdb98683b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb2bc040 con 0x5649cb36af40 2023-08-25T08:59:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.169+0000 7fdb6f7fe700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fdb680a0150 0x7fdb680a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36af40 0x5649cb36b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36af40 0x5649cb36b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:37420/0 (socket says 172.21.15.99:37420) 2023-08-25T08:59:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/841476315 learned_addr learned my addr 172.21.15.99:0/841476315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7e59c700 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb365450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb365990 0x5649cb365dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/841476315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 msgr2=0x5649cb365450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb365450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/841476315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb365990 msgr2=0x5649cb365dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb365990 0x5649cb365dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/841476315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649cb1375d0 con 0x5649cb36af40 2023-08-25T08:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fdb680a0150 0x7fdb680a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb6f7fe700 1 -- 172.21.15.99:0/3442838499 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdb74014070 con 0x7fdb80103480 2023-08-25T08:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.170+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36af40 0x5649cb36b320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb780095a0 tx=0x7fdb780095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61cf5fdbe400b485 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.171+0000 7fdb6cff9700 1 -- 172.21.15.99:0/841476315 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78016880 con 0x5649cb36af40 2023-08-25T08:59:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.171+0000 7fdb6cff9700 1 -- 172.21.15.99:0/841476315 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb7800f040 con 0x5649cb36af40 2023-08-25T08:59:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.171+0000 7fdb6cff9700 1 -- 172.21.15.99:0/841476315 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78022800 con 0x5649cb36af40 2023-08-25T08:59:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 -- 172.21.15.99:0/841476315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36af40 msgr2=0x5649cb36b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fdb680a0150 0x7fdb680a2610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb78020000 tx=0x7fdb78021000 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36af40 0x5649cb36b320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb780095a0 tx=0x7fdb780095d0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 -- 172.21.15.99:0/841476315 shutdown_connections 2023-08-25T08:59:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb365990 0x5649cb365dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36af40 0x5649cb36b320 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 --2- 172.21.15.99:0/841476315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb365450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 -- 172.21.15.99:0/841476315 >> 172.21.15.99:0/841476315 conn(0x5649cb2da080 msgr2=0x5649cb35fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 -- 172.21.15.99:0/841476315 shutdown_connections 2023-08-25T08:59:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.172+0000 7fdb98683b80 1 -- 172.21.15.99:0/841476315 wait complete. 2023-08-25T08:59:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.173+0000 7fdb98683b80 1 -- start start 2023-08-25T08:59:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.173+0000 7fdb98683b80 1 Processor -- start 2023-08-25T08:59:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 0x5649cb3715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb36e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb36edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb98683b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb2b7930 con 0x5649cb36af40 2023-08-25T08:59:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb98683b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb255b50 con 0x5649cb36b860 2023-08-25T08:59:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb98683b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb255850 con 0x5649cb365990 2023-08-25T08:59:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb98683b80 10 client.0 _ll_register_callbacks cb 0x5649cb19fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:59:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7ed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb36edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7ed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb36edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:58468/0 (socket says 172.21.15.99:58468) 2023-08-25T08:59:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/627484172 learned_addr learned my addr 172.21.15.99:0/627484172 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb36e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/627484172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 msgr2=0x5649cb3715b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T08:59:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 0x5649cb3715b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/627484172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 msgr2=0x5649cb36e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb36e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.174+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/627484172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb78009250 con 0x5649cb36b860 2023-08-25T08:59:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.175+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb36edd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009830 tx=0x7fdb780225a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de448b485319d0f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.175+0000 7fdb65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T08:59:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.175+0000 7fdb65ffb700 1 -- 172.21.15.99:0/627484172 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800e020 con 0x5649cb36b860 2023-08-25T08:59:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.175+0000 7fdb98683b80 1 -- 172.21.15.99:0/627484172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649cb335160 con 0x5649cb36b860 2023-08-25T08:59:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.175+0000 7fdb65ffb700 1 -- 172.21.15.99:0/627484172 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdb78022640 con 0x5649cb36b860 2023-08-25T08:59:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.175+0000 7fdb65ffb700 1 -- 172.21.15.99:0/627484172 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb78029070 con 0x5649cb36b860 2023-08-25T08:59:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.175+0000 7fdb98683b80 10 client.25292 fetch_fsmap learned FSMap version 23 2023-08-25T08:59:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb98683b80 1 -- 172.21.15.99:0/627484172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5649cb0e3d80 con 0x5649cb36b860 2023-08-25T08:59:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb65ffb700 1 -- 172.21.15.99:0/627484172 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb7803a080 con 0x5649cb36b860 2023-08-25T08:59:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb65ffb700 1 -- 172.21.15.99:0/627484172 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb78003a00 con 0x5649cb36b860 2023-08-25T08:59:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb98683b80 10 client.25292 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:59:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb98683b80 10 client.25292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T08:59:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb98683b80 20 client.25292 populate_metadata read hostname 'smithi099' 2023-08-25T08:59:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb98683b80 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5649cb37f0d0 0x5649cb3814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb98683b80 4 client.25292 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:59:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.176+0000 7fdb98683b80 1 -- 172.21.15.99:0/627484172 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649cb25db40 con 0x5649cb37f0d0 2023-08-25T08:59:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.177+0000 7fdb7e59c700 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5649cb37f0d0 0x5649cb3814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.177+0000 7fdb7e59c700 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5649cb37f0d0 0x5649cb3814f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376e9c732008ce66 server_cookie=47a8c4ed938fc541 in_seq=0 out_seq=0 2023-08-25T08:59:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.177+0000 7fdb65ffb700 10 client.25292 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb65ffb700 1 -- 172.21.15.99:0/627484172 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5649cb25db40 con 0x5649cb37f0d0 2023-08-25T08:59:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb65ffb700 10 client.25292 handle_command_reply: tid=0 2023-08-25T08:59:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 client.25292 shutdown 2023-08-25T08:59:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 -- 172.21.15.99:0/627484172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 msgr2=0x5649cb36edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb36edd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009830 tx=0x7fdb780225a0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 -- 172.21.15.99:0/627484172 shutdown_connections 2023-08-25T08:59:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5649cb37f0d0 0x5649cb3814f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36b860 0x5649cb36edd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 0x5649cb3715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 --2- 172.21.15.99:0/627484172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb36e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb65ffb700 0 client.25292 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.178+0000 7fdb98683b80 1 -- 172.21.15.99:0/627484172 >> 172.21.15.99:0/627484172 conn(0x5649cb2da080 msgr2=0x5649cb35a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.179+0000 7fdb98683b80 1 -- 172.21.15.99:0/627484172 shutdown_connections 2023-08-25T08:59:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.179+0000 7fdb98683b80 1 -- 172.21.15.99:0/627484172 wait complete. 2023-08-25T08:59:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.179+0000 7fdb98683b80 20 client.25292 trim_cache size 0 max 16384 2023-08-25T08:59:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb98683b80 1 Processor -- start 2023-08-25T08:59:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb98683b80 1 -- start start 2023-08-25T08:59:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 0x5649cb365450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36af40 0x5649cb376240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36b860 0x5649cb36e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb98683b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb1398c0 con 0x5649cb36b860 2023-08-25T08:59:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb98683b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb36fe40 con 0x5649cb36af40 2023-08-25T08:59:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb98683b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb36f9c0 con 0x5649cb365990 2023-08-25T08:59:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb7ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36b860 0x5649cb36e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb7ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36b860 0x5649cb36e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39016/0 (socket says 172.21.15.99:39016) 2023-08-25T08:59:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/3007100796 learned_addr learned my addr 172.21.15.99:0/3007100796 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.195+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36af40 0x5649cb376240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb7e59c700 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 0x5649cb365450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/3007100796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 msgr2=0x5649cb365450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 0x5649cb365450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/3007100796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36af40 msgr2=0x5649cb376240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36af40 0x5649cb376240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb7ed9d700 1 -- 172.21.15.99:0/3007100796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649cb361d00 con 0x5649cb36b860 2023-08-25T08:59:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36af40 0x5649cb376240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36b860 0x5649cb36e040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009800 tx=0x7fdb780061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a8b9207b82e2c0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb6d7fa700 1 -- 172.21.15.99:0/3007100796 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78029030 con 0x5649cb36b860 2023-08-25T08:59:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb6d7fa700 1 -- 172.21.15.99:0/3007100796 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649cb335160 con 0x5649cb36b860 2023-08-25T08:59:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.196+0000 7fdb6d7fa700 1 -- 172.21.15.99:0/3007100796 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb780188a0 con 0x5649cb36b860 2023-08-25T08:59:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 -- 172.21.15.99:0/3007100796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36b860 msgr2=0x5649cb36e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36b860 0x5649cb36e040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009800 tx=0x7fdb780061f0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 -- 172.21.15.99:0/3007100796 shutdown_connections 2023-08-25T08:59:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36b860 0x5649cb36e040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb365990 0x5649cb365450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 --2- 172.21.15.99:0/3007100796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb36af40 0x5649cb376240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 -- 172.21.15.99:0/3007100796 >> 172.21.15.99:0/3007100796 conn(0x5649cb2da080 msgr2=0x5649cb368af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 -- 172.21.15.99:0/3007100796 shutdown_connections 2023-08-25T08:59:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.197+0000 7fdb98683b80 1 -- 172.21.15.99:0/3007100796 wait complete. 2023-08-25T08:59:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.198+0000 7fdb98683b80 1 -- start start 2023-08-25T08:59:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.198+0000 7fdb98683b80 1 Processor -- start 2023-08-25T08:59:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.198+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb365990 0x5649cb372410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.198+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb372950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.198+0000 7fdb98683b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36b860 0x5649cb3cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.198+0000 7fdb98683b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb3cc400 con 0x5649cb36af40 2023-08-25T08:59:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.198+0000 7fdb98683b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb3cc570 con 0x5649cb365990 2023-08-25T08:59:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.198+0000 7fdb98683b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cb3cc730 con 0x5649cb36b860 2023-08-25T08:59:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb98683b80 10 client.0 _ll_register_callbacks cb 0x7fdb740c8e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:59:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb372950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb365990 0x5649cb372410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb365990 0x5649cb372410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:58476/0 (socket says 172.21.15.99:58476) 2023-08-25T08:59:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7e59c700 1 -- 172.21.15.99:0/1170987680 learned_addr learned my addr 172.21.15.99:0/1170987680 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb372950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39030/0 (socket says 172.21.15.99:39030) 2023-08-25T08:59:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7ed9d700 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36b860 0x5649cb3cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7e59c700 1 -- 172.21.15.99:0/1170987680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36b860 msgr2=0x5649cb3cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7e59c700 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36b860 0x5649cb3cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7e59c700 1 -- 172.21.15.99:0/1170987680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 msgr2=0x5649cb372950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7e59c700 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb372950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.199+0000 7fdb7e59c700 1 -- 172.21.15.99:0/1170987680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb801a59f0 con 0x5649cb365990 2023-08-25T08:59:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb7e59c700 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb365990 0x5649cb372410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009810 tx=0x7fdb70025500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11cbe5dbfb905980 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T08:59:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb65ffb700 1 -- 172.21.15.99:0/1170987680 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7001c040 con 0x5649cb365990 2023-08-25T08:59:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb98683b80 1 -- 172.21.15.99:0/1170987680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649cb358600 con 0x5649cb365990 2023-08-25T08:59:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb65ffb700 1 -- 172.21.15.99:0/1170987680 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdb70025a00 con 0x5649cb365990 2023-08-25T08:59:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb65ffb700 1 -- 172.21.15.99:0/1170987680 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb70025bc0 con 0x5649cb365990 2023-08-25T08:59:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb98683b80 10 client.25298 fetch_fsmap learned FSMap version 23 2023-08-25T08:59:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb98683b80 1 -- 172.21.15.99:0/1170987680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5649cb0e3d80 con 0x5649cb365990 2023-08-25T08:59:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.200+0000 7fdb65ffb700 1 -- 172.21.15.99:0/1170987680 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb70039020 con 0x5649cb365990 2023-08-25T08:59:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.201+0000 7fdb65ffb700 1 -- 172.21.15.99:0/1170987680 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb70015020 con 0x5649cb365990 2023-08-25T08:59:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.201+0000 7fdb98683b80 10 client.25298 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:59:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.201+0000 7fdb98683b80 10 client.25298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T08:59:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.201+0000 7fdb98683b80 20 client.25298 populate_metadata read hostname 'smithi099' 2023-08-25T08:59:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.201+0000 7fdb98683b80 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5649cb3d3fc0 0x5649cb3d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.201+0000 7fdb98683b80 4 client.25298 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T08:59:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.201+0000 7fdb98683b80 1 -- 172.21.15.99:0/1170987680 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649cb2b0910 con 0x5649cb3d3fc0 2023-08-25T08:59:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.201+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5649cb3d3fc0 0x5649cb3d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.202+0000 7fdb7dd9b700 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5649cb3d3fc0 0x5649cb3d63e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f92d5d6a3f4f2f server_cookie=6fb9c0c1d956b8ed in_seq=0 out_seq=0 2023-08-25T08:59:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.202+0000 7fdb65ffb700 10 client.25298 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb65ffb700 1 -- 172.21.15.99:0/1170987680 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649cb2b0910 con 0x5649cb3d3fc0 2023-08-25T08:59:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb65ffb700 10 client.25298 handle_command_reply: tid=0 2023-08-25T08:59:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 client.25298 shutdown 2023-08-25T08:59:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 -- 172.21.15.99:0/1170987680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb365990 msgr2=0x5649cb372410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb365990 0x5649cb372410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009810 tx=0x7fdb70025500 comp rx=0 tx=0).stop 2023-08-25T08:59:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 -- 172.21.15.99:0/1170987680 shutdown_connections 2023-08-25T08:59:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649cb36b860 0x5649cb3cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5649cb3d3fc0 0x5649cb3d63e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649cb365990 0x5649cb372410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 --2- 172.21.15.99:0/1170987680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649cb36af40 0x5649cb372950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb65ffb700 0 client.25298 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 -- 172.21.15.99:0/1170987680 >> 172.21.15.99:0/1170987680 conn(0x5649cb2da080 msgr2=0x5649cb36bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 -- 172.21.15.99:0/1170987680 shutdown_connections 2023-08-25T08:59:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 1 -- 172.21.15.99:0/1170987680 wait complete. 2023-08-25T08:59:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.215+0000 7fdb98683b80 20 client.25298 trim_cache size 0 max 16384 2023-08-25T08:59:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T08:59:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T08:59:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T08:59:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T08:59:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 -- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fdb680a0150 msgr2=0x7fdb680a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fdb680a0150 0x7fdb680a2610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb78020000 tx=0x7fdb78021000 comp rx=0 tx=0).stop 2023-08-25T08:59:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 -- 172.21.15.99:0/3442838499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80103480 msgr2=0x7fdb8019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80103480 0x7fdb8019f120 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400e9f0 tx=0x7fdb7400ed00 comp rx=0 tx=0).stop 2023-08-25T08:59:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 -- 172.21.15.99:0/3442838499 shutdown_connections 2023-08-25T08:59:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fdb680a0150 0x7fdb680a2610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb80108ad0 0x7fdb80198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb80102ac0 0x7fdb8019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 --2- 172.21.15.99:0/3442838499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb80103480 0x7fdb8019f120 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 -- 172.21.15.99:0/3442838499 >> 172.21.15.99:0/3442838499 conn(0x7fdb800fe570 msgr2=0x7fdb800febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 -- 172.21.15.99:0/3442838499 shutdown_connections 2023-08-25T08:59:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:26.216+0000 7fdb84dff700 1 -- 172.21.15.99:0/3442838499 wait complete. 2023-08-25T08:59:27.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:27 smithi181 ceph-mon[131296]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-08-25T08:59:27.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:27 smithi099 ceph-mon[97641]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-08-25T08:59:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:27 smithi157 ceph-mon[130879]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-08-25T08:59:29.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:29 smithi181 ceph-mon[131296]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 18 op/s 2023-08-25T08:59:29.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:29 smithi099 ceph-mon[97641]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 18 op/s 2023-08-25T08:59:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:29 smithi157 ceph-mon[130879]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 18 op/s 2023-08-25T08:59:31.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:31 smithi181 ceph-mon[131296]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T08:59:31.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:31 smithi099 ceph-mon[97641]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T08:59:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:31 smithi157 ceph-mon[130879]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T08:59:33.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:33 smithi181 ceph-mon[131296]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-08-25T08:59:33.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:33 smithi099 ceph-mon[97641]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-08-25T08:59:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:33 smithi157 ceph-mon[130879]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-08-25T08:59:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:35 smithi157 ceph-mon[130879]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-08-25T08:59:35.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:35 smithi181 ceph-mon[131296]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-08-25T08:59:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:35 smithi099 ceph-mon[97641]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-08-25T08:59:37.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:37 smithi181 ceph-mon[131296]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-25T08:59:37.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:37 smithi099 ceph-mon[97641]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-25T08:59:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:37 smithi157 ceph-mon[130879]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-25T08:59:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/3050593815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e41068a0 msgr2=0x7fd3e4106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/3050593815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e41068a0 0x7fd3e4106c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40099c0 tx=0x7fd3d4009cd0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/3050593815 shutdown_connections 2023-08-25T08:59:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/3050593815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e40686d0 0x7fd3e410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/3050593815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e41068a0 0x7fd3e4106c80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/3050593815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e4067d10 0x7fd3e4068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/3050593815 >> 172.21.15.99:0/3050593815 conn(0x7fd3e4075050 msgr2=0x7fd3e4075460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/3050593815 shutdown_connections 2023-08-25T08:59:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.392+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/3050593815 wait complete. 2023-08-25T08:59:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.394+0000 7fd3e8f9d700 1 Processor -- start 2023-08-25T08:59:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.394+0000 7fd3e8f9d700 1 -- start start 2023-08-25T08:59:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.394+0000 7fd3e8f9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4067d10 0x7fd3e4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.394+0000 7fd3e8f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e40686d0 0x7fd3e4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.394+0000 7fd3e8f9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e41068a0 0x7fd3e419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e8f9d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419d930 con 0x7fd3e4067d10 2023-08-25T08:59:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e8f9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419daa0 con 0x7fd3e41068a0 2023-08-25T08:59:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e8f9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419dc60 con 0x7fd3e40686d0 2023-08-25T08:59:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4067d10 0x7fd3e4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4067d10 0x7fd3e4198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40078/0 (socket says 172.21.15.99:40078) 2023-08-25T08:59:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e37fe700 1 -- 172.21.15.99:0/2057375115 learned_addr learned my addr 172.21.15.99:0/2057375115 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e40686d0 0x7fd3e4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e41068a0 0x7fd3e419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e37fe700 1 -- 172.21.15.99:0/2057375115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e40686d0 msgr2=0x7fd3e4198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e40686d0 0x7fd3e4198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e37fe700 1 -- 172.21.15.99:0/2057375115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e41068a0 msgr2=0x7fd3e419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e41068a0 0x7fd3e419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e37fe700 1 -- 172.21.15.99:0/2057375115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d40095f0 con 0x7fd3e4067d10 2023-08-25T08:59:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.395+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e41068a0 0x7fd3e419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:59:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.396+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4067d10 0x7fd3e4198990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400b580 tx=0x7fd3d400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.396+0000 7fd3e0ff9700 1 -- 172.21.15.99:0/2057375115 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d401d070 con 0x7fd3e4067d10 2023-08-25T08:59:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.396+0000 7fd3e0ff9700 1 -- 172.21.15.99:0/2057375115 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d4005510 con 0x7fd3e4067d10 2023-08-25T08:59:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.396+0000 7fd3e0ff9700 1 -- 172.21.15.99:0/2057375115 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4004b40 con 0x7fd3e4067d10 2023-08-25T08:59:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.396+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/2057375115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e419dee0 con 0x7fd3e4067d10 2023-08-25T08:59:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.396+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/2057375115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e41a5610 con 0x7fd3e4067d10 2023-08-25T08:59:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.401+0000 7fd3e0ff9700 1 -- 172.21.15.99:0/2057375115 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 134847+0+0 (secure 0 0 0) 0x7fd3d4004cc0 con 0x7fd3e4067d10 2023-08-25T08:59:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.402+0000 7fd3fdb38b80 1 Processor -- start 2023-08-25T08:59:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3fdb38b80 1 -- start start 2023-08-25T08:59:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d0eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0ebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383d053750 con 0x56383d0f1930 2023-08-25T08:59:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383d044720 con 0x56383d0f1010 2023-08-25T08:59:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383d042040 con 0x56383d0eb990 2023-08-25T08:59:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3e0ff9700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd3cc0a01d0 0x7fd3cc0a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.403+0000 7fd3e2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0ebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0ebdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52862/0 (socket says 172.21.15.99:52862) 2023-08-25T08:59:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e2ffd700 1 -- 172.21.15.99:0/2591418378 learned_addr learned my addr 172.21.15.99:0/2591418378 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2591418378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e2ffd700 1 -- 172.21.15.99:0/2591418378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 msgr2=0x56383d0f13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/2591418378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e2ffd700 1 -- 172.21.15.99:0/2591418378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 msgr2=0x56383d0eb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/2591418378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d0eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e0ff9700 1 -- 172.21.15.99:0/2057375115 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd3d40cc680 con 0x7fd3e4067d10 2023-08-25T08:59:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e2ffd700 1 -- 172.21.15.99:0/2591418378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56383cebd5d0 con 0x56383d0eb990 2023-08-25T08:59:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd3cc0a01d0 0x7fd3cc0a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.404+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/2591418378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0ebdf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009cf0 tx=0x7fd3d8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e26f5bc070277fe6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.405+0000 7fd3c9ffb700 1 -- 172.21.15.99:0/2591418378 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8011e30 con 0x56383d0eb990 2023-08-25T08:59:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.405+0000 7fd3c9ffb700 1 -- 172.21.15.99:0/2591418378 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d8004510 con 0x56383d0eb990 2023-08-25T08:59:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.405+0000 7fd3c9ffb700 1 -- 172.21.15.99:0/2591418378 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8005280 con 0x56383d0eb990 2023-08-25T08:59:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd3cc0a01d0 0x7fd3cc0a2690 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009550 tx=0x7fd3dc00b040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2591418378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 msgr2=0x56383d0ebdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/2591418378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0ebdf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009cf0 tx=0x7fd3d8011440 comp rx=0 tx=0).stop 2023-08-25T08:59:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2591418378 shutdown_connections 2023-08-25T08:59:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/2591418378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0ebdf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/2591418378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/2591418378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d0eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2591418378 >> 172.21.15.99:0/2591418378 conn(0x56383d060080 msgr2=0x56383d0e5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2591418378 shutdown_connections 2023-08-25T08:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2591418378 wait complete. 2023-08-25T08:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.406+0000 7fd3fdb38b80 1 -- start start 2023-08-25T08:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.407+0000 7fd3fdb38b80 1 Processor -- start 2023-08-25T08:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.407+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.407+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.407+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d0f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.407+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383d03d930 con 0x56383d0f1930 2023-08-25T08:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.407+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383cfdbb50 con 0x56383d0f1010 2023-08-25T08:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.407+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383cfdb850 con 0x56383d0eb990 2023-08-25T08:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.407+0000 7fd3fdb38b80 10 client.0 _ll_register_callbacks cb 0x56383cf25a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52864/0 (socket says 172.21.15.99:52864) 2023-08-25T08:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e37fe700 1 -- 172.21.15.99:0/298847835 learned_addr learned my addr 172.21.15.99:0/298847835 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d0f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e37fe700 1 -- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 msgr2=0x56383d0f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e37fe700 1 -- 172.21.15.99:0/298847835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 msgr2=0x56383d0f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d0f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e37fe700 1 -- 172.21.15.99:0/298847835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d8009920 con 0x56383d0eb990 2023-08-25T08:59:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d0f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T08:59:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.408+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.409+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0f4c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009fd0 tx=0x7fd3d400fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23bea75d64c19fd8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.409+0000 7fd3c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T08:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.409+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/298847835 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d409a310 con 0x56383d0eb990 2023-08-25T08:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.409+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/298847835 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd3d4099c10 con 0x56383d0eb990 2023-08-25T08:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.409+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/298847835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56383d0bb160 con 0x56383d0eb990 2023-08-25T08:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.409+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/298847835 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3d409da40 con 0x56383d0eb990 2023-08-25T08:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3fdb38b80 10 client.25294 fetch_fsmap learned FSMap version 23 2023-08-25T08:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/298847835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56383ce69d80 con 0x56383d0eb990 2023-08-25T08:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/298847835 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56383d0bb160 con 0x56383d0eb990 2023-08-25T08:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/298847835 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd3d40c9fb0 con 0x56383d0eb990 2023-08-25T08:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3fdb38b80 10 client.25294 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3fdb38b80 10 client.25294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T08:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3fdb38b80 20 client.25294 populate_metadata read hostname 'smithi099' 2023-08-25T08:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56383d11e7f0 0x56383d120be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3fdb38b80 4 client.25294 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.410+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/298847835 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56383cfe3b40 con 0x56383d11e7f0 2023-08-25T08:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.411+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56383d11e7f0 0x56383d120be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.411+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56383d11e7f0 0x56383d120be0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57546e36f2cf6113 server_cookie=1842eef9c3a35cce in_seq=0 out_seq=0 2023-08-25T08:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.411+0000 7fd3c2ffd700 10 client.25294 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/298847835 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56383cfe3b40 con 0x56383d11e7f0 2023-08-25T08:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3c2ffd700 10 client.25294 handle_command_reply: tid=0 2023-08-25T08:59:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 client.25294 shutdown 2023-08-25T08:59:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/298847835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 msgr2=0x56383d0f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0f4c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009fd0 tx=0x7fd3d400fe60 comp rx=0 tx=0).stop 2023-08-25T08:59:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/298847835 shutdown_connections 2023-08-25T08:59:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d0f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d0f4c00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56383d11e7f0 0x56383d120be0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/298847835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d0f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/298847835 >> 172.21.15.99:0/298847835 conn(0x56383d060080 msgr2=0x56383d0ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.412+0000 7fd3c2ffd700 0 client.25294 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.413+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/298847835 shutdown_connections 2023-08-25T08:59:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.413+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/298847835 wait complete. 2023-08-25T08:59:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.413+0000 7fd3fdb38b80 20 client.25294 trim_cache size 0 max 16384 2023-08-25T08:59:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.441+0000 7fd3fdb38b80 1 Processor -- start 2023-08-25T08:59:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3fdb38b80 1 -- start start 2023-08-25T08:59:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0eb990 0x56383d0f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1010 0x56383d11ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0f1930 0x56383d0e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383cebf8c0 con 0x56383d0f1010 2023-08-25T08:59:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383cfe3b70 con 0x56383d0eb990 2023-08-25T08:59:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383d0e26e0 con 0x56383d0f1930 2023-08-25T08:59:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3e37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0eb990 0x56383d0f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3e37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0eb990 0x56383d0f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47056/0 (socket says 172.21.15.99:47056) 2023-08-25T08:59:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.442+0000 7fd3e37fe700 1 -- 172.21.15.99:0/2149236975 learned_addr learned my addr 172.21.15.99:0/2149236975 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0f1930 0x56383d0e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1010 0x56383d11ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e3fff700 1 -- 172.21.15.99:0/2149236975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0eb990 msgr2=0x56383d0f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0eb990 0x56383d0f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e3fff700 1 -- 172.21.15.99:0/2149236975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1010 msgr2=0x56383d11ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1010 0x56383d11ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e3fff700 1 -- 172.21.15.99:0/2149236975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56383ce52c00 con 0x56383d0f1930 2023-08-25T08:59:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0eb990 0x56383d0f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1010 0x56383d11ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T08:59:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.443+0000 7fd3e3fff700 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0f1930 0x56383d0e5bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0165a0 tx=0x7fd3dc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2fdce199de3f57 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.444+0000 7fd3ca7fc700 1 -- 172.21.15.99:0/2149236975 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc021410 con 0x56383d0f1930 2023-08-25T08:59:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.444+0000 7fd3ca7fc700 1 -- 172.21.15.99:0/2149236975 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3dc00f030 con 0x56383d0f1930 2023-08-25T08:59:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.444+0000 7fd3ca7fc700 1 -- 172.21.15.99:0/2149236975 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc020850 con 0x56383d0f1930 2023-08-25T08:59:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2149236975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0f1930 msgr2=0x56383d0e5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0f1930 0x56383d0e5bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0165a0 tx=0x7fd3dc014000 comp rx=0 tx=0).stop 2023-08-25T08:59:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2149236975 shutdown_connections 2023-08-25T08:59:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0f1930 0x56383d0e5bd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0eb990 0x56383d0f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/2149236975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1010 0x56383d11ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2149236975 >> 172.21.15.99:0/2149236975 conn(0x56383d060080 msgr2=0x56383d0ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2149236975 shutdown_connections 2023-08-25T08:59:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.445+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/2149236975 wait complete. 2023-08-25T08:59:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.446+0000 7fd3fdb38b80 1 -- start start 2023-08-25T08:59:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.446+0000 7fd3fdb38b80 1 Processor -- start 2023-08-25T08:59:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d102a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d102f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3fdb38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d171610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383d171b50 con 0x56383d0f1930 2023-08-25T08:59:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383d171c90 con 0x56383d0f1010 2023-08-25T08:59:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3fdb38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383d171dd0 con 0x56383d0eb990 2023-08-25T08:59:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3fdb38b80 10 client.0 _ll_register_callbacks cb 0x7fd3d4096040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:59:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3e2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3e2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d102f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47068/0 (socket says 172.21.15.99:47068) 2023-08-25T08:59:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.447+0000 7fd3e2ffd700 1 -- 172.21.15.99:0/4261108970 learned_addr learned my addr 172.21.15.99:0/4261108970 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3e37fe700 1 -- 172.21.15.99:0/4261108970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 msgr2=0x56383d102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d102f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3e37fe700 1 -- 172.21.15.99:0/4261108970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 msgr2=0x56383d171610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d171610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3e37fe700 1 -- 172.21.15.99:0/4261108970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56383ce69d80 con 0x56383d0eb990 2023-08-25T08:59:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3e37fe700 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d102a10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4000c00 tx=0x7fd3d40996a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6622112e74a573f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T08:59:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/4261108970 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4095c50 con 0x56383d0eb990 2023-08-25T08:59:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/4261108970 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd3d409a310 con 0x56383d0eb990 2023-08-25T08:59:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.448+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/4261108970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56383d0de600 con 0x56383d0eb990 2023-08-25T08:59:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/4261108970 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3d4095db0 con 0x56383d0eb990 2023-08-25T08:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3fdb38b80 10 client.25300 fetch_fsmap learned FSMap version 23 2023-08-25T08:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/4261108970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd3dc008df0 con 0x56383d0eb990 2023-08-25T08:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/4261108970 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d40c7020 con 0x56383d0eb990 2023-08-25T08:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/4261108970 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd3d4098b70 con 0x56383d0eb990 2023-08-25T08:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3fdb38b80 10 client.25300 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3fdb38b80 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T08:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3fdb38b80 20 client.25300 populate_metadata read hostname 'smithi099' 2023-08-25T08:59:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56383d1731a0 0x56383d1755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3fdb38b80 4 client.25300 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T08:59:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.449+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/4261108970 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56383d036910 con 0x56383d1731a0 2023-08-25T08:59:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.450+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56383d1731a0 0x56383d1755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.450+0000 7fd3c2ffd700 10 client.25300 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.450+0000 7fd3e2ffd700 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56383d1731a0 0x56383d1755c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8447cbd97a138c9a server_cookie=e0b36400400aa09f in_seq=0 out_seq=0 2023-08-25T08:59:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.457+0000 7fd3c2ffd700 1 -- 172.21.15.99:0/4261108970 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56383d036910 con 0x56383d1731a0 2023-08-25T08:59:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.457+0000 7fd3c2ffd700 10 client.25300 handle_command_reply: tid=0 2023-08-25T08:59:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.457+0000 7fd3fdb38b80 1 client.25300 shutdown 2023-08-25T08:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/4261108970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 msgr2=0x56383d102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d102a10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4000c00 tx=0x7fd3d40996a0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/4261108970 shutdown_connections 2023-08-25T08:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56383d0f1930 0x56383d171610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383d0eb990 0x56383d102a10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56383d1731a0 0x56383d1755c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 --2- 172.21.15.99:0/4261108970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56383d0f1010 0x56383d102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3c2ffd700 0 client.25300 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/4261108970 >> 172.21.15.99:0/4261108970 conn(0x56383d060080 msgr2=0x56383d0e5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/4261108970 shutdown_connections 2023-08-25T08:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 1 -- 172.21.15.99:0/4261108970 wait complete. 2023-08-25T08:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.458+0000 7fd3fdb38b80 20 client.25300 trim_cache size 0 max 16384 2023-08-25T08:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T08:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T08:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T08:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T08:59:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd3cc0a01d0 msgr2=0x7fd3cc0a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd3cc0a01d0 0x7fd3cc0a2690 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009550 tx=0x7fd3dc00b040 comp rx=0 tx=0).stop 2023-08-25T08:59:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4067d10 msgr2=0x7fd3e4198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4067d10 0x7fd3e4198990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400b580 tx=0x7fd3d400f690 comp rx=0 tx=0).stop 2023-08-25T08:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/2057375115 shutdown_connections 2023-08-25T08:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd3cc0a01d0 0x7fd3cc0a2690 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3e41068a0 0x7fd3e419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4067d10 0x7fd3e4198990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 --2- 172.21.15.99:0/2057375115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e40686d0 0x7fd3e4198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/2057375115 >> 172.21.15.99:0/2057375115 conn(0x7fd3e4075050 msgr2=0x7fd3e410b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.459+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/2057375115 shutdown_connections 2023-08-25T08:59:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:39.460+0000 7fd3e8f9d700 1 -- 172.21.15.99:0/2057375115 wait complete. 2023-08-25T08:59:39.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:39 smithi181 ceph-mon[131296]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-25T08:59:39.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:39 smithi099 ceph-mon[97641]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-25T08:59:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:39 smithi157 ceph-mon[130879]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-25T08:59:41.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:41 smithi181 ceph-mon[131296]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-08-25T08:59:41.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:41 smithi099 ceph-mon[97641]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-08-25T08:59:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:41 smithi157 ceph-mon[130879]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-08-25T08:59:43.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:43 smithi181 ceph-mon[131296]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-08-25T08:59:43.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:43 smithi099 ceph-mon[97641]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-08-25T08:59:43.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:43 smithi157 ceph-mon[130879]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-08-25T08:59:45.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:45 smithi157 ceph-mon[130879]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2023-08-25T08:59:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:45 smithi181 ceph-mon[131296]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2023-08-25T08:59:45.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:45 smithi099 ceph-mon[97641]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2023-08-25T08:59:47.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:47 smithi181 ceph-mon[131296]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-08-25T08:59:47.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:47 smithi099 ceph-mon[97641]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-08-25T08:59:47.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:47 smithi157 ceph-mon[130879]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-08-25T08:59:49.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:49 smithi181 ceph-mon[131296]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-08-25T08:59:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:49 smithi099 ceph-mon[97641]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-08-25T08:59:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:49 smithi157 ceph-mon[130879]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-08-25T08:59:51.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:51 smithi181 ceph-mon[131296]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-08-25T08:59:51.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:51 smithi099 ceph-mon[97641]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-08-25T08:59:51.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:51 smithi157 ceph-mon[130879]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-08-25T08:59:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 -- 172.21.15.99:0/1473426526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 msgr2=0x7f44f8106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 --2- 172.21.15.99:0/1473426526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 0x7f44f8106c90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009990 tx=0x7f44e8009ca0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 -- 172.21.15.99:0/1473426526 shutdown_connections 2023-08-25T08:59:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 --2- 172.21.15.99:0/1473426526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f8101260 0x7f44f810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 --2- 172.21.15.99:0/1473426526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 0x7f44f8106c90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 --2- 172.21.15.99:0/1473426526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f81008a0 0x7f44f8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 -- 172.21.15.99:0/1473426526 >> 172.21.15.99:0/1473426526 conn(0x7f44f80fc370 msgr2=0x7f44f80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 -- 172.21.15.99:0/1473426526 shutdown_connections 2023-08-25T08:59:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.609+0000 7f44fcbff700 1 -- 172.21.15.99:0/1473426526 wait complete. 2023-08-25T08:59:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.610+0000 7f44fcbff700 1 Processor -- start 2023-08-25T08:59:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44fcbff700 1 -- start start 2023-08-25T08:59:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44fcbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f81008a0 0x7f44f819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44fcbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f8101260 0x7f44f819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44fcbff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 0x7f44f81967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44fcbff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8196e10 con 0x7f44f81068b0 2023-08-25T08:59:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44fcbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8196f80 con 0x7f44f8101260 2023-08-25T08:59:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44fcbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8197140 con 0x7f44f81008a0 2023-08-25T08:59:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44f5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f8101260 0x7f44f819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44f6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 0x7f44f81967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44f5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f8101260 0x7f44f819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:39426/0 (socket says 172.21.15.99:39426) 2023-08-25T08:59:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44f5d9b700 1 -- 172.21.15.99:0/2953448373 learned_addr learned my addr 172.21.15.99:0/2953448373 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.611+0000 7f44f659c700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f81008a0 0x7f44f819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44f6d9d700 1 -- 172.21.15.99:0/2953448373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f81008a0 msgr2=0x7f44f819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f81008a0 0x7f44f819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44f6d9d700 1 -- 172.21.15.99:0/2953448373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f8101260 msgr2=0x7f44f819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f8101260 0x7f44f819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44f6d9d700 1 -- 172.21.15.99:0/2953448373 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80095f0 con 0x7f44f81068b0 2023-08-25T08:59:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f8101260 0x7f44f819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 0x7f44f81967a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f44f000efc0 tx=0x7f44f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44e77fe700 1 -- 172.21.15.99:0/2953448373 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0018070 con 0x7f44f81068b0 2023-08-25T08:59:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44fcbff700 1 -- 172.21.15.99:0/2953448373 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f81973c0 con 0x7f44f81068b0 2023-08-25T08:59:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44e77fe700 1 -- 172.21.15.99:0/2953448373 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f000f040 con 0x7f44f81068b0 2023-08-25T08:59:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44e77fe700 1 -- 172.21.15.99:0/2953448373 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f000cd40 con 0x7f44f81068b0 2023-08-25T08:59:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.612+0000 7f44fcbff700 1 -- 172.21.15.99:0/2953448373 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f81a3930 con 0x7f44f81068b0 2023-08-25T08:59:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.615+0000 7f44e77fe700 1 -- 172.21.15.99:0/2953448373 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 134847+0+0 (secure 0 0 0) 0x7f44f0003a00 con 0x7f44f81068b0 2023-08-25T08:59:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.617+0000 7f4510488b80 1 Processor -- start 2023-08-25T08:59:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.617+0000 7f4510488b80 1 -- start start 2023-08-25T08:59:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f44e77fe700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f44e00a0150 0x7f44e00a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f4510488b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd14320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f4510488b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd0e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f4510488b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f4510488b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bc76750 con 0x56403bd0e990 2023-08-25T08:59:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f4510488b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bc67720 con 0x56403bd13f40 2023-08-25T08:59:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f4510488b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bc65040 con 0x56403bd14860 2023-08-25T08:59:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f44e77fe700 1 -- 172.21.15.99:0/2953448373 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f44f00cb410 con 0x7f44f81068b0 2023-08-25T08:59:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f44f6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f44f659c700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f44e00a0150 0x7f44e00a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f44f5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd14320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f44f6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38572/0 (socket says 172.21.15.99:38572) 2023-08-25T08:59:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f44f6d9d700 1 -- 172.21.15.99:0/1045196554 learned_addr learned my addr 172.21.15.99:0/1045196554 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.618+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd0e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44f6d9d700 1 -- 172.21.15.99:0/1045196554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 msgr2=0x56403bd0e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd0e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44f6d9d700 1 -- 172.21.15.99:0/1045196554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 msgr2=0x56403bd14320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd14320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44f6d9d700 1 -- 172.21.15.99:0/1045196554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403bae05d0 con 0x56403bd0e990 2023-08-25T08:59:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd14320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T08:59:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44f659c700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f44e00a0150 0x7f44e00a2610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f44e800b2e0 tx=0x7f44e8005fb0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0edd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44f00a01f0 tx=0x7f44f00987f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d522cf526c30f9b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44e4ff9700 1 -- 172.21.15.99:0/1045196554 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0076250 con 0x56403bd0e990 2023-08-25T08:59:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44e4ff9700 1 -- 172.21.15.99:0/1045196554 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f0076070 con 0x56403bd0e990 2023-08-25T08:59:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.619+0000 7f44e4ff9700 1 -- 172.21.15.99:0/1045196554 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0075c60 con 0x56403bd0e990 2023-08-25T08:59:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 -- 172.21.15.99:0/1045196554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 msgr2=0x56403bd0edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0edd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44f00a01f0 tx=0x7f44f00987f0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 -- 172.21.15.99:0/1045196554 shutdown_connections 2023-08-25T08:59:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0edd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd14320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 --2- 172.21.15.99:0/1045196554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd0e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 -- 172.21.15.99:0/1045196554 >> 172.21.15.99:0/1045196554 conn(0x56403bc83080 msgr2=0x56403bd08d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 -- 172.21.15.99:0/1045196554 shutdown_connections 2023-08-25T08:59:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.620+0000 7f4510488b80 1 -- 172.21.15.99:0/1045196554 wait complete. 2023-08-25T08:59:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 1 -- start start 2023-08-25T08:59:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 1 Processor -- start 2023-08-25T08:59:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd0e990 0x56403bd1a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd13f40 0x56403bd17940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd17fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bc60930 con 0x56403bd13f40 2023-08-25T08:59:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bbfeb50 con 0x56403bd0e990 2023-08-25T08:59:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bbfe850 con 0x56403bd14860 2023-08-25T08:59:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.621+0000 7f4510488b80 10 client.0 _ll_register_callbacks cb 0x56403bb48a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:59:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd0e990 0x56403bd1a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd17fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd13f40 0x56403bd17940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd0e990 0x56403bd1a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:39450/0 (socket says 172.21.15.99:39450) 2023-08-25T08:59:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f659c700 1 -- 172.21.15.99:0/2928811081 learned_addr learned my addr 172.21.15.99:0/2928811081 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f6d9d700 1 -- 172.21.15.99:0/2928811081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd0e990 msgr2=0x56403bd1a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd0e990 0x56403bd1a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f6d9d700 1 -- 172.21.15.99:0/2928811081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd13f40 msgr2=0x56403bd17940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd13f40 0x56403bd17940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f6d9d700 1 -- 172.21.15.99:0/2928811081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec01d040 con 0x56403bd14860 2023-08-25T08:59:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd17fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x56403bd1ad20 tx=0x7f44f00c8a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=622d0353e9b85d95 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T08:59:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44ddffb700 1 -- 172.21.15.99:0/2928811081 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0004ad0 con 0x56403bd14860 2023-08-25T08:59:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f4510488b80 1 -- 172.21.15.99:0/2928811081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56403bcde160 con 0x56403bd14860 2023-08-25T08:59:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.622+0000 7f44ddffb700 1 -- 172.21.15.99:0/2928811081 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f44f001b320 con 0x56403bd14860 2023-08-25T08:59:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f44ddffb700 1 -- 172.21.15.99:0/2928811081 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44f00990a0 con 0x56403bd14860 2023-08-25T08:59:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f4510488b80 10 client.25312 fetch_fsmap learned FSMap version 23 2023-08-25T08:59:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f4510488b80 1 -- 172.21.15.99:0/2928811081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f44f81a3930 con 0x56403bd14860 2023-08-25T08:59:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f44ddffb700 1 -- 172.21.15.99:0/2928811081 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56403bcde160 con 0x56403bd14860 2023-08-25T08:59:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f44ddffb700 1 -- 172.21.15.99:0/2928811081 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f44f00cb050 con 0x56403bd14860 2023-08-25T08:59:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f4510488b80 10 client.25312 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:59:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f4510488b80 10 client.25312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T08:59:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f4510488b80 20 client.25312 populate_metadata read hostname 'smithi099' 2023-08-25T08:59:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f4510488b80 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56403bd27d10 0x56403bd2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f4510488b80 4 client.25312 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T08:59:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.623+0000 7f4510488b80 1 -- 172.21.15.99:0/2928811081 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56403bc06b40 con 0x56403bd27d10 2023-08-25T08:59:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.624+0000 7f44f659c700 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56403bd27d10 0x56403bd2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.624+0000 7f44f659c700 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56403bd27d10 0x56403bd2a100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f264bfcc1f489fbd server_cookie=aa5ccd3abbdafe59 in_seq=0 out_seq=0 2023-08-25T08:59:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.624+0000 7f44ddffb700 10 client.25312 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f44ddffb700 1 -- 172.21.15.99:0/2928811081 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56403bc06b40 con 0x56403bd27d10 2023-08-25T08:59:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f44ddffb700 10 client.25312 handle_command_reply: tid=0 2023-08-25T08:59:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 client.25312 shutdown 2023-08-25T08:59:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 -- 172.21.15.99:0/2928811081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 msgr2=0x56403bd17fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd17fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x56403bd1ad20 tx=0x7f44f00c8a20 comp rx=0 tx=0).stop 2023-08-25T08:59:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 -- 172.21.15.99:0/2928811081 shutdown_connections 2023-08-25T08:59:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd17fc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56403bd27d10 0x56403bd2a100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd0e990 0x56403bd1a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 --2- 172.21.15.99:0/2928811081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd13f40 0x56403bd17940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f44ddffb700 0 client.25312 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 -- 172.21.15.99:0/2928811081 >> 172.21.15.99:0/2928811081 conn(0x56403bc83080 msgr2=0x56403bd03820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 -- 172.21.15.99:0/2928811081 shutdown_connections 2023-08-25T08:59:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.625+0000 7f4510488b80 1 -- 172.21.15.99:0/2928811081 wait complete. 2023-08-25T08:59:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.626+0000 7f4510488b80 20 client.25312 trim_cache size 0 max 16384 2023-08-25T08:59:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f4510488b80 1 Processor -- start 2023-08-25T08:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f4510488b80 1 -- start start 2023-08-25T08:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f4510488b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f4510488b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd13f40 0x56403bd11a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f4510488b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd14860 0x56403bd11fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f4510488b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bae28c0 con 0x56403bd0e990 2023-08-25T08:59:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f4510488b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bd09250 con 0x56403bd14860 2023-08-25T08:59:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f4510488b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bd09a60 con 0x56403bd13f40 2023-08-25T08:59:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f44f659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f44f5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd13f40 0x56403bd11a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f44f5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd13f40 0x56403bd11a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:40174/0 (socket says 172.21.15.99:40174) 2023-08-25T08:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f44f5d9b700 1 -- 172.21.15.99:0/2042977086 learned_addr learned my addr 172.21.15.99:0/2042977086 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.641+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/2042977086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd14860 0x56403bd11fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44f5d9b700 1 -- 172.21.15.99:0/2042977086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd14860 msgr2=0x56403bd11fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/2042977086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd14860 0x56403bd11fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44f5d9b700 1 -- 172.21.15.99:0/2042977086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 msgr2=0x56403bd0e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/2042977086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44f5d9b700 1 -- 172.21.15.99:0/2042977086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403bd0f040 con 0x56403bd13f40 2023-08-25T08:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/2042977086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd13f40 0x56403bd11a90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00f6b0 tx=0x7f44ec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba275ab0f2cb964d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44e57fa700 1 -- 172.21.15.99:0/2042977086 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec004020 con 0x56403bd13f40 2023-08-25T08:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44e57fa700 1 -- 172.21.15.99:0/2042977086 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44ec003680 con 0x56403bd13f40 2023-08-25T08:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.642+0000 7f44e57fa700 1 -- 172.21.15.99:0/2042977086 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec022400 con 0x56403bd13f40 2023-08-25T08:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 -- 172.21.15.99:0/2042977086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd13f40 msgr2=0x56403bd11a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 --2- 172.21.15.99:0/2042977086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd13f40 0x56403bd11a90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00f6b0 tx=0x7f44ec0047c0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 -- 172.21.15.99:0/2042977086 shutdown_connections 2023-08-25T08:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 --2- 172.21.15.99:0/2042977086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd14860 0x56403bd11fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 --2- 172.21.15.99:0/2042977086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd0e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 --2- 172.21.15.99:0/2042977086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd13f40 0x56403bd11a90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 -- 172.21.15.99:0/2042977086 >> 172.21.15.99:0/2042977086 conn(0x56403bc83080 msgr2=0x56403bd239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 -- 172.21.15.99:0/2042977086 shutdown_connections 2023-08-25T08:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.643+0000 7f4510488b80 1 -- 172.21.15.99:0/2042977086 wait complete. 2023-08-25T08:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.644+0000 7f4510488b80 1 -- start start 2023-08-25T08:59:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.644+0000 7f4510488b80 1 Processor -- start 2023-08-25T08:59:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f4510488b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd1d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f4510488b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd1d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f4510488b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd7aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f4510488b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bd1e080 con 0x56403bd0e990 2023-08-25T08:59:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f4510488b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bd7b020 con 0x56403bd13f40 2023-08-25T08:59:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f4510488b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bd7b160 con 0x56403bd14860 2023-08-25T08:59:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f4510488b80 10 client.0 _ll_register_callbacks cb 0x7f44ec009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T08:59:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd1d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd1d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38612/0 (socket says 172.21.15.99:38612) 2023-08-25T08:59:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f659c700 1 -- 172.21.15.99:0/3471508046 learned_addr learned my addr 172.21.15.99:0/3471508046 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T08:59:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd1d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f6d9d700 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd7aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f659c700 1 -- 172.21.15.99:0/3471508046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 msgr2=0x56403bd7aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f659c700 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd7aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f659c700 1 -- 172.21.15.99:0/3471508046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 msgr2=0x56403bd1d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T08:59:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f659c700 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd1d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.645+0000 7f44f659c700 1 -- 172.21.15.99:0/3471508046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec00f470 con 0x56403bd0e990 2023-08-25T08:59:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f44f659c700 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd1d230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f44e8000c00 tx=0x7f44e8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02ca415c4e17243 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T08:59:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f44ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T08:59:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f44ddffb700 1 -- 172.21.15.99:0/3471508046 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e802b040 con 0x56403bd0e990 2023-08-25T08:59:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f4510488b80 1 -- 172.21.15.99:0/3471508046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56403bd01600 con 0x56403bd0e990 2023-08-25T08:59:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f44ddffb700 1 -- 172.21.15.99:0/3471508046 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f44e8033e00 con 0x56403bd0e990 2023-08-25T08:59:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f44ddffb700 1 -- 172.21.15.99:0/3471508046 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44e8033040 con 0x56403bd0e990 2023-08-25T08:59:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f4510488b80 10 client.15279 fetch_fsmap learned FSMap version 23 2023-08-25T08:59:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f4510488b80 1 -- 172.21.15.99:0/3471508046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56403ba8cd80 con 0x56403bd0e990 2023-08-25T08:59:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.646+0000 7f44ddffb700 1 -- 172.21.15.99:0/3471508046 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44e8047080 con 0x56403bd0e990 2023-08-25T08:59:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.647+0000 7f44ddffb700 1 -- 172.21.15.99:0/3471508046 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f44e8039020 con 0x56403bd0e990 2023-08-25T08:59:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.647+0000 7f4510488b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T08:59:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.647+0000 7f4510488b80 10 client.15279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T08:59:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.647+0000 7f4510488b80 20 client.15279 populate_metadata read hostname 'smithi099' 2023-08-25T08:59:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.647+0000 7f4510488b80 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56403bd7c7f0 0x56403bd7ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T08:59:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.647+0000 7f4510488b80 4 client.15279 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T08:59:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.647+0000 7f4510488b80 1 -- 172.21.15.99:0/3471508046 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56403bc59910 con 0x56403bd7c7f0 2023-08-25T08:59:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.647+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56403bd7c7f0 0x56403bd7ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T08:59:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.648+0000 7f44ddffb700 10 client.15279 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.648+0000 7f44f5d9b700 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56403bd7c7f0 0x56403bd7ec10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c584e6729a5a8fb1 server_cookie=9791e00675d07c65 in_seq=0 out_seq=0 2023-08-25T08:59:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f44ddffb700 1 -- 172.21.15.99:0/3471508046 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56403bc59910 con 0x56403bd7c7f0 2023-08-25T08:59:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f44ddffb700 10 client.15279 handle_command_reply: tid=0 2023-08-25T08:59:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 client.15279 shutdown 2023-08-25T08:59:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 -- 172.21.15.99:0/3471508046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 msgr2=0x56403bd1d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd1d230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f44e8000c00 tx=0x7f44e8009ca0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 -- 172.21.15.99:0/3471508046 shutdown_connections 2023-08-25T08:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403bd14860 0x56403bd7aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56403bd0e990 0x56403bd1d230 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56403bd7c7f0 0x56403bd7ec10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 --2- 172.21.15.99:0/3471508046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403bd13f40 0x56403bd1d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 -- 172.21.15.99:0/3471508046 >> 172.21.15.99:0/3471508046 conn(0x56403bc83080 msgr2=0x56403bd11a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f44ddffb700 0 client.15279 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T08:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 -- 172.21.15.99:0/3471508046 shutdown_connections 2023-08-25T08:59:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 1 -- 172.21.15.99:0/3471508046 wait complete. 2023-08-25T08:59:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.655+0000 7f4510488b80 20 client.15279 trim_cache size 0 max 16384 2023-08-25T08:59:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T08:59:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T08:59:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T08:59:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T08:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 -- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f44e00a0150 msgr2=0x7f44e00a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f44e00a0150 0x7f44e00a2610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f44e800b2e0 tx=0x7f44e8005fb0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 -- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 msgr2=0x7f44f81967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T08:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 0x7f44f81967a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f44f000efc0 tx=0x7f44f000c3a0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 -- 172.21.15.99:0/2953448373 shutdown_connections 2023-08-25T08:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f44e00a0150 0x7f44e00a2610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f81068b0 0x7f44f81967a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f81008a0 0x7f44f819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 --2- 172.21.15.99:0/2953448373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f8101260 0x7f44f819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T08:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 -- 172.21.15.99:0/2953448373 >> 172.21.15.99:0/2953448373 conn(0x7f44f80fc370 msgr2=0x7f44f80fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T08:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 -- 172.21.15.99:0/2953448373 shutdown_connections 2023-08-25T08:59:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T08:59:52.656+0000 7f44fcbff700 1 -- 172.21.15.99:0/2953448373 wait complete. 2023-08-25T08:59:53.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:53 smithi181 ceph-mon[131296]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-08-25T08:59:53.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:53 smithi099 ceph-mon[97641]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-08-25T08:59:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:53 smithi157 ceph-mon[130879]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-08-25T08:59:53.732 INFO:tasks.workunit.client.0.smithi099.stderr:Note: switching to '4b369cf18ed1391a426ab4ae86da834e9c074f81'. 2023-08-25T08:59:53.733 INFO:tasks.workunit.client.0.smithi099.stderr: 2023-08-25T08:59:53.733 INFO:tasks.workunit.client.0.smithi099.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-08-25T08:59:53.733 INFO:tasks.workunit.client.0.smithi099.stderr:changes and commit them, and you can discard any commits you make in this 2023-08-25T08:59:53.733 INFO:tasks.workunit.client.0.smithi099.stderr:state without impacting any branches by switching back to a branch. 2023-08-25T08:59:53.733 INFO:tasks.workunit.client.0.smithi099.stderr: 2023-08-25T08:59:53.733 INFO:tasks.workunit.client.0.smithi099.stderr:If you want to create a new branch to retain commits you create, you may 2023-08-25T08:59:53.734 INFO:tasks.workunit.client.0.smithi099.stderr:do so (now or later) by using -c with the switch command. Example: 2023-08-25T08:59:53.734 INFO:tasks.workunit.client.0.smithi099.stderr: 2023-08-25T08:59:53.734 INFO:tasks.workunit.client.0.smithi099.stderr: git switch -c 2023-08-25T08:59:53.734 INFO:tasks.workunit.client.0.smithi099.stderr: 2023-08-25T08:59:53.735 INFO:tasks.workunit.client.0.smithi099.stderr:Or undo this operation with: 2023-08-25T08:59:53.735 INFO:tasks.workunit.client.0.smithi099.stderr: 2023-08-25T08:59:53.735 INFO:tasks.workunit.client.0.smithi099.stderr: git switch - 2023-08-25T08:59:53.735 INFO:tasks.workunit.client.0.smithi099.stderr: 2023-08-25T08:59:53.736 INFO:tasks.workunit.client.0.smithi099.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-08-25T08:59:53.736 INFO:tasks.workunit.client.0.smithi099.stderr: 2023-08-25T08:59:53.736 INFO:tasks.workunit.client.0.smithi099.stderr:HEAD is now at 4b369cf18ed qa: inherit RunCephCmd in CephTestCase instead of CephFSTestCase 2023-08-25T08:59:53.748 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-08-25T08:59:53.800 INFO:tasks.workunit.client.0.smithi099.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-08-25T08:59:53.805 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-08-25T08:59:53.806 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-08-25T08:59:54.403 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-08-25T08:59:54.489 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-08-25T08:59:54.553 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-08-25T08:59:54.557 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-08-25T08:59:54.558 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-08-25T08:59:54.620 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-08-25T08:59:54.627 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T08:59:54.627 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-08-25T08:59:54.672 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-08-25T08:59:54.673 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-08-25T08:59:54.674 DEBUG:teuthology.orchestra.run.smithi099:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4b369cf18ed1391a426ab4ae86da834e9c074f81 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-08-25T08:59:54.772 INFO:tasks.workunit.client.0.smithi099.stderr:+ dbench 1 2023-08-25T08:59:54.778 INFO:tasks.workunit.client.0.smithi099.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-08-25T08:59:54.778 INFO:tasks.workunit.client.0.smithi099.stdout: 2023-08-25T08:59:54.778 INFO:tasks.workunit.client.0.smithi099.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-08-25T08:59:54.782 INFO:tasks.workunit.client.0.smithi099.stdout:0 of 1 processes prepared for launch 0 sec 2023-08-25T08:59:54.882 INFO:tasks.workunit.client.0.smithi099.stdout:1 of 1 processes prepared for launch 0 sec 2023-08-25T08:59:54.883 INFO:tasks.workunit.client.0.smithi099.stdout:releasing clients 2023-08-25T08:59:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:55 smithi157 ceph-mon[130879]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2023-08-25T08:59:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:55 smithi181 ceph-mon[131296]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2023-08-25T08:59:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:55 smithi099 ceph-mon[97641]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2023-08-25T08:59:55.883 INFO:tasks.workunit.client.0.smithi099.stdout: 1 3329 35.37 MB/sec warmup 1 sec latency 6.198 ms 2023-08-25T08:59:56.883 INFO:tasks.workunit.client.0.smithi099.stdout: 1 6162 24.06 MB/sec warmup 2 sec latency 5.933 ms 2023-08-25T08:59:57.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:57 smithi181 ceph-mon[131296]: pgmap v26: 97 pgs: 97 active+clean; 2.8 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 153 KiB/s wr, 31 op/s 2023-08-25T08:59:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:57 smithi099 ceph-mon[97641]: pgmap v26: 97 pgs: 97 active+clean; 2.8 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 153 KiB/s wr, 31 op/s 2023-08-25T08:59:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:57 smithi157 ceph-mon[130879]: pgmap v26: 97 pgs: 97 active+clean; 2.8 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 153 KiB/s wr, 31 op/s 2023-08-25T08:59:57.883 INFO:tasks.workunit.client.0.smithi099.stdout: 1 8913 21.52 MB/sec warmup 3 sec latency 6.438 ms 2023-08-25T08:59:58.883 INFO:tasks.workunit.client.0.smithi099.stdout: 1 11562 19.76 MB/sec warmup 4 sec latency 9.111 ms 2023-08-25T08:59:59.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 08:59:59 smithi181 ceph-mon[131296]: pgmap v27: 97 pgs: 97 active+clean; 3.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-08-25T08:59:59.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 08:59:59 smithi099 ceph-mon[97641]: pgmap v27: 97 pgs: 97 active+clean; 3.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-08-25T08:59:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 08:59:59 smithi157 ceph-mon[130879]: pgmap v27: 97 pgs: 97 active+clean; 3.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-08-25T08:59:59.883 INFO:tasks.workunit.client.0.smithi099.stdout: 1 14571 19.48 MB/sec warmup 5 sec latency 20.621 ms 2023-08-25T09:00:00.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:00 smithi181 ceph-mon[131296]: overall HEALTH_OK 2023-08-25T09:00:00.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:00 smithi099 ceph-mon[97641]: overall HEALTH_OK 2023-08-25T09:00:00.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:00 smithi157 ceph-mon[130879]: overall HEALTH_OK 2023-08-25T09:00:00.884 INFO:tasks.workunit.client.0.smithi099.stdout: 1 17427 18.47 MB/sec warmup 6 sec latency 14.925 ms 2023-08-25T09:00:01.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:01 smithi157 ceph-mon[130879]: pgmap v28: 97 pgs: 97 active+clean; 3.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-08-25T09:00:01.643 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:00:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T09:00:00.792+0000 7fecf38af700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T09:00:01.643 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:00:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T09:00:00.792+0000 7fecf38af700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T09:00:01.643 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:00:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T09:00:00.792+0000 7fecf38af700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T09:00:01.643 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:00:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T09:00:00.792+0000 7fecf38af700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T09:00:01.644 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:00:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T09:00:00.792+0000 7fecf38af700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T09:00:01.644 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:01 smithi099 ceph-mon[97641]: pgmap v28: 97 pgs: 97 active+clean; 3.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-08-25T09:00:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:01 smithi181 ceph-mon[131296]: pgmap v28: 97 pgs: 97 active+clean; 3.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-08-25T09:00:01.884 INFO:tasks.workunit.client.0.smithi099.stdout: 1 18570 16.72 MB/sec warmup 7 sec latency 100.697 ms 2023-08-25T09:00:02.884 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21613 16.93 MB/sec warmup 8 sec latency 37.884 ms 2023-08-25T09:00:03.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:03 smithi099 ceph-mon[97641]: pgmap v29: 97 pgs: 97 active+clean; 10 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 905 KiB/s wr, 141 op/s 2023-08-25T09:00:03.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:03 smithi099 ceph-mon[97641]: mgrmap e25: x(active, since 52s), standbys: y, z 2023-08-25T09:00:03.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:03 smithi157 ceph-mon[130879]: pgmap v29: 97 pgs: 97 active+clean; 10 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 905 KiB/s wr, 141 op/s 2023-08-25T09:00:03.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:03 smithi157 ceph-mon[130879]: mgrmap e25: x(active, since 52s), standbys: y, z 2023-08-25T09:00:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:03 smithi181 ceph-mon[131296]: pgmap v29: 97 pgs: 97 active+clean; 10 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 905 KiB/s wr, 141 op/s 2023-08-25T09:00:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:03 smithi181 ceph-mon[131296]: mgrmap e25: x(active, since 52s), standbys: y, z 2023-08-25T09:00:03.884 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22238 15.16 MB/sec warmup 9 sec latency 558.786 ms 2023-08-25T09:00:04.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:04 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:00:04.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:04 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:00:04.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:04 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:00:04.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22238 13.64 MB/sec warmup 10 sec latency 1558.932 ms 2023-08-25T09:00:05.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:05 smithi099 ceph-mon[97641]: pgmap v30: 97 pgs: 97 active+clean; 10 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 905 KiB/s wr, 139 op/s 2023-08-25T09:00:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:05 smithi157 ceph-mon[130879]: pgmap v30: 97 pgs: 97 active+clean; 10 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 905 KiB/s wr, 139 op/s 2023-08-25T09:00:05.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:05 smithi181 ceph-mon[131296]: pgmap v30: 97 pgs: 97 active+clean; 10 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 905 KiB/s wr, 139 op/s 2023-08-25T09:00:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.812+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/2072082652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb84069150 msgr2=0x7ffb84069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.812+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/2072082652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb84069150 0x7ffb84069530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffb740099c0 tx=0x7ffb74009cd0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.812+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/2072082652 shutdown_connections 2023-08-25T09:00:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.812+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/2072082652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb84105ce0 0x7ffb8410d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.812+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/2072082652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb84069150 0x7ffb84069530 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.812+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/2072082652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84105320 0x7ffb841057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.812+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/2072082652 >> 172.21.15.99:0/2072082652 conn(0x7ffb84076920 msgr2=0x7ffb84076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.812+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/2072082652 shutdown_connections 2023-08-25T09:00:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.813+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/2072082652 wait complete. 2023-08-25T09:00:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb8a7ff700 1 Processor -- start 2023-08-25T09:00:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb8a7ff700 1 -- start start 2023-08-25T09:00:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb8a7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84069150 0x7ffb840ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb8a7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb84105320 0x7ffb840fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb8a7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb84105ce0 0x7ffb841a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb8a7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb841a5150 con 0x7ffb84105ce0 2023-08-25T09:00:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb8a7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb841a5290 con 0x7ffb84069150 2023-08-25T09:00:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb8a7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb841a5400 con 0x7ffb84105320 2023-08-25T09:00:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb88d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb84105ce0 0x7ffb841a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.814+0000 7ffb837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb84105320 0x7ffb840fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84069150 0x7ffb840ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb84105320 0x7ffb840fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34514/0 (socket says 172.21.15.99:34514) 2023-08-25T09:00:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb837fe700 1 -- 172.21.15.99:0/3318117964 learned_addr learned my addr 172.21.15.99:0/3318117964 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb837fe700 1 -- 172.21.15.99:0/3318117964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84069150 msgr2=0x7ffb840ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb837fe700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84069150 0x7ffb840ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb837fe700 1 -- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb84105ce0 msgr2=0x7ffb841a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb837fe700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb84105ce0 0x7ffb841a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb837fe700 1 -- 172.21.15.99:0/3318117964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78009520 con 0x7ffb84105320 2023-08-25T09:00:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb837fe700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb84105320 0x7ffb840fff80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800eb30 tx=0x7ffb7800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.815+0000 7ffb817fa700 1 -- 172.21.15.99:0/3318117964 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78009880 con 0x7ffb84105320 2023-08-25T09:00:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.816+0000 7ffb817fa700 1 -- 172.21.15.99:0/3318117964 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb780044e0 con 0x7ffb84105320 2023-08-25T09:00:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.816+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/3318117964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb740095f0 con 0x7ffb84105320 2023-08-25T09:00:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.816+0000 7ffb817fa700 1 -- 172.21.15.99:0/3318117964 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78003d80 con 0x7ffb84105320 2023-08-25T09:00:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.816+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/3318117964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb841a5a60 con 0x7ffb84105320 2023-08-25T09:00:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.819+0000 7ffb817fa700 1 -- 172.21.15.99:0/3318117964 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) v1 ==== 134904+0+0 (secure 0 0 0) 0x7ffb78003f20 con 0x7ffb84105320 2023-08-25T09:00:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb9e080b80 1 Processor -- start 2023-08-25T09:00:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb9e080b80 1 -- start start 2023-08-25T09:00:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a204320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a1fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a166750 con 0x55bb4a204860 2023-08-25T09:00:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a157720 con 0x55bb4a1fe990 2023-08-25T09:00:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a155040 con 0x55bb4a203f40 2023-08-25T09:00:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb88d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a204320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a1fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.820+0000 7ffb88d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a204320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34530/0 (socket says 172.21.15.99:34530) 2023-08-25T09:00:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb837fe700 1 --2- 172.21.15.99:0/538958647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb88d9c700 1 -- 172.21.15.99:0/538958647 learned_addr learned my addr 172.21.15.99:0/538958647 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb817fa700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ffb6c0a01e0 0x7ffb6c0a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb83fff700 1 -- 172.21.15.99:0/538958647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 msgr2=0x55bb4a204320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb83fff700 1 --2- 172.21.15.99:0/538958647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a204320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb83fff700 1 -- 172.21.15.99:0/538958647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 msgr2=0x55bb4a1fedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb83fff700 1 --2- 172.21.15.99:0/538958647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb817fa700 1 -- 172.21.15.99:0/3318117964 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ffb78014070 con 0x7ffb84105320 2023-08-25T09:00:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb83fff700 1 -- 172.21.15.99:0/538958647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb49fd05d0 con 0x55bb4a204860 2023-08-25T09:00:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ffb6c0a01e0 0x7ffb6c0a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.821+0000 7ffb83fff700 1 --2- 172.21.15.99:0/538958647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a1fe450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffb740058e0 tx=0x7ffb74005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ffd62cd28e5d2e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb6a7fc700 1 -- 172.21.15.99:0/538958647 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400b990 con 0x55bb4a204860 2023-08-25T09:00:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb6a7fc700 1 -- 172.21.15.99:0/538958647 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7401e040 con 0x55bb4a204860 2023-08-25T09:00:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb6a7fc700 1 -- 172.21.15.99:0/538958647 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400fdc0 con 0x55bb4a204860 2023-08-25T09:00:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ffb6c0a01e0 0x7ffb6c0a26a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0075c0 tx=0x7ffb7c009040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb9e080b80 1 -- 172.21.15.99:0/538958647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 msgr2=0x55bb4a1fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/538958647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a1fe450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffb740058e0 tx=0x7ffb74005ee0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb9e080b80 1 -- 172.21.15.99:0/538958647 shutdown_connections 2023-08-25T09:00:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/538958647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/538958647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a204320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/538958647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a1fe450 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb9e080b80 1 -- 172.21.15.99:0/538958647 >> 172.21.15.99:0/538958647 conn(0x55bb4a173080 msgr2=0x55bb4a1f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.822+0000 7ffb9e080b80 1 -- 172.21.15.99:0/538958647 shutdown_connections 2023-08-25T09:00:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.823+0000 7ffb9e080b80 1 -- 172.21.15.99:0/538958647 wait complete. 2023-08-25T09:00:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.823+0000 7ffb9e080b80 1 -- start start 2023-08-25T09:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.823+0000 7ffb9e080b80 1 Processor -- start 2023-08-25T09:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a20a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a207890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a207ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a150930 con 0x55bb4a204860 2023-08-25T09:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a0eeb50 con 0x55bb4a1fe990 2023-08-25T09:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a0ee850 con 0x55bb4a203f40 2023-08-25T09:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb9e080b80 10 client.0 _ll_register_callbacks cb 0x55bb4a038a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:00:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb88d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a207ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a20a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb88d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a207ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52980/0 (socket says 172.21.15.99:52980) 2023-08-25T09:00:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb88d9c700 1 -- 172.21.15.99:0/1775962371 learned_addr learned my addr 172.21.15.99:0/1775962371 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.824+0000 7ffb837fe700 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a207890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb88d9c700 1 -- 172.21.15.99:0/1775962371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 msgr2=0x55bb4a207890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a207890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb88d9c700 1 -- 172.21.15.99:0/1775962371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 msgr2=0x55bb4a20a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a20a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb88d9c700 1 -- 172.21.15.99:0/1775962371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8410e020 con 0x55bb4a204860 2023-08-25T09:00:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb83fff700 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a20a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb837fe700 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a207890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a207ea0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c017ec0 tx=0x7ffb7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9383593edb2356 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb637fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:00:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb637fe700 1 -- 172.21.15.99:0/1775962371 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00e040 con 0x55bb4a204860 2023-08-25T09:00:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb637fe700 1 -- 172.21.15.99:0/1775962371 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ffb7c01c070 con 0x55bb4a204860 2023-08-25T09:00:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb9e080b80 1 -- 172.21.15.99:0/1775962371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb4a1ce160 con 0x55bb4a204860 2023-08-25T09:00:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb637fe700 1 -- 172.21.15.99:0/1775962371 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb7c003a00 con 0x55bb4a204860 2023-08-25T09:00:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb9e080b80 10 client.15297 fetch_fsmap learned FSMap version 24 2023-08-25T09:00:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.825+0000 7ffb9e080b80 1 -- 172.21.15.99:0/1775962371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bb49f7cd80 con 0x55bb4a204860 2023-08-25T09:00:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb637fe700 1 -- 172.21.15.99:0/1775962371 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7c030020 con 0x55bb4a204860 2023-08-25T09:00:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb637fe700 1 -- 172.21.15.99:0/1775962371 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffb7c023450 con 0x55bb4a204860 2023-08-25T09:00:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb9e080b80 10 client.15297 fetch_fsmap finished waiting for FSMap version 24 2023-08-25T09:00:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb9e080b80 10 client.15297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:00:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb9e080b80 20 client.15297 populate_metadata read hostname 'smithi099' 2023-08-25T09:00:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bb4a218130 0x55bb4a21a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb9e080b80 4 client.15297 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:00:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb9e080b80 1 -- 172.21.15.99:0/1775962371 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb4a0f6b40 con 0x55bb4a218130 2023-08-25T09:00:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.826+0000 7ffb83fff700 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bb4a218130 0x55bb4a21a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.827+0000 7ffb83fff700 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bb4a218130 0x55bb4a21a550 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb42ac76bf898cac server_cookie=a08ae22079de6f1d in_seq=0 out_seq=0 2023-08-25T09:00:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.827+0000 7ffb637fe700 10 client.15297 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb637fe700 1 -- 172.21.15.99:0/1775962371 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bb4a0f6b40 con 0x55bb4a218130 2023-08-25T09:00:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb637fe700 10 client.15297 handle_command_reply: tid=0 2023-08-25T09:00:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 client.15297 shutdown 2023-08-25T09:00:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 -- 172.21.15.99:0/1775962371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 msgr2=0x55bb4a207ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a207ea0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c017ec0 tx=0x7ffb7c004ad0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 -- 172.21.15.99:0/1775962371 shutdown_connections 2023-08-25T09:00:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a207ea0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bb4a218130 0x55bb4a21a550 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a1fe990 0x55bb4a20a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/1775962371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a207890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb637fe700 0 client.15297 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 -- 172.21.15.99:0/1775962371 >> 172.21.15.99:0/1775962371 conn(0x55bb4a173080 msgr2=0x55bb4a1f3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 -- 172.21.15.99:0/1775962371 shutdown_connections 2023-08-25T09:00:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.828+0000 7ffb9e080b80 1 -- 172.21.15.99:0/1775962371 wait complete. 2023-08-25T09:00:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.829+0000 7ffb9e080b80 20 client.15297 trim_cache size 0 max 16384 2023-08-25T09:00:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb9e080b80 1 Processor -- start 2023-08-25T09:00:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb9e080b80 1 -- start start 2023-08-25T09:00:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a20f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a204860 0x55bb4a20a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb49fd28c0 con 0x55bb4a1fe990 2023-08-25T09:00:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a20f750 con 0x55bb4a204860 2023-08-25T09:00:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a205e10 con 0x55bb4a203f40 2023-08-25T09:00:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52986/0 (socket says 172.21.15.99:52986) 2023-08-25T09:00:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb83fff700 1 -- 172.21.15.99:0/598920678 learned_addr learned my addr 172.21.15.99:0/598920678 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a204860 0x55bb4a20a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb837fe700 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a20f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb83fff700 1 -- 172.21.15.99:0/598920678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 msgr2=0x55bb4a20f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb83fff700 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a20f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb83fff700 1 -- 172.21.15.99:0/598920678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a204860 msgr2=0x55bb4a20a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb83fff700 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a204860 0x55bb4a20a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb83fff700 1 -- 172.21.15.99:0/598920678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb4a1ff040 con 0x55bb4a1fe990 2023-08-25T09:00:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.844+0000 7ffb837fe700 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a20f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:00:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.845+0000 7ffb83fff700 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fe450 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffb740098e0 tx=0x7ffb74005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ada1b019edf9ad7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.845+0000 7ffb6affd700 1 -- 172.21.15.99:0/598920678 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74031070 con 0x55bb4a1fe990 2023-08-25T09:00:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.845+0000 7ffb6affd700 1 -- 172.21.15.99:0/598920678 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7401e040 con 0x55bb4a1fe990 2023-08-25T09:00:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.845+0000 7ffb6affd700 1 -- 172.21.15.99:0/598920678 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74034a80 con 0x55bb4a1fe990 2023-08-25T09:00:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.845+0000 7ffb9e080b80 1 -- 172.21.15.99:0/598920678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a1fe990 msgr2=0x55bb4a1fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.845+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fe450 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffb740098e0 tx=0x7ffb74005fb0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 -- 172.21.15.99:0/598920678 shutdown_connections 2023-08-25T09:00:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a204860 0x55bb4a20a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a1fe990 0x55bb4a1fe450 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/598920678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a203f40 0x55bb4a20f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 -- 172.21.15.99:0/598920678 >> 172.21.15.99:0/598920678 conn(0x55bb4a173080 msgr2=0x55bb4a201af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 -- 172.21.15.99:0/598920678 shutdown_connections 2023-08-25T09:00:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 -- 172.21.15.99:0/598920678 wait complete. 2023-08-25T09:00:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 -- start start 2023-08-25T09:00:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.846+0000 7ffb9e080b80 1 Processor -- start 2023-08-25T09:00:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a1fe990 0x55bb4a20f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a203f40 0x55bb4a20b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb9e080b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a20b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a20bd20 con 0x55bb4a204860 2023-08-25T09:00:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a20be90 con 0x55bb4a203f40 2023-08-25T09:00:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb9e080b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4a20c050 con 0x55bb4a1fe990 2023-08-25T09:00:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb9e080b80 10 client.0 _ll_register_callbacks cb 0x7ffb7c017860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:00:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb88d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a20b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb88d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a20b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52994/0 (socket says 172.21.15.99:52994) 2023-08-25T09:00:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb88d9c700 1 -- 172.21.15.99:0/2247531832 learned_addr learned my addr 172.21.15.99:0/2247531832 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb83fff700 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a1fe990 0x55bb4a20f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb837fe700 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a203f40 0x55bb4a20b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb88d9c700 1 -- 172.21.15.99:0/2247531832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a1fe990 msgr2=0x55bb4a20f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a1fe990 0x55bb4a20f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb88d9c700 1 -- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a203f40 msgr2=0x55bb4a20b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a203f40 0x55bb4a20b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb88d9c700 1 -- 172.21.15.99:0/2247531832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb49fd05d0 con 0x55bb4a204860 2023-08-25T09:00:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.847+0000 7ffb837fe700 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a203f40 0x55bb4a20b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:00:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb88d9c700 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a20b700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c017fd0 tx=0x7ffb7c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d904d64c35222b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb637fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:00:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb637fe700 1 -- 172.21.15.99:0/2247531832 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00e040 con 0x55bb4a204860 2023-08-25T09:00:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb637fe700 1 -- 172.21.15.99:0/2247531832 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ffb7c01c040 con 0x55bb4a204860 2023-08-25T09:00:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 1 -- 172.21.15.99:0/2247531832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb4a1f1600 con 0x55bb4a204860 2023-08-25T09:00:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb637fe700 1 -- 172.21.15.99:0/2247531832 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb7c004d40 con 0x55bb4a204860 2023-08-25T09:00:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 10 client.15303 fetch_fsmap learned FSMap version 24 2023-08-25T09:00:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 1 -- 172.21.15.99:0/2247531832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bb49f65c00 con 0x55bb4a204860 2023-08-25T09:00:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb637fe700 1 -- 172.21.15.99:0/2247531832 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb4a1f1600 con 0x55bb4a204860 2023-08-25T09:00:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb637fe700 1 -- 172.21.15.99:0/2247531832 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffb7c004470 con 0x55bb4a204860 2023-08-25T09:00:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 10 client.15303 fetch_fsmap finished waiting for FSMap version 24 2023-08-25T09:00:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:00:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 20 client.15303 populate_metadata read hostname 'smithi099' 2023-08-25T09:00:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bb4a26c5c0 0x55bb4a26e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 4 client.15303 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:00:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.848+0000 7ffb9e080b80 1 -- 172.21.15.99:0/2247531832 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bb4a149910 con 0x55bb4a26c5c0 2023-08-25T09:00:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.849+0000 7ffb83fff700 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bb4a26c5c0 0x55bb4a26e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.849+0000 7ffb83fff700 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bb4a26c5c0 0x55bb4a26e9e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53c63054a243dc7 server_cookie=fc9ab5b82fb9c614 in_seq=0 out_seq=0 2023-08-25T09:00:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:05.849+0000 7ffb637fe700 10 client.15303 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:05.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22238 12.40 MB/sec warmup 11 sec latency 2559.039 ms 2023-08-25T09:00:06.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22238 11.37 MB/sec warmup 12 sec latency 3559.159 ms 2023-08-25T09:00:07.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:07 smithi099 ceph-mon[97641]: pgmap v31: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.2 MiB/s wr, 240 op/s 2023-08-25T09:00:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:07 smithi157 ceph-mon[130879]: pgmap v31: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.2 MiB/s wr, 240 op/s 2023-08-25T09:00:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:07 smithi181 ceph-mon[131296]: pgmap v31: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.2 MiB/s wr, 240 op/s 2023-08-25T09:00:07.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22304 10.51 MB/sec warmup 13 sec latency 4486.147 ms 2023-08-25T09:00:08.884 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22304 9.76 MB/sec warmup 14 sec latency 1031.697 ms 2023-08-25T09:00:09.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:09 smithi099 ceph-mon[97641]: pgmap v32: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-08-25T09:00:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:09 smithi157 ceph-mon[130879]: pgmap v32: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-08-25T09:00:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:09 smithi181 ceph-mon[131296]: pgmap v32: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-08-25T09:00:09.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22304 9.10 MB/sec warmup 15 sec latency 2031.839 ms 2023-08-25T09:00:10.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22304 8.54 MB/sec warmup 16 sec latency 3031.949 ms 2023-08-25T09:00:11.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:11 smithi099 ceph-mon[97641]: pgmap v33: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-08-25T09:00:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:11 smithi157 ceph-mon[130879]: pgmap v33: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-08-25T09:00:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:11 smithi181 ceph-mon[131296]: pgmap v33: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-08-25T09:00:11.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22304 8.03 MB/sec warmup 17 sec latency 4032.090 ms 2023-08-25T09:00:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.840+0000 7ffb637fe700 1 -- 172.21.15.99:0/2247531832 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bb4a149910 con 0x55bb4a26c5c0 2023-08-25T09:00:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.840+0000 7ffb637fe700 10 client.15303 handle_command_reply: tid=0 2023-08-25T09:00:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.840+0000 7ffb9e080b80 1 client.15303 shutdown 2023-08-25T09:00:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.840+0000 7ffb9e080b80 1 -- 172.21.15.99:0/2247531832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 msgr2=0x55bb4a20b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.840+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a20b700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c017fd0 tx=0x7ffb7c0093c0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.840+0000 7ffb9e080b80 1 -- 172.21.15.99:0/2247531832 shutdown_connections 2023-08-25T09:00:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb4a204860 0x55bb4a20b700 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb4a1fe990 0x55bb4a20f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bb4a26c5c0 0x55bb4a26e9e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb9e080b80 1 --2- 172.21.15.99:0/2247531832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb4a203f40 0x55bb4a20b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb637fe700 0 client.15303 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb9e080b80 1 -- 172.21.15.99:0/2247531832 >> 172.21.15.99:0/2247531832 conn(0x55bb4a173080 msgr2=0x55bb4a204c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb9e080b80 1 -- 172.21.15.99:0/2247531832 shutdown_connections 2023-08-25T09:00:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb9e080b80 1 -- 172.21.15.99:0/2247531832 wait complete. 2023-08-25T09:00:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.841+0000 7ffb9e080b80 20 client.15303 trim_cache size 0 max 16384 2023-08-25T09:00:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:00:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:00:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:00:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:00:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.842+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ffb6c0a01e0 msgr2=0x7ffb6c0a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.842+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ffb6c0a01e0 0x7ffb6c0a26a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0075c0 tx=0x7ffb7c009040 comp rx=0 tx=0).stop 2023-08-25T09:00:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.842+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/3318117964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb84105320 msgr2=0x7ffb840fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.842+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb84105320 0x7ffb840fff80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800eb30 tx=0x7ffb7800ee40 comp rx=0 tx=0).stop 2023-08-25T09:00:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.843+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/3318117964 shutdown_connections 2023-08-25T09:00:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.843+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7ffb6c0a01e0 0x7ffb6c0a26a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.843+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb84105ce0 0x7ffb841a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.843+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84069150 0x7ffb840ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.843+0000 7ffb8a7ff700 1 --2- 172.21.15.99:0/3318117964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb84105320 0x7ffb840fff80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.843+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/3318117964 >> 172.21.15.99:0/3318117964 conn(0x7ffb84076920 msgr2=0x7ffb840fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.843+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/3318117964 shutdown_connections 2023-08-25T09:00:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:12.843+0000 7ffb8a7ff700 1 -- 172.21.15.99:0/3318117964 wait complete. 2023-08-25T09:00:12.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 22417 7.65 MB/sec warmup 18 sec latency 4965.168 ms 2023-08-25T09:00:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:13 smithi099 ceph-mon[97641]: pgmap v34: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 218 op/s 2023-08-25T09:00:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:13 smithi157 ceph-mon[130879]: pgmap v34: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 218 op/s 2023-08-25T09:00:13.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:13 smithi181 ceph-mon[131296]: pgmap v34: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 218 op/s 2023-08-25T09:00:13.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 24944 8.07 MB/sec warmup 19 sec latency 11.296 ms 2023-08-25T09:00:14.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 25665 7.76 MB/sec warmup 20 sec latency 302.480 ms 2023-08-25T09:00:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:15 smithi157 ceph-mon[130879]: pgmap v35: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 394 KiB/s wr, 106 op/s 2023-08-25T09:00:15.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:15 smithi181 ceph-mon[131296]: pgmap v35: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 394 KiB/s wr, 106 op/s 2023-08-25T09:00:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:15 smithi099 ceph-mon[97641]: pgmap v35: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 394 KiB/s wr, 106 op/s 2023-08-25T09:00:15.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 25665 7.39 MB/sec warmup 21 sec latency 1302.587 ms 2023-08-25T09:00:16.885 INFO:tasks.workunit.client.0.smithi099.stdout: 1 25665 7.06 MB/sec warmup 22 sec latency 2302.682 ms 2023-08-25T09:00:17.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:17 smithi181 ceph-mon[131296]: pgmap v36: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 743 KiB/s wr, 143 op/s 2023-08-25T09:00:17.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:17 smithi099 ceph-mon[97641]: pgmap v36: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 743 KiB/s wr, 143 op/s 2023-08-25T09:00:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:17 smithi157 ceph-mon[130879]: pgmap v36: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 743 KiB/s wr, 143 op/s 2023-08-25T09:00:17.886 INFO:tasks.workunit.client.0.smithi099.stdout: 1 25787 6.76 MB/sec warmup 23 sec latency 3227.823 ms 2023-08-25T09:00:18.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:18 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:00:18.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:18 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:00:18.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:18 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:00:18.886 INFO:tasks.workunit.client.0.smithi099.stdout: 1 28075 7.01 MB/sec warmup 24 sec latency 16.512 ms 2023-08-25T09:00:19.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:19 smithi099 ceph-mon[97641]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 374 KiB/s wr, 42 op/s 2023-08-25T09:00:19.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:19 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:00:19.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:19 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:00:19.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:19 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:00:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:19 smithi157 ceph-mon[130879]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 374 KiB/s wr, 42 op/s 2023-08-25T09:00:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:19 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:00:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:19 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:00:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:19 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:00:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:19 smithi181 ceph-mon[131296]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 374 KiB/s wr, 42 op/s 2023-08-25T09:00:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:19 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:00:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:19 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:00:19.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:19 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:00:19.886 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29139 6.97 MB/sec warmup 25 sec latency 177.711 ms 2023-08-25T09:00:20.886 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29139 6.70 MB/sec warmup 26 sec latency 1177.773 ms 2023-08-25T09:00:21.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:21 smithi099 ceph-mon[97641]: pgmap v38: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 374 KiB/s wr, 42 op/s 2023-08-25T09:00:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:21 smithi157 ceph-mon[130879]: pgmap v38: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 374 KiB/s wr, 42 op/s 2023-08-25T09:00:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:21 smithi181 ceph-mon[131296]: pgmap v38: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 374 KiB/s wr, 42 op/s 2023-08-25T09:00:21.886 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29139 6.45 MB/sec warmup 27 sec latency 2177.903 ms 2023-08-25T09:00:22.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29207 6.23 MB/sec warmup 28 sec latency 3103.559 ms 2023-08-25T09:00:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:23 smithi099 ceph-mon[97641]: pgmap v39: 97 pgs: 97 active+clean; 15 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 573 KiB/s wr, 80 op/s 2023-08-25T09:00:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:23 smithi157 ceph-mon[130879]: pgmap v39: 97 pgs: 97 active+clean; 15 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 573 KiB/s wr, 80 op/s 2023-08-25T09:00:23.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:23 smithi181 ceph-mon[131296]: pgmap v39: 97 pgs: 97 active+clean; 15 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 573 KiB/s wr, 80 op/s 2023-08-25T09:00:23.888 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29207 6.01 MB/sec warmup 29 sec latency 1025.555 ms 2023-08-25T09:00:24.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29207 5.81 MB/sec warmup 30 sec latency 2025.695 ms 2023-08-25T09:00:25.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:25 smithi157 ceph-mon[130879]: pgmap v40: 97 pgs: 97 active+clean; 15 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 548 KiB/s wr, 75 op/s 2023-08-25T09:00:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:25 smithi181 ceph-mon[131296]: pgmap v40: 97 pgs: 97 active+clean; 15 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 548 KiB/s wr, 75 op/s 2023-08-25T09:00:25.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:25 smithi099 ceph-mon[97641]: pgmap v40: 97 pgs: 97 active+clean; 15 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 548 KiB/s wr, 75 op/s 2023-08-25T09:00:25.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29207 5.62 MB/sec warmup 31 sec latency 3025.805 ms 2023-08-25T09:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.990+0000 7f35bcbff700 1 -- 172.21.15.99:0/127374320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b810d160 msgr2=0x7f35b810f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.990+0000 7f35bcbff700 1 --2- 172.21.15.99:0/127374320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b810d160 0x7f35b810f590 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35b000b0a0 tx=0x7f35b000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:00:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.990+0000 7f35bcbff700 1 -- 172.21.15.99:0/127374320 shutdown_connections 2023-08-25T09:00:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.990+0000 7f35bcbff700 1 --2- 172.21.15.99:0/127374320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b810d160 0x7f35b810f590 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.990+0000 7f35bcbff700 1 --2- 172.21.15.99:0/127374320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b8072cf0 0x7f35b80730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.990+0000 7f35bcbff700 1 --2- 172.21.15.99:0/127374320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b8073610 0x7f35b810cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.990+0000 7f35bcbff700 1 -- 172.21.15.99:0/127374320 >> 172.21.15.99:0/127374320 conn(0x7f35b80fc3d0 msgr2=0x7f35b80fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.991+0000 7f35bcbff700 1 -- 172.21.15.99:0/127374320 shutdown_connections 2023-08-25T09:00:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.991+0000 7f35bcbff700 1 -- 172.21.15.99:0/127374320 wait complete. 2023-08-25T09:00:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35bcbff700 1 Processor -- start 2023-08-25T09:00:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35bcbff700 1 -- start start 2023-08-25T09:00:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35bcbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b8072cf0 0x7f35b8198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35bcbff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b8073610 0x7f35b8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35bcbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b810d160 0x7f35b819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35bcbff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819df40 con 0x7f35b8073610 2023-08-25T09:00:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35bcbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819e0b0 con 0x7f35b810d160 2023-08-25T09:00:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35bcbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819e270 con 0x7f35b8072cf0 2023-08-25T09:00:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35b659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b8072cf0 0x7f35b8198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35b659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b8072cf0 0x7f35b8198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:46544/0 (socket says 172.21.15.99:46544) 2023-08-25T09:00:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35b659c700 1 -- 172.21.15.99:0/2845301279 learned_addr learned my addr 172.21.15.99:0/2845301279 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b8073610 0x7f35b8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.992+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b810d160 0x7f35b819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35b659c700 1 -- 172.21.15.99:0/2845301279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b810d160 msgr2=0x7f35b819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35b659c700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b810d160 0x7f35b819d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35b659c700 1 -- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b8073610 msgr2=0x7f35b8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35b659c700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b8073610 0x7f35b8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35b659c700 1 -- 172.21.15.99:0/2845301279 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0009d00 con 0x7f35b8072cf0 2023-08-25T09:00:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35b659c700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b8072cf0 0x7f35b8198d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35a800e9a0 tx=0x7f35a800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35a77fe700 1 -- 172.21.15.99:0/2845301279 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8009790 con 0x7f35b8072cf0 2023-08-25T09:00:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35bcbff700 1 -- 172.21.15.99:0/2845301279 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b819e550 con 0x7f35b8072cf0 2023-08-25T09:00:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35a77fe700 1 -- 172.21.15.99:0/2845301279 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a8010430 con 0x7f35b8072cf0 2023-08-25T09:00:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35a77fe700 1 -- 172.21.15.99:0/2845301279 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8018680 con 0x7f35b8072cf0 2023-08-25T09:00:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.993+0000 7f35bcbff700 1 -- 172.21.15.99:0/2845301279 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b81a5d90 con 0x7f35b8072cf0 2023-08-25T09:00:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.996+0000 7f35a77fe700 1 -- 172.21.15.99:0/2845301279 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) v1 ==== 134904+0+0 (secure 0 0 0) 0x7f35a80105a0 con 0x7f35b8072cf0 2023-08-25T09:00:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35d0547b80 1 Processor -- start 2023-08-25T09:00:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35d0547b80 1 -- start start 2023-08-25T09:00:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35a77fe700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f35a00a0260 0x7f35a00a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35a77fe700 1 -- 172.21.15.99:0/2845301279 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35a8014070 con 0x7f35b8072cf0 2023-08-25T09:00:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67eed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ee7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f35a00a0260 0x7f35a00a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ee7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35d0547b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67e4f750 con 0x55fe67eed860 2023-08-25T09:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35d0547b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67e40720 con 0x55fe67ee7990 2023-08-25T09:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35d0547b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67e3e040 con 0x55fe67eecf40 2023-08-25T09:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.998+0000 7f35b5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ee7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ee7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57648/0 (socket says 172.21.15.99:57648) 2023-08-25T09:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b5d9b700 1 -- 172.21.15.99:0/2823880603 learned_addr learned my addr 172.21.15.99:0/2823880603 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b659c700 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67eed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ee7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b5d9b700 1 -- 172.21.15.99:0/2823880603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 msgr2=0x55fe67eed320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67eed320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b5d9b700 1 -- 172.21.15.99:0/2823880603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 msgr2=0x55fe67ee7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ee7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f35a00a0260 0x7f35a00a2720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35b000b780 tx=0x7f35b00095a0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b5d9b700 1 -- 172.21.15.99:0/2823880603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe67cb95d0 con 0x55fe67eed860 2023-08-25T09:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ee7450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009ca0 tx=0x7f35ac011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bc1a6b8be5f0a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2823880603 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac011cc0 con 0x55fe67eed860 2023-08-25T09:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2823880603 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35ac011e20 con 0x55fe67eed860 2023-08-25T09:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:25.999+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2823880603 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac019430 con 0x55fe67eed860 2023-08-25T09:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 -- 172.21.15.99:0/2823880603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 msgr2=0x55fe67ee7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ee7450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009ca0 tx=0x7f35ac011440 comp rx=0 tx=0).stop 2023-08-25T09:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 -- 172.21.15.99:0/2823880603 shutdown_connections 2023-08-25T09:00:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ee7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67eed320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 --2- 172.21.15.99:0/2823880603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ee7450 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 -- 172.21.15.99:0/2823880603 >> 172.21.15.99:0/2823880603 conn(0x55fe67e5c080 msgr2=0x55fe67ee1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 -- 172.21.15.99:0/2823880603 shutdown_connections 2023-08-25T09:00:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.000+0000 7f35d0547b80 1 -- 172.21.15.99:0/2823880603 wait complete. 2023-08-25T09:00:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.001+0000 7f35d0547b80 1 -- start start 2023-08-25T09:00:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.001+0000 7f35d0547b80 1 Processor -- start 2023-08-25T09:00:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67ee7990 0x55fe67ef3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67ef0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ef0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35d0547b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67e39930 con 0x55fe67eed860 2023-08-25T09:00:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35d0547b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67dd7b50 con 0x55fe67eecf40 2023-08-25T09:00:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35d0547b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67dd7850 con 0x55fe67ee7990 2023-08-25T09:00:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35d0547b80 10 client.0 _ll_register_callbacks cb 0x55fe67d21a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:00:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35b6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ef0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35b6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ef0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57654/0 (socket says 172.21.15.99:57654) 2023-08-25T09:00:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35b5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67ef0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35b6d9d700 1 -- 172.21.15.99:0/1295584939 learned_addr learned my addr 172.21.15.99:0/1295584939 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.002+0000 7f35b659c700 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67ee7990 0x55fe67ef3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35b6d9d700 1 -- 172.21.15.99:0/1295584939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67ee7990 msgr2=0x55fe67ef3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67ee7990 0x55fe67ef3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35b6d9d700 1 -- 172.21.15.99:0/1295584939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 msgr2=0x55fe67ef0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67ef0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35b6d9d700 1 -- 172.21.15.99:0/1295584939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac009900 con 0x55fe67eed860 2023-08-25T09:00:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35b659c700 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67ee7990 0x55fe67ef3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67ef0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ef0ec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f35b0015000 tx=0x7f35b000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d6949d00154c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f359dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:00:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f359dffb700 1 -- 172.21.15.99:0/1295584939 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b000e040 con 0x55fe67eed860 2023-08-25T09:00:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35d0547b80 1 -- 172.21.15.99:0/1295584939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe67eb7160 con 0x55fe67eed860 2023-08-25T09:00:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f359dffb700 1 -- 172.21.15.99:0/1295584939 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35b001e930 con 0x55fe67eed860 2023-08-25T09:00:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f359dffb700 1 -- 172.21.15.99:0/1295584939 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35b001eaf0 con 0x55fe67eed860 2023-08-25T09:00:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35d0547b80 10 client.15315 fetch_fsmap learned FSMap version 25 2023-08-25T09:00:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.003+0000 7f35d0547b80 1 -- 172.21.15.99:0/1295584939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fe67c65d80 con 0x55fe67eed860 2023-08-25T09:00:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f359dffb700 1 -- 172.21.15.99:0/1295584939 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe67eb7160 con 0x55fe67eed860 2023-08-25T09:00:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f359dffb700 1 -- 172.21.15.99:0/1295584939 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f35b001c430 con 0x55fe67eed860 2023-08-25T09:00:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f35d0547b80 10 client.15315 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:00:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f35d0547b80 10 client.15315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:00:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f35d0547b80 20 client.15315 populate_metadata read hostname 'smithi099' 2023-08-25T09:00:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fe67f01250 0x55fe67f03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f35d0547b80 4 client.15315 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:00:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f35d0547b80 1 -- 172.21.15.99:0/1295584939 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe67ddfb40 con 0x55fe67f01250 2023-08-25T09:00:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.004+0000 7f35b659c700 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fe67f01250 0x55fe67f03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.005+0000 7f359dffb700 10 client.15315 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.005+0000 7f35b659c700 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fe67f01250 0x55fe67f03670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306ee2a93e0ab4d4 server_cookie=3af038a8766f59f6 in_seq=0 out_seq=0 2023-08-25T09:00:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.005+0000 7f359dffb700 1 -- 172.21.15.99:0/1295584939 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe67ddfb40 con 0x55fe67f01250 2023-08-25T09:00:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.005+0000 7f359dffb700 10 client.15315 handle_command_reply: tid=0 2023-08-25T09:00:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 client.15315 shutdown 2023-08-25T09:00:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 -- 172.21.15.99:0/1295584939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 msgr2=0x55fe67ef0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ef0ec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f35b0015000 tx=0x7f35b000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 -- 172.21.15.99:0/1295584939 shutdown_connections 2023-08-25T09:00:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67ef0ec0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67ee7990 0x55fe67ef3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fe67f01250 0x55fe67f03670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1295584939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67ef0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 -- 172.21.15.99:0/1295584939 >> 172.21.15.99:0/1295584939 conn(0x55fe67e5c080 msgr2=0x55fe67edc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f359dffb700 0 client.15315 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 -- 172.21.15.99:0/1295584939 shutdown_connections 2023-08-25T09:00:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 1 -- 172.21.15.99:0/1295584939 wait complete. 2023-08-25T09:00:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.006+0000 7f35d0547b80 20 client.15315 trim_cache size 0 max 16384 2023-08-25T09:00:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.021+0000 7f35d0547b80 1 Processor -- start 2023-08-25T09:00:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.021+0000 7f35d0547b80 1 -- start start 2023-08-25T09:00:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67ee7990 0x55fe67eeaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67eedc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eed860 0x55fe67eee180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35d0547b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67cbb8c0 con 0x55fe67ee7990 2023-08-25T09:00:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35d0547b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67ef4030 con 0x55fe67eecf40 2023-08-25T09:00:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35d0547b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67ede960 con 0x55fe67eed860 2023-08-25T09:00:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67ee7990 0x55fe67eeaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67ee7990 0x55fe67eeaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57668/0 (socket says 172.21.15.99:57668) 2023-08-25T09:00:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 -- 172.21.15.99:0/3143227953 learned_addr learned my addr 172.21.15.99:0/3143227953 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eed860 0x55fe67eee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67eedc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 -- 172.21.15.99:0/3143227953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eed860 msgr2=0x55fe67eee180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eed860 0x55fe67eee180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 -- 172.21.15.99:0/3143227953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 msgr2=0x55fe67eedc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67eedc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 -- 172.21.15.99:0/3143227953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe67c4ec00 con 0x55fe67ee7990 2023-08-25T09:00:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.022+0000 7f35b659c700 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67ee7990 0x55fe67eeaaf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f35a8095400 tx=0x7f35a8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363cb25ebcd2e765 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35a57fa700 1 -- 172.21.15.99:0/3143227953 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a809cd30 con 0x55fe67ee7990 2023-08-25T09:00:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35a57fa700 1 -- 172.21.15.99:0/3143227953 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a80a0d10 con 0x55fe67ee7990 2023-08-25T09:00:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35a57fa700 1 -- 172.21.15.99:0/3143227953 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a80b6a00 con 0x55fe67ee7990 2023-08-25T09:00:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 -- 172.21.15.99:0/3143227953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67ee7990 msgr2=0x55fe67eeaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67ee7990 0x55fe67eeaaf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f35a8095400 tx=0x7f35a8010c80 comp rx=0 tx=0).stop 2023-08-25T09:00:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 -- 172.21.15.99:0/3143227953 shutdown_connections 2023-08-25T09:00:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eed860 0x55fe67eee180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67ee7990 0x55fe67eeaaf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 --2- 172.21.15.99:0/3143227953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67eecf40 0x55fe67eedc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 -- 172.21.15.99:0/3143227953 >> 172.21.15.99:0/3143227953 conn(0x55fe67e5c080 msgr2=0x55fe67efd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 -- 172.21.15.99:0/3143227953 shutdown_connections 2023-08-25T09:00:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.023+0000 7f35d0547b80 1 -- 172.21.15.99:0/3143227953 wait complete. 2023-08-25T09:00:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 1 -- start start 2023-08-25T09:00:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 1 Processor -- start 2023-08-25T09:00:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ef6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67ef69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67f54080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67f545c0 con 0x55fe67eed860 2023-08-25T09:00:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67f54700 con 0x55fe67ee7990 2023-08-25T09:00:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67f54840 con 0x55fe67eecf40 2023-08-25T09:00:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.024+0000 7f35d0547b80 10 client.0 _ll_register_callbacks cb 0x7f35b0003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:00:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ef6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67f54080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67ef69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67f54080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57682/0 (socket says 172.21.15.99:57682) 2023-08-25T09:00:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ef6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:59522/0 (socket says 172.21.15.99:59522) 2023-08-25T09:00:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b6d9d700 1 -- 172.21.15.99:0/1467398834 learned_addr learned my addr 172.21.15.99:0/1467398834 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b659c700 1 -- 172.21.15.99:0/1467398834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 msgr2=0x55fe67ef69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b659c700 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67ef69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b659c700 1 -- 172.21.15.99:0/1467398834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 msgr2=0x55fe67f54080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b659c700 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67f54080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b659c700 1 -- 172.21.15.99:0/1467398834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b81a5d90 con 0x55fe67ee7990 2023-08-25T09:00:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b6d9d700 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67f54080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67ef69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f35b659c700 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ef6480 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c510 tx=0x7f35a8010bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c971498b3364c24c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.025+0000 7f359dffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:00:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f359dffb700 1 -- 172.21.15.99:0/1467398834 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8010880 con 0x55fe67ee7990 2023-08-25T09:00:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 1 -- 172.21.15.99:0/1467398834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe67eda600 con 0x55fe67ee7990 2023-08-25T09:00:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f359dffb700 1 -- 172.21.15.99:0/1467398834 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35a80a0880 con 0x55fe67ee7990 2023-08-25T09:00:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f359dffb700 1 -- 172.21.15.99:0/1467398834 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a80109e0 con 0x55fe67ee7990 2023-08-25T09:00:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 10 client.25376 fetch_fsmap learned FSMap version 25 2023-08-25T09:00:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 1 -- 172.21.15.99:0/1467398834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fe67c8c1f0 con 0x55fe67ee7990 2023-08-25T09:00:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f359dffb700 1 -- 172.21.15.99:0/1467398834 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a807b020 con 0x55fe67ee7990 2023-08-25T09:00:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f359dffb700 1 -- 172.21.15.99:0/1467398834 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f35a80c92f0 con 0x55fe67ee7990 2023-08-25T09:00:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 10 client.25376 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:00:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 10 client.25376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:00:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 20 client.25376 populate_metadata read hostname 'smithi099' 2023-08-25T09:00:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fe67f55d20 0x55fe67f58140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 4 client.25376 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:00:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.026+0000 7f35d0547b80 1 -- 172.21.15.99:0/1467398834 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe67e32910 con 0x55fe67f55d20 2023-08-25T09:00:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.027+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fe67f55d20 0x55fe67f58140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.027+0000 7f359dffb700 10 client.25376 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.027+0000 7f35b5d9b700 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fe67f55d20 0x55fe67f58140 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc5ab0a1fa9d56ef server_cookie=d965272e1a4d4cc7 in_seq=0 out_seq=0 2023-08-25T09:00:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f359dffb700 1 -- 172.21.15.99:0/1467398834 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe67e32910 con 0x55fe67f55d20 2023-08-25T09:00:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f359dffb700 10 client.25376 handle_command_reply: tid=0 2023-08-25T09:00:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 client.25376 shutdown 2023-08-25T09:00:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 -- 172.21.15.99:0/1467398834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 msgr2=0x55fe67ef6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ef6480 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c510 tx=0x7f35a8010bf0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 -- 172.21.15.99:0/1467398834 shutdown_connections 2023-08-25T09:00:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe67eed860 0x55fe67f54080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fe67f55d20 0x55fe67f58140 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe67ee7990 0x55fe67ef6480 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 --2- 172.21.15.99:0/1467398834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe67eecf40 0x55fe67ef69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 -- 172.21.15.99:0/1467398834 >> 172.21.15.99:0/1467398834 conn(0x55fe67e5c080 msgr2=0x55fe67efd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f359dffb700 0 client.25376 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 -- 172.21.15.99:0/1467398834 shutdown_connections 2023-08-25T09:00:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 1 -- 172.21.15.99:0/1467398834 wait complete. 2023-08-25T09:00:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.065+0000 7f35d0547b80 20 client.25376 trim_cache size 0 max 16384 2023-08-25T09:00:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:00:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:00:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:00:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:00:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 -- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f35a00a0260 msgr2=0x7f35a00a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f35a00a0260 0x7f35a00a2720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35b000b780 tx=0x7f35b00095a0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 -- 172.21.15.99:0/2845301279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b8072cf0 msgr2=0x7f35b8198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b8072cf0 0x7f35b8198d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35a800e9a0 tx=0x7f35a800ecb0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 -- 172.21.15.99:0/2845301279 shutdown_connections 2023-08-25T09:00:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f35a00a0260 0x7f35a00a2720 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b810d160 0x7f35b819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b8072cf0 0x7f35b8198d50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 --2- 172.21.15.99:0/2845301279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b8073610 0x7f35b8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 -- 172.21.15.99:0/2845301279 >> 172.21.15.99:0/2845301279 conn(0x7f35b80fc3d0 msgr2=0x7f35b810f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 -- 172.21.15.99:0/2845301279 shutdown_connections 2023-08-25T09:00:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:26.066+0000 7f35bcbff700 1 -- 172.21.15.99:0/2845301279 wait complete. 2023-08-25T09:00:26.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:26 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:00:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:26 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:00:26.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:26 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:00:26.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29207 5.45 MB/sec warmup 32 sec latency 4025.891 ms 2023-08-25T09:00:27.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:27 smithi157 ceph-mon[130879]: pgmap v41: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 103 op/s 2023-08-25T09:00:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:27 smithi181 ceph-mon[131296]: pgmap v41: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 103 op/s 2023-08-25T09:00:27.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:27 smithi099 ceph-mon[97641]: pgmap v41: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 103 op/s 2023-08-25T09:00:27.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 29329 5.29 MB/sec warmup 33 sec latency 4950.899 ms 2023-08-25T09:00:28.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30753 5.32 MB/sec warmup 34 sec latency 476.206 ms 2023-08-25T09:00:29.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:29 smithi099 ceph-mon[97641]: pgmap v42: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T09:00:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:29 smithi157 ceph-mon[130879]: pgmap v42: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T09:00:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:29 smithi181 ceph-mon[131296]: pgmap v42: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T09:00:29.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30807 5.17 MB/sec warmup 35 sec latency 1051.363 ms 2023-08-25T09:00:30.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30807 5.03 MB/sec warmup 36 sec latency 1389.970 ms 2023-08-25T09:00:31.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:31 smithi099 ceph-mon[97641]: pgmap v43: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T09:00:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:31 smithi157 ceph-mon[130879]: pgmap v43: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T09:00:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:31 smithi181 ceph-mon[131296]: pgmap v43: 97 pgs: 97 active+clean; 30 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T09:00:31.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30807 4.89 MB/sec warmup 37 sec latency 2390.101 ms 2023-08-25T09:00:32.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30858 4.77 MB/sec warmup 38 sec latency 3297.470 ms 2023-08-25T09:00:33.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:33 smithi099 ceph-mon[97641]: pgmap v44: 97 pgs: 97 active+clean; 39 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 100 op/s 2023-08-25T09:00:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:33 smithi157 ceph-mon[130879]: pgmap v44: 97 pgs: 97 active+clean; 39 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 100 op/s 2023-08-25T09:00:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:33 smithi181 ceph-mon[131296]: pgmap v44: 97 pgs: 97 active+clean; 39 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 100 op/s 2023-08-25T09:00:33.887 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30872 4.64 MB/sec warmup 39 sec latency 978.410 ms 2023-08-25T09:00:34.888 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30976 4.55 MB/sec warmup 40 sec latency 1553.182 ms 2023-08-25T09:00:35.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:35 smithi157 ceph-mon[130879]: pgmap v45: 97 pgs: 97 active+clean; 39 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.7 MiB/s wr, 62 op/s 2023-08-25T09:00:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:35 smithi181 ceph-mon[131296]: pgmap v45: 97 pgs: 97 active+clean; 39 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.7 MiB/s wr, 62 op/s 2023-08-25T09:00:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:35 smithi099 ceph-mon[97641]: pgmap v45: 97 pgs: 97 active+clean; 39 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.7 MiB/s wr, 62 op/s 2023-08-25T09:00:35.889 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30976 4.44 MB/sec warmup 41 sec latency 1370.046 ms 2023-08-25T09:00:36.888 INFO:tasks.workunit.client.0.smithi099.stdout: 1 30976 4.33 MB/sec warmup 42 sec latency 2370.118 ms 2023-08-25T09:00:37.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:37 smithi099 ceph-mon[97641]: pgmap v46: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.0 MiB/s wr, 91 op/s 2023-08-25T09:00:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:37 smithi157 ceph-mon[130879]: pgmap v46: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.0 MiB/s wr, 91 op/s 2023-08-25T09:00:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:37 smithi181 ceph-mon[131296]: pgmap v46: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.0 MiB/s wr, 91 op/s 2023-08-25T09:00:37.888 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31013 4.23 MB/sec warmup 43 sec latency 3269.894 ms 2023-08-25T09:00:38.888 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31013 4.14 MB/sec warmup 44 sec latency 1063.990 ms 2023-08-25T09:00:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 -- 172.21.15.99:0/4122342985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c101200 msgr2=0x7f3c2c10d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 --2- 172.21.15.99:0/4122342985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c101200 0x7f3c2c10d710 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009f90 tx=0x7f3c2400b330 comp rx=0 tx=0).stop 2023-08-25T09:00:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 -- 172.21.15.99:0/4122342985 shutdown_connections 2023-08-25T09:00:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 --2- 172.21.15.99:0/4122342985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c101200 0x7f3c2c10d710 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 --2- 172.21.15.99:0/4122342985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c106850 0x7f3c2c106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 --2- 172.21.15.99:0/4122342985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c100840 0x7f3c2c100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 -- 172.21.15.99:0/4122342985 >> 172.21.15.99:0/4122342985 conn(0x7f3c2c0fc310 msgr2=0x7f3c2c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 -- 172.21.15.99:0/4122342985 shutdown_connections 2023-08-25T09:00:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.222+0000 7f3c33fff700 1 -- 172.21.15.99:0/4122342985 wait complete. 2023-08-25T09:00:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.223+0000 7f3c33fff700 1 Processor -- start 2023-08-25T09:00:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c33fff700 1 -- start start 2023-08-25T09:00:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c33fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c100840 0x7f3c2c072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c33fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c101200 0x7f3c2c073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c106850 0x7f3c2c1a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c33fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c073990 con 0x7f3c2c100840 2023-08-25T09:00:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c33fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c1a4fc0 con 0x7f3c2c101200 2023-08-25T09:00:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c33fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c1a5100 con 0x7f3c2c106850 2023-08-25T09:00:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c31d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c100840 0x7f3c2c072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c31d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c100840 0x7f3c2c072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56088/0 (socket says 172.21.15.99:56088) 2023-08-25T09:00:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c31d9b700 1 -- 172.21.15.99:0/2292871483 learned_addr learned my addr 172.21.15.99:0/2292871483 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c3159a700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c101200 0x7f3c2c073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c3259c700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c106850 0x7f3c2c1a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.224+0000 7f3c31d9b700 1 -- 172.21.15.99:0/2292871483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c106850 msgr2=0x7f3c2c1a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c106850 0x7f3c2c1a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c31d9b700 1 -- 172.21.15.99:0/2292871483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c101200 msgr2=0x7f3c2c073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c101200 0x7f3c2c073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c31d9b700 1 -- 172.21.15.99:0/2292871483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24009c40 con 0x7f3c2c100840 2023-08-25T09:00:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c3159a700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c101200 0x7f3c2c073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:00:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c100840 0x7f3c2c072c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e9c0 tx=0x7f3c2800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c22ffd700 1 -- 172.21.15.99:0/2292871483 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800cbd0 con 0x7f3c2c100840 2023-08-25T09:00:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c22ffd700 1 -- 172.21.15.99:0/2292871483 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c2800cd30 con 0x7f3c2c100840 2023-08-25T09:00:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c33fff700 1 -- 172.21.15.99:0/2292871483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c1a5360 con 0x7f3c2c100840 2023-08-25T09:00:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c22ffd700 1 -- 172.21.15.99:0/2292871483 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28005580 con 0x7f3c2c100840 2023-08-25T09:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.225+0000 7f3c33fff700 1 -- 172.21.15.99:0/2292871483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c1a5840 con 0x7f3c2c100840 2023-08-25T09:00:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.228+0000 7f3c22ffd700 1 -- 172.21.15.99:0/2292871483 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) v1 ==== 134904+0+0 (secure 0 0 0) 0x7f3c28004750 con 0x7f3c2c100840 2023-08-25T09:00:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.229+0000 7f3c478b1b80 1 Processor -- start 2023-08-25T09:00:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c22ffd700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3c180a01e0 0x7f3c180a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c22ffd700 1 -- 172.21.15.99:0/2292871483 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c28014070 con 0x7f3c2c100840 2023-08-25T09:00:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c478b1b80 1 -- start start 2023-08-25T09:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c3259c700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3c180a01e0 0x7f3c180a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a72f750 con 0x55e96a7cd860 2023-08-25T09:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a720720 con 0x55e96a7ccf40 2023-08-25T09:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a71e040 con 0x55e96a7c7990 2023-08-25T09:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c31d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.230+0000 7f3c31d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48514/0 (socket says 172.21.15.99:48514) 2023-08-25T09:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c31d9b700 1 -- 172.21.15.99:0/2030236315 learned_addr learned my addr 172.21.15.99:0/2030236315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c3159a700 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c3259c700 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c31d9b700 1 -- 172.21.15.99:0/2030236315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 msgr2=0x55e96a7c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c3259c700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3c180a01e0 0x7f3c180a26a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009790 tx=0x7f3c240096c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c31d9b700 1 -- 172.21.15.99:0/2030236315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 msgr2=0x55e96a7c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c31d9b700 1 -- 172.21.15.99:0/2030236315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e96a5995d0 con 0x55e96a7ccf40 2023-08-25T09:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7cd320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c280a08e0 tx=0x7f3c280989e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c73f3c02f5a8e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c17fff700 1 -- 172.21.15.99:0/2030236315 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28099000 con 0x55e96a7ccf40 2023-08-25T09:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c17fff700 1 -- 172.21.15.99:0/2030236315 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c2809ce30 con 0x55e96a7ccf40 2023-08-25T09:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.231+0000 7f3c17fff700 1 -- 172.21.15.99:0/2030236315 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c280994c0 con 0x55e96a7ccf40 2023-08-25T09:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2030236315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 msgr2=0x55e96a7cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7cd320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c280a08e0 tx=0x7f3c280989e0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2030236315 shutdown_connections 2023-08-25T09:00:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7cd320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/2030236315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2030236315 >> 172.21.15.99:0/2030236315 conn(0x55e96a73c080 msgr2=0x55e96a7c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2030236315 shutdown_connections 2023-08-25T09:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.232+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2030236315 wait complete. 2023-08-25T09:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.233+0000 7f3c478b1b80 1 -- start start 2023-08-25T09:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.233+0000 7f3c478b1b80 1 Processor -- start 2023-08-25T09:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7d36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a719930 con 0x55e96a7cd860 2023-08-25T09:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a6b7b50 con 0x55e96a7ccf40 2023-08-25T09:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a6b7850 con 0x55e96a7c7990 2023-08-25T09:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c478b1b80 10 client.0 _ll_register_callbacks cb 0x55e96a601a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c3159a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c3159a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48526/0 (socket says 172.21.15.99:48526) 2023-08-25T09:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c3159a700 1 -- 172.21.15.99:0/1322792874 learned_addr learned my addr 172.21.15.99:0/1322792874 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c3259c700 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.234+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7d36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c3159a700 1 -- 172.21.15.99:0/1322792874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 msgr2=0x55e96a7d36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c3159a700 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7d36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c3159a700 1 -- 172.21.15.99:0/1322792874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 msgr2=0x55e96a7d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c3159a700 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c3159a700 1 -- 172.21.15.99:0/1322792874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c1a5840 con 0x55e96a7ccf40 2023-08-25T09:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c3259c700 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c3159a700 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7d09b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c012720 tx=0x7f3c1c012a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d9fd4fdda31eb76 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c157fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c157fa700 1 -- 172.21.15.99:0/1322792874 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c017070 con 0x55e96a7ccf40 2023-08-25T09:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c478b1b80 1 -- 172.21.15.99:0/1322792874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e96a797160 con 0x55e96a7ccf40 2023-08-25T09:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c157fa700 1 -- 172.21.15.99:0/1322792874 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c1c011770 con 0x55e96a7ccf40 2023-08-25T09:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c157fa700 1 -- 172.21.15.99:0/1322792874 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c1c011930 con 0x55e96a7ccf40 2023-08-25T09:00:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c478b1b80 10 client.25388 fetch_fsmap learned FSMap version 25 2023-08-25T09:00:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.235+0000 7f3c478b1b80 1 -- 172.21.15.99:0/1322792874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e96a545d80 con 0x55e96a7ccf40 2023-08-25T09:00:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c157fa700 1 -- 172.21.15.99:0/1322792874 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e96a797160 con 0x55e96a7ccf40 2023-08-25T09:00:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c157fa700 1 -- 172.21.15.99:0/1322792874 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3c1c00fba0 con 0x55e96a7ccf40 2023-08-25T09:00:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c478b1b80 10 client.25388 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:00:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c478b1b80 10 client.25388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:00:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c478b1b80 20 client.25388 populate_metadata read hostname 'smithi099' 2023-08-25T09:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e96a7e12b0 0x55e96a7e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c478b1b80 4 client.25388 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c478b1b80 1 -- 172.21.15.99:0/1322792874 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e96a6bfb40 con 0x55e96a7e12b0 2023-08-25T09:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.236+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e96a7e12b0 0x55e96a7e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.237+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e96a7e12b0 0x55e96a7e36d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fa82b7f94ef8ca server_cookie=86cd4fcc87310c6e in_seq=0 out_seq=0 2023-08-25T09:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.237+0000 7f3c157fa700 10 client.25388 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.237+0000 7f3c157fa700 1 -- 172.21.15.99:0/1322792874 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e96a6bfb40 con 0x55e96a7e12b0 2023-08-25T09:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.237+0000 7f3c157fa700 10 client.25388 handle_command_reply: tid=0 2023-08-25T09:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 client.25388 shutdown 2023-08-25T09:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 -- 172.21.15.99:0/1322792874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 msgr2=0x55e96a7d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7d09b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c012720 tx=0x7f3c1c012a30 comp rx=0 tx=0).stop 2023-08-25T09:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 -- 172.21.15.99:0/1322792874 shutdown_connections 2023-08-25T09:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7d36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e96a7e12b0 0x55e96a7e36d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/1322792874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7d09b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c157fa700 0 client.25388 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 -- 172.21.15.99:0/1322792874 >> 172.21.15.99:0/1322792874 conn(0x55e96a73c080 msgr2=0x55e96a7bc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 -- 172.21.15.99:0/1322792874 shutdown_connections 2023-08-25T09:00:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 1 -- 172.21.15.99:0/1322792874 wait complete. 2023-08-25T09:00:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.238+0000 7f3c478b1b80 20 client.25388 trim_cache size 0 max 16384 2023-08-25T09:00:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.254+0000 7f3c478b1b80 1 Processor -- start 2023-08-25T09:00:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.254+0000 7f3c478b1b80 1 -- start start 2023-08-25T09:00:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.254+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.254+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.254+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.254+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a59b8c0 con 0x55e96a7cd860 2023-08-25T09:00:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.254+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a7c2ac0 con 0x55e96a7ccf40 2023-08-25T09:00:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.254+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a7c3770 con 0x55e96a7c7990 2023-08-25T09:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3159a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56122/0 (socket says 172.21.15.99:56122) 2023-08-25T09:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3259c700 1 -- 172.21.15.99:0/2528854159 learned_addr learned my addr 172.21.15.99:0/2528854159 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3159a700 1 -- 172.21.15.99:0/2528854159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 msgr2=0x55e96a7d8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3159a700 1 --2- 172.21.15.99:0/2528854159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7d8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3159a700 1 -- 172.21.15.99:0/2528854159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 msgr2=0x55e96a7cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3159a700 1 --2- 172.21.15.99:0/2528854159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3159a700 1 -- 172.21.15.99:0/2528854159 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e96a7bbb40 con 0x55e96a7ccf40 2023-08-25T09:00:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c3159a700 1 --2- 172.21.15.99:0/2528854159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7dd480 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00ff80 tx=0x7f3c1c009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dd72c603be895a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c20ff9700 1 -- 172.21.15.99:0/2528854159 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c017040 con 0x55e96a7ccf40 2023-08-25T09:00:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c20ff9700 1 -- 172.21.15.99:0/2528854159 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c1c0056f0 con 0x55e96a7ccf40 2023-08-25T09:00:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.255+0000 7f3c20ff9700 1 -- 172.21.15.99:0/2528854159 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0038e0 con 0x55e96a7ccf40 2023-08-25T09:00:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2528854159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 msgr2=0x55e96a7dd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/2528854159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7dd480 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00ff80 tx=0x7f3c1c009350 comp rx=0 tx=0).stop 2023-08-25T09:00:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2528854159 shutdown_connections 2023-08-25T09:00:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/2528854159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/2528854159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7c7990 0x55e96a7d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/2528854159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7ccf40 0x55e96a7dd480 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2528854159 >> 172.21.15.99:0/2528854159 conn(0x55e96a73c080 msgr2=0x55e96a7c1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2528854159 shutdown_connections 2023-08-25T09:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.256+0000 7f3c478b1b80 1 -- 172.21.15.99:0/2528854159 wait complete. 2023-08-25T09:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 1 -- start start 2023-08-25T09:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 1 Processor -- start 2023-08-25T09:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7c7990 0x55e96a7c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7ccf40 0x55e96a7ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a7c8dc0 con 0x55e96a7cd860 2023-08-25T09:00:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a7c8f00 con 0x55e96a7c7990 2023-08-25T09:00:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96a7c90c0 con 0x55e96a7ccf40 2023-08-25T09:00:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.257+0000 7f3c478b1b80 10 client.0 _ll_register_callbacks cb 0x7f3c1c009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:00:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7ccf40 0x55e96a7ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c31d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7c7990 0x55e96a7c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7ccf40 0x55e96a7ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:35382/0 (socket says 172.21.15.99:35382) 2023-08-25T09:00:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 -- 172.21.15.99:0/657575667 learned_addr learned my addr 172.21.15.99:0/657575667 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 -- 172.21.15.99:0/657575667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7c7990 msgr2=0x55e96a7c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7c7990 0x55e96a7c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 -- 172.21.15.99:0/657575667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 msgr2=0x55e96a7c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 -- 172.21.15.99:0/657575667 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e96a545d80 con 0x55e96a7ccf40 2023-08-25T09:00:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7c7990 0x55e96a7c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c3159a700 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7ccf40 0x55e96a7ca1b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0126f0 tx=0x7f3c1c011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38b3f092f81f80f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.258+0000 7f3c157fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:00:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c157fa700 1 -- 172.21.15.99:0/657575667 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c017070 con 0x55e96a7ccf40 2023-08-25T09:00:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c478b1b80 1 -- 172.21.15.99:0/657575667 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e96a7ba600 con 0x55e96a7ccf40 2023-08-25T09:00:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c157fa700 1 -- 172.21.15.99:0/657575667 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c1c005280 con 0x55e96a7ccf40 2023-08-25T09:00:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c157fa700 1 -- 172.21.15.99:0/657575667 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c1c005480 con 0x55e96a7ccf40 2023-08-25T09:00:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c478b1b80 10 client.25378 fetch_fsmap learned FSMap version 25 2023-08-25T09:00:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c478b1b80 1 -- 172.21.15.99:0/657575667 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e96a52ec00 con 0x55e96a7ccf40 2023-08-25T09:00:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c157fa700 1 -- 172.21.15.99:0/657575667 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e96a7ba600 con 0x55e96a7ccf40 2023-08-25T09:00:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c157fa700 1 -- 172.21.15.99:0/657575667 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3c1c01c8e0 con 0x55e96a7ccf40 2023-08-25T09:00:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c478b1b80 10 client.25378 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:00:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c478b1b80 10 client.25378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:00:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.259+0000 7f3c478b1b80 20 client.25378 populate_metadata read hostname 'smithi099' 2023-08-25T09:00:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.260+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e96a835e60 0x55e96a838280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.260+0000 7f3c478b1b80 4 client.25378 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:00:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.260+0000 7f3c478b1b80 1 -- 172.21.15.99:0/657575667 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e96a712910 con 0x55e96a835e60 2023-08-25T09:00:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.260+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e96a835e60 0x55e96a838280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.260+0000 7f3c31d9b700 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e96a835e60 0x55e96a838280 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b820acfdf5b9db server_cookie=6f1a5e1a9bca075 in_seq=0 out_seq=0 2023-08-25T09:00:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.260+0000 7f3c157fa700 10 client.25378 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.303+0000 7f3c157fa700 1 -- 172.21.15.99:0/657575667 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e96a712910 con 0x55e96a835e60 2023-08-25T09:00:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.303+0000 7f3c157fa700 10 client.25378 handle_command_reply: tid=0 2023-08-25T09:00:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.303+0000 7f3c478b1b80 1 client.25378 shutdown 2023-08-25T09:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.303+0000 7f3c478b1b80 1 -- 172.21.15.99:0/657575667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7ccf40 msgr2=0x55e96a7ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.303+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7ccf40 0x55e96a7ca1b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0126f0 tx=0x7f3c1c011d30 comp rx=0 tx=0).stop 2023-08-25T09:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 1 -- 172.21.15.99:0/657575667 shutdown_connections 2023-08-25T09:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e96a7cd860 0x55e96a7c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e96a835e60 0x55e96a838280 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e96a7c7990 0x55e96a7c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 1 --2- 172.21.15.99:0/657575667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e96a7ccf40 0x55e96a7ca1b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c157fa700 0 client.25378 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 1 -- 172.21.15.99:0/657575667 >> 172.21.15.99:0/657575667 conn(0x55e96a73c080 msgr2=0x55e96a7dd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 1 -- 172.21.15.99:0/657575667 shutdown_connections 2023-08-25T09:00:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 1 -- 172.21.15.99:0/657575667 wait complete. 2023-08-25T09:00:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c478b1b80 20 client.25378 trim_cache size 0 max 16384 2023-08-25T09:00:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:00:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:00:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:00:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:00:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c33fff700 1 -- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3c180a01e0 msgr2=0x7f3c180a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.304+0000 7f3c33fff700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3c180a01e0 0x7f3c180a26a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009790 tx=0x7f3c240096c0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 -- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c100840 msgr2=0x7f3c2c072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c100840 0x7f3c2c072c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e9c0 tx=0x7f3c2800ed80 comp rx=0 tx=0).stop 2023-08-25T09:00:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 -- 172.21.15.99:0/2292871483 shutdown_connections 2023-08-25T09:00:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3c180a01e0 0x7f3c180a26a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c106850 0x7f3c2c1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c2c100840 0x7f3c2c072c20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 --2- 172.21.15.99:0/2292871483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c101200 0x7f3c2c073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 -- 172.21.15.99:0/2292871483 >> 172.21.15.99:0/2292871483 conn(0x7f3c2c0fc310 msgr2=0x7f3c2c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 -- 172.21.15.99:0/2292871483 shutdown_connections 2023-08-25T09:00:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:39.305+0000 7f3c33fff700 1 -- 172.21.15.99:0/2292871483 wait complete. 2023-08-25T09:00:39.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:39 smithi099 ceph-mon[97641]: pgmap v47: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2023-08-25T09:00:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:39 smithi157 ceph-mon[130879]: pgmap v47: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2023-08-25T09:00:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:39 smithi181 ceph-mon[131296]: pgmap v47: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2023-08-25T09:00:39.888 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31176 4.11 MB/sec warmup 45 sec latency 1450.419 ms 2023-08-25T09:00:40.888 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31294 4.03 MB/sec warmup 46 sec latency 1025.530 ms 2023-08-25T09:00:41.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:41 smithi099 ceph-mon[97641]: pgmap v48: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2023-08-25T09:00:41.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:41 smithi157 ceph-mon[130879]: pgmap v48: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2023-08-25T09:00:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:41 smithi181 ceph-mon[131296]: pgmap v48: 97 pgs: 97 active+clean; 32 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2023-08-25T09:00:41.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31294 3.94 MB/sec warmup 47 sec latency 1496.597 ms 2023-08-25T09:00:42.889 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31345 3.89 MB/sec warmup 48 sec latency 2394.750 ms 2023-08-25T09:00:43.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:43 smithi099 ceph-mon[97641]: pgmap v49: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:00:43.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:43 smithi099 ceph-mon[97641]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-08-25T09:00:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:43 smithi157 ceph-mon[130879]: pgmap v49: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:00:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:43 smithi157 ceph-mon[130879]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-08-25T09:00:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:43 smithi181 ceph-mon[131296]: pgmap v49: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:00:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:43 smithi181 ceph-mon[131296]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-08-25T09:00:43.889 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31345 3.81 MB/sec warmup 49 sec latency 1058.450 ms 2023-08-25T09:00:44.888 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31345 3.73 MB/sec warmup 50 sec latency 2058.499 ms 2023-08-25T09:00:45.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:45 smithi157 ceph-mon[130879]: pgmap v50: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 379 KiB/s wr, 45 op/s 2023-08-25T09:00:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:45 smithi181 ceph-mon[131296]: pgmap v50: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 379 KiB/s wr, 45 op/s 2023-08-25T09:00:45.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:45 smithi099 ceph-mon[97641]: pgmap v50: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 379 KiB/s wr, 45 op/s 2023-08-25T09:00:45.889 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31345 3.66 MB/sec warmup 51 sec latency 3058.607 ms 2023-08-25T09:00:46.889 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31345 3.59 MB/sec warmup 52 sec latency 4058.750 ms 2023-08-25T09:00:47.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:47 smithi099 ceph-mon[97641]: pgmap v51: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 385 KiB/s wr, 50 op/s 2023-08-25T09:00:47.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:47 smithi157 ceph-mon[130879]: pgmap v51: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 385 KiB/s wr, 50 op/s 2023-08-25T09:00:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:47 smithi181 ceph-mon[131296]: pgmap v51: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 385 KiB/s wr, 50 op/s 2023-08-25T09:00:47.890 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31390 3.52 MB/sec warmup 53 sec latency 4962.934 ms 2023-08-25T09:00:48.889 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31390 3.45 MB/sec warmup 54 sec latency 1079.990 ms 2023-08-25T09:00:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:49 smithi099 ceph-mon[97641]: pgmap v52: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 21 op/s 2023-08-25T09:00:49.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:49 smithi157 ceph-mon[130879]: pgmap v52: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 21 op/s 2023-08-25T09:00:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:49 smithi181 ceph-mon[131296]: pgmap v52: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 21 op/s 2023-08-25T09:00:49.889 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31390 3.39 MB/sec warmup 55 sec latency 2080.124 ms 2023-08-25T09:00:50.889 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31390 3.33 MB/sec warmup 56 sec latency 3080.296 ms 2023-08-25T09:00:51.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:51 smithi099 ceph-mon[97641]: pgmap v53: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 21 op/s 2023-08-25T09:00:51.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:51 smithi157 ceph-mon[130879]: pgmap v53: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 21 op/s 2023-08-25T09:00:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:51 smithi181 ceph-mon[131296]: pgmap v53: 97 pgs: 97 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 21 op/s 2023-08-25T09:00:51.890 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31390 3.27 MB/sec warmup 57 sec latency 4080.459 ms 2023-08-25T09:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 -- 172.21.15.99:0/1087478237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 msgr2=0x7f09fc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 --2- 172.21.15.99:0/1087478237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 0x7f09fc10f550 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f09f400b0a0 tx=0x7f09f400b3b0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 -- 172.21.15.99:0/1087478237 shutdown_connections 2023-08-25T09:00:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 --2- 172.21.15.99:0/1087478237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 0x7f09fc10f550 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 --2- 172.21.15.99:0/1087478237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc072d50 0x7f09fc073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 --2- 172.21.15.99:0/1087478237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09fc073670 0x7f09fc10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 -- 172.21.15.99:0/1087478237 >> 172.21.15.99:0/1087478237 conn(0x7f09fc0fc370 msgr2=0x7f09fc0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 -- 172.21.15.99:0/1087478237 shutdown_connections 2023-08-25T09:00:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.459+0000 7f0a043ff700 1 -- 172.21.15.99:0/1087478237 wait complete. 2023-08-25T09:00:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.460+0000 7f0a043ff700 1 Processor -- start 2023-08-25T09:00:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a043ff700 1 -- start start 2023-08-25T09:00:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a043ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09fc072d50 0x7f09fc19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a043ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc073670 0x7f09fc19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a043ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 0x7f09fc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a043ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc199130 con 0x7f09fc10d0e0 2023-08-25T09:00:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a043ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1992a0 con 0x7f09fc072d50 2023-08-25T09:00:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a043ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc199460 con 0x7f09fc073670 2023-08-25T09:00:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a0219b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09fc072d50 0x7f09fc19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a0219b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09fc072d50 0x7f09fc19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:56644/0 (socket says 172.21.15.99:56644) 2023-08-25T09:00:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a0299c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 0x7f09fc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a0219b700 1 -- 172.21.15.99:0/2297801482 learned_addr learned my addr 172.21.15.99:0/2297801482 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.461+0000 7f0a0199a700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc073670 0x7f09fc19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a0299c700 1 -- 172.21.15.99:0/2297801482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc073670 msgr2=0x7f09fc19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a0299c700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc073670 0x7f09fc19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a0299c700 1 -- 172.21.15.99:0/2297801482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09fc072d50 msgr2=0x7f09fc19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a0299c700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09fc072d50 0x7f09fc19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a0299c700 1 -- 172.21.15.99:0/2297801482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f4009d00 con 0x7f09fc10d0e0 2023-08-25T09:00:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a0219b700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09fc072d50 0x7f09fc19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a0299c700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 0x7f09fc198ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f09f400b0a0 tx=0x7f09f4007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f09f37fe700 1 -- 172.21.15.99:0/2297801482 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400e040 con 0x7f09fc10d0e0 2023-08-25T09:00:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f09f37fe700 1 -- 172.21.15.99:0/2297801482 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f401d470 con 0x7f09fc10d0e0 2023-08-25T09:00:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a043ff700 1 -- 172.21.15.99:0/2297801482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc1996e0 con 0x7f09fc10d0e0 2023-08-25T09:00:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f09f37fe700 1 -- 172.21.15.99:0/2297801482 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f40052a0 con 0x7f09fc10d0e0 2023-08-25T09:00:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.462+0000 7f0a043ff700 1 -- 172.21.15.99:0/2297801482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc1a5c50 con 0x7f09fc10d0e0 2023-08-25T09:00:52.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.466+0000 7f09f37fe700 1 -- 172.21.15.99:0/2297801482 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f09f401d5e0 con 0x7f09fc10d0e0 2023-08-25T09:00:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.467+0000 7f09f37fe700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f09e80a00c0 0x7f09e80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.467+0000 7f0a17c7fb80 1 Processor -- start 2023-08-25T09:00:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.467+0000 7f0a17c7fb80 1 -- start start 2023-08-25T09:00:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.467+0000 7f09f37fe700 1 -- 172.21.15.99:0/2297801482 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f09f40cc9b0 con 0x7f09fc10d0e0 2023-08-25T09:00:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.467+0000 7f0a0219b700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f09e80a00c0 0x7f09e80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x55706439f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706439f860 0x557064399450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557064399990 0x557064399dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557064301750 con 0x55706439ef40 2023-08-25T09:00:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570642f2720 con 0x55706439f860 2023-08-25T09:00:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570642f0040 con 0x557064399990 2023-08-25T09:00:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0219b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x55706439f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0299c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557064399990 0x557064399dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0199a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706439f860 0x557064399450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0299c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557064399990 0x557064399dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:36370/0 (socket says 172.21.15.99:36370) 2023-08-25T09:00:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0299c700 1 -- 172.21.15.99:0/4181258655 learned_addr learned my addr 172.21.15.99:0/4181258655 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0219b700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f09e80a00c0 0x7f09e80a2580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f09f8013650 tx=0x7f09f8013a10 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0299c700 1 -- 172.21.15.99:0/4181258655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706439f860 msgr2=0x557064399450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0299c700 1 --2- 172.21.15.99:0/4181258655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706439f860 0x557064399450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0299c700 1 -- 172.21.15.99:0/4181258655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 msgr2=0x55706439f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0299c700 1 --2- 172.21.15.99:0/4181258655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x55706439f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.468+0000 7f0a0299c700 1 -- 172.21.15.99:0/4181258655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706416b5d0 con 0x557064399990 2023-08-25T09:00:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.469+0000 7f0a0299c700 1 --2- 172.21.15.99:0/4181258655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557064399990 0x557064399dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09f4096b80 tx=0x7f09f4009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d823f9ef634f3bed server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.469+0000 7f09f0bfe700 1 -- 172.21.15.99:0/4181258655 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f409a810 con 0x557064399990 2023-08-25T09:00:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.469+0000 7f09f0bfe700 1 -- 172.21.15.99:0/4181258655 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f409a110 con 0x557064399990 2023-08-25T09:00:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.469+0000 7f09f0bfe700 1 -- 172.21.15.99:0/4181258655 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4096820 con 0x557064399990 2023-08-25T09:00:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.469+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/4181258655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557064399990 msgr2=0x557064399dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.469+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/4181258655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557064399990 0x557064399dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09f4096b80 tx=0x7f09f4009c90 comp rx=0 tx=0).stop 2023-08-25T09:00:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/4181258655 shutdown_connections 2023-08-25T09:00:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/4181258655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557064399990 0x557064399dd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/4181258655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x55706439f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/4181258655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706439f860 0x557064399450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/4181258655 >> 172.21.15.99:0/4181258655 conn(0x55706430e080 msgr2=0x557064393d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/4181258655 shutdown_connections 2023-08-25T09:00:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/4181258655 wait complete. 2023-08-25T09:00:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 -- start start 2023-08-25T09:00:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.470+0000 7f0a17c7fb80 1 Processor -- start 2023-08-25T09:00:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557064399990 0x5570643a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x5570643a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439f860 0x5570643a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570642eb930 con 0x55706439ef40 2023-08-25T09:00:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557064289b50 con 0x557064399990 2023-08-25T09:00:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557064289850 con 0x55706439f860 2023-08-25T09:00:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a17c7fb80 10 client.0 _ll_register_callbacks cb 0x5570641d3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:00:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0199a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x5570643a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0299c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439f860 0x5570643a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0299c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439f860 0x5570643a2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:36382/0 (socket says 172.21.15.99:36382) 2023-08-25T09:00:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0199a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x5570643a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53842/0 (socket says 172.21.15.99:53842) 2023-08-25T09:00:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0299c700 1 -- 172.21.15.99:0/3866977445 learned_addr learned my addr 172.21.15.99:0/3866977445 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0219b700 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557064399990 0x5570643a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0299c700 1 -- 172.21.15.99:0/3866977445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557064399990 msgr2=0x5570643a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0299c700 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557064399990 0x5570643a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0299c700 1 -- 172.21.15.99:0/3866977445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 msgr2=0x5570643a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0299c700 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x5570643a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.471+0000 7f0a0299c700 1 -- 172.21.15.99:0/3866977445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc1a5c50 con 0x55706439f860 2023-08-25T09:00:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f0a0299c700 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439f860 0x5570643a2ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09f4006130 tx=0x7f09f401db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e2684384cf6455 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f09e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:00:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f09e5ffb700 1 -- 172.21.15.99:0/3866977445 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f409a110 con 0x55706439f860 2023-08-25T09:00:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f09e5ffb700 1 -- 172.21.15.99:0/3866977445 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f09f40a1e90 con 0x55706439f860 2023-08-25T09:00:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/3866977445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557064369160 con 0x55706439f860 2023-08-25T09:00:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f09e5ffb700 1 -- 172.21.15.99:0/3866977445 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09f409df40 con 0x55706439f860 2023-08-25T09:00:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f0a17c7fb80 10 client.25390 fetch_fsmap learned FSMap version 25 2023-08-25T09:00:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/3866977445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557064117d80 con 0x55706439f860 2023-08-25T09:00:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.472+0000 7f09e5ffb700 1 -- 172.21.15.99:0/3866977445 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557064369160 con 0x55706439f860 2023-08-25T09:00:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.473+0000 7f09e5ffb700 1 -- 172.21.15.99:0/3866977445 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f09f4099600 con 0x55706439f860 2023-08-25T09:00:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.473+0000 7f0a17c7fb80 10 client.25390 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:00:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.473+0000 7f0a17c7fb80 10 client.25390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:00:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.473+0000 7f0a17c7fb80 20 client.25390 populate_metadata read hostname 'smithi099' 2023-08-25T09:00:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.473+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570643b2b60 0x5570643b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.473+0000 7f0a17c7fb80 4 client.25390 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:00:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.473+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/3866977445 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557064291b40 con 0x5570643b2b60 2023-08-25T09:00:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.473+0000 7f0a0219b700 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570643b2b60 0x5570643b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.474+0000 7f09e5ffb700 10 client.25390 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.474+0000 7f0a0219b700 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570643b2b60 0x5570643b4f80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a318b538ca07ad server_cookie=5816a7d48e6fa1e1 in_seq=0 out_seq=0 2023-08-25T09:00:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.474+0000 7f09e5ffb700 1 -- 172.21.15.99:0/3866977445 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557064291b40 con 0x5570643b2b60 2023-08-25T09:00:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.474+0000 7f09e5ffb700 10 client.25390 handle_command_reply: tid=0 2023-08-25T09:00:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 client.25390 shutdown 2023-08-25T09:00:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/3866977445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439f860 msgr2=0x5570643a2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439f860 0x5570643a2ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09f4006130 tx=0x7f09f401db10 comp rx=0 tx=0).stop 2023-08-25T09:00:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/3866977445 shutdown_connections 2023-08-25T09:00:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439f860 0x5570643a2ef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570643b2b60 0x5570643b4f80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557064399990 0x5570643a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/3866977445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55706439ef40 0x5570643a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f09e5ffb700 0 client.25390 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/3866977445 >> 172.21.15.99:0/3866977445 conn(0x55706430e080 msgr2=0x55706438e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/3866977445 shutdown_connections 2023-08-25T09:00:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/3866977445 wait complete. 2023-08-25T09:00:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.475+0000 7f0a17c7fb80 20 client.25390 trim_cache size 0 max 16384 2023-08-25T09:00:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.490+0000 7f0a17c7fb80 1 Processor -- start 2023-08-25T09:00:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.490+0000 7f0a17c7fb80 1 -- start start 2023-08-25T09:00:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.490+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x557064399450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.490+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x5570643aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.490+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x5570643a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.490+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706416d8c0 con 0x557064399990 2023-08-25T09:00:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.490+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557064390560 con 0x55706430e080 2023-08-25T09:00:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.490+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557064394860 con 0x55706439ef40 2023-08-25T09:00:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0199a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x5570643aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x557064399450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x557064399450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:56676/0 (socket says 172.21.15.99:56676) 2023-08-25T09:00:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 -- 172.21.15.99:0/489046657 learned_addr learned my addr 172.21.15.99:0/489046657 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0299c700 1 --2- 172.21.15.99:0/489046657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x5570643a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 -- 172.21.15.99:0/489046657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 msgr2=0x5570643a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 --2- 172.21.15.99:0/489046657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x5570643a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 -- 172.21.15.99:0/489046657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 msgr2=0x5570643aa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 --2- 172.21.15.99:0/489046657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x5570643aa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 -- 172.21.15.99:0/489046657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557064100c00 con 0x55706430e080 2023-08-25T09:00:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.491+0000 7f0a0219b700 1 --2- 172.21.15.99:0/489046657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x557064399450 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09f80096b0 tx=0x7f09f801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14aae55d6ced7d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f09f13ff700 1 -- 172.21.15.99:0/489046657 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8023040 con 0x55706430e080 2023-08-25T09:00:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f09f13ff700 1 -- 172.21.15.99:0/489046657 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f8028430 con 0x55706430e080 2023-08-25T09:00:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f09f13ff700 1 -- 172.21.15.99:0/489046657 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8018070 con 0x55706430e080 2023-08-25T09:00:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/489046657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 msgr2=0x557064399450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/489046657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x557064399450 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09f80096b0 tx=0x7f09f801d000 comp rx=0 tx=0).stop 2023-08-25T09:00:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/489046657 shutdown_connections 2023-08-25T09:00:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/489046657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x5570643a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/489046657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x557064399450 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/489046657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x5570643aa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/489046657 >> 172.21.15.99:0/489046657 conn(0x5570643a2480 msgr2=0x557064393cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/489046657 shutdown_connections 2023-08-25T09:00:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.492+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/489046657 wait complete. 2023-08-25T09:00:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.493+0000 7f0a17c7fb80 1 -- start start 2023-08-25T09:00:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.493+0000 7f0a17c7fb80 1 Processor -- start 2023-08-25T09:00:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.493+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x55706439bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.493+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x55706439a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.493+0000 7f0a17c7fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x55706439adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706439c5b0 con 0x557064399990 2023-08-25T09:00:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706439b300 con 0x55706430e080 2023-08-25T09:00:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a17c7fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706439b4c0 con 0x55706439ef40 2023-08-25T09:00:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a17c7fb80 10 client.0 _ll_register_callbacks cb 0x5570641d3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:00:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0199a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x55706439a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0299c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x55706439adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0199a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x55706439a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53856/0 (socket says 172.21.15.99:53856) 2023-08-25T09:00:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0199a700 1 -- 172.21.15.99:0/1332817596 learned_addr learned my addr 172.21.15.99:0/1332817596 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:00:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0219b700 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x55706439bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0299c700 1 -- 172.21.15.99:0/1332817596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 msgr2=0x55706439bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0299c700 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x55706439bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0299c700 1 -- 172.21.15.99:0/1332817596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 msgr2=0x55706439a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:00:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0299c700 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x55706439a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0299c700 1 -- 172.21.15.99:0/1332817596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f80120b0 con 0x55706439ef40 2023-08-25T09:00:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0199a700 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x55706439a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:00:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f0a0299c700 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x55706439adc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09f409a0a0 tx=0x7f09f40ca190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393925d8c8e3b9b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:00:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.494+0000 7f09e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:00:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f09e5ffb700 1 -- 172.21.15.99:0/1332817596 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f40a1c50 con 0x55706439ef40 2023-08-25T09:00:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/1332817596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55706438c600 con 0x55706439ef40 2023-08-25T09:00:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f09e5ffb700 1 -- 172.21.15.99:0/1332817596 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f09f40a1e90 con 0x55706439ef40 2023-08-25T09:00:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f09e5ffb700 1 -- 172.21.15.99:0/1332817596 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09f409df40 con 0x55706439ef40 2023-08-25T09:00:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 10 client.25396 fetch_fsmap learned FSMap version 25 2023-08-25T09:00:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/1332817596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55706413e1f0 con 0x55706439ef40 2023-08-25T09:00:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f09e5ffb700 1 -- 172.21.15.99:0/1332817596 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55706438c600 con 0x55706439ef40 2023-08-25T09:00:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f09e5ffb700 1 -- 172.21.15.99:0/1332817596 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f09f40a8fa0 con 0x55706439ef40 2023-08-25T09:00:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 10 client.25396 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:00:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:00:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 20 client.25396 populate_metadata read hostname 'smithi099' 2023-08-25T09:00:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55706439f860 0x557064409770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:00:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 4 client.25396 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:00:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.495+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/1332817596 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5570642e4910 con 0x55706439f860 2023-08-25T09:00:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.496+0000 7f0a0219b700 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55706439f860 0x557064409770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:00:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.496+0000 7f09e5ffb700 10 client.25396 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.496+0000 7f0a0219b700 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55706439f860 0x557064409770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e3a663e1f33ea3a server_cookie=50dd48091bd4c725 in_seq=0 out_seq=0 2023-08-25T09:00:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f09e5ffb700 1 -- 172.21.15.99:0/1332817596 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5570642e4910 con 0x55706439f860 2023-08-25T09:00:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f09e5ffb700 10 client.25396 handle_command_reply: tid=0 2023-08-25T09:00:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 client.25396 shutdown 2023-08-25T09:00:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/1332817596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 msgr2=0x55706439adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x55706439adc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09f409a0a0 tx=0x7f09f40ca190 comp rx=0 tx=0).stop 2023-08-25T09:00:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/1332817596 shutdown_connections 2023-08-25T09:00:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55706439ef40 0x55706439adc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55706439f860 0x557064409770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55706430e080 0x55706439bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f09e5ffb700 0 client.25396 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:00:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 --2- 172.21.15.99:0/1332817596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557064399990 0x55706439a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/1332817596 >> 172.21.15.99:0/1332817596 conn(0x5570643a2480 msgr2=0x55706439ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/1332817596 shutdown_connections 2023-08-25T09:00:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 1 -- 172.21.15.99:0/1332817596 wait complete. 2023-08-25T09:00:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a17c7fb80 20 client.25396 trim_cache size 0 max 16384 2023-08-25T09:00:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:00:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:00:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:00:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:00:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 -- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f09e80a00c0 msgr2=0x7f09e80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f09e80a00c0 0x7f09e80a2580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f09f8013650 tx=0x7f09f8013a10 comp rx=0 tx=0).stop 2023-08-25T09:00:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 -- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 msgr2=0x7f09fc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:00:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 0x7f09fc198ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f09f400b0a0 tx=0x7f09f4007990 comp rx=0 tx=0).stop 2023-08-25T09:00:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 -- 172.21.15.99:0/2297801482 shutdown_connections 2023-08-25T09:00:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f09e80a00c0 0x7f09e80a2580 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09fc10d0e0 0x7f09fc198ac0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09fc072d50 0x7f09fc19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 --2- 172.21.15.99:0/2297801482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc073670 0x7f09fc19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:00:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.533+0000 7f0a043ff700 1 -- 172.21.15.99:0/2297801482 >> 172.21.15.99:0/2297801482 conn(0x7f09fc0fc370 msgr2=0x7f09fc104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:00:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.534+0000 7f0a043ff700 1 -- 172.21.15.99:0/2297801482 shutdown_connections 2023-08-25T09:00:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:00:52.534+0000 7f0a043ff700 1 -- 172.21.15.99:0/2297801482 wait complete. 2023-08-25T09:00:52.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 31768 3.25 MB/sec warmup 58 sec latency 4695.730 ms 2023-08-25T09:00:53.571 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:53 smithi099 ceph-mon[97641]: pgmap v54: 97 pgs: 97 active+clean; 28 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 287 KiB/s wr, 36 op/s 2023-08-25T09:00:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:53 smithi157 ceph-mon[130879]: pgmap v54: 97 pgs: 97 active+clean; 28 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 287 KiB/s wr, 36 op/s 2023-08-25T09:00:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:53 smithi181 ceph-mon[131296]: pgmap v54: 97 pgs: 97 active+clean; 28 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 287 KiB/s wr, 36 op/s 2023-08-25T09:00:53.890 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32631 3.27 MB/sec warmup 59 sec latency 236.560 ms 2023-08-25T09:00:54.890 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32631 3.22 MB/sec warmup 60 sec latency 1236.664 ms 2023-08-25T09:00:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:55 smithi157 ceph-mon[130879]: pgmap v55: 97 pgs: 97 active+clean; 28 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-08-25T09:00:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:55 smithi181 ceph-mon[131296]: pgmap v55: 97 pgs: 97 active+clean; 28 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-08-25T09:00:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:55 smithi099 ceph-mon[97641]: pgmap v55: 97 pgs: 97 active+clean; 28 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-08-25T09:00:55.890 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32631 3.17 MB/sec warmup 61 sec latency 2236.756 ms 2023-08-25T09:00:56.890 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32687 3.12 MB/sec warmup 62 sec latency 2864.880 ms 2023-08-25T09:00:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:57 smithi099 ceph-mon[97641]: pgmap v56: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 577 KiB/s wr, 52 op/s 2023-08-25T09:00:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:57 smithi157 ceph-mon[130879]: pgmap v56: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 577 KiB/s wr, 52 op/s 2023-08-25T09:00:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:57 smithi181 ceph-mon[131296]: pgmap v56: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 577 KiB/s wr, 52 op/s 2023-08-25T09:00:57.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32755 3.07 MB/sec warmup 63 sec latency 1251.335 ms 2023-08-25T09:00:58.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32755 3.02 MB/sec warmup 64 sec latency 1029.883 ms 2023-08-25T09:00:59.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:00:59 smithi099 ceph-mon[97641]: pgmap v57: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 571 KiB/s wr, 47 op/s 2023-08-25T09:00:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:00:59 smithi157 ceph-mon[130879]: pgmap v57: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 571 KiB/s wr, 47 op/s 2023-08-25T09:00:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:00:59 smithi181 ceph-mon[131296]: pgmap v57: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 571 KiB/s wr, 47 op/s 2023-08-25T09:00:59.890 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32755 2.98 MB/sec warmup 65 sec latency 2030.051 ms 2023-08-25T09:01:00.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32765 2.93 MB/sec warmup 66 sec latency 3023.556 ms 2023-08-25T09:01:01.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:01 smithi099 ceph-mon[97641]: pgmap v58: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 571 KiB/s wr, 47 op/s 2023-08-25T09:01:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:01 smithi157 ceph-mon[130879]: pgmap v58: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 571 KiB/s wr, 47 op/s 2023-08-25T09:01:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:01 smithi181 ceph-mon[131296]: pgmap v58: 97 pgs: 97 active+clean; 29 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 571 KiB/s wr, 47 op/s 2023-08-25T09:01:01.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 34977 3.07 MB/sec warmup 67 sec latency 20.311 ms 2023-08-25T09:01:02.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 36297 3.13 MB/sec warmup 68 sec latency 31.226 ms 2023-08-25T09:01:03.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:03 smithi099 ceph-mon[97641]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 763 KiB/s wr, 75 op/s 2023-08-25T09:01:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:03 smithi157 ceph-mon[130879]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 763 KiB/s wr, 75 op/s 2023-08-25T09:01:03.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:03 smithi181 ceph-mon[131296]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 763 KiB/s wr, 75 op/s 2023-08-25T09:01:03.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 38779 3.28 MB/sec warmup 69 sec latency 12.800 ms 2023-08-25T09:01:04.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39722 3.31 MB/sec warmup 70 sec latency 219.546 ms 2023-08-25T09:01:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:05 smithi157 ceph-mon[130879]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 515 KiB/s wr, 61 op/s 2023-08-25T09:01:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:05 smithi181 ceph-mon[131296]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 515 KiB/s wr, 61 op/s 2023-08-25T09:01:05.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:05 smithi099 ceph-mon[97641]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 515 KiB/s wr, 61 op/s 2023-08-25T09:01:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 -- 172.21.15.99:0/3858648466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac100880 msgr2=0x7fc1ac100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/3858648466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac100880 0x7fc1ac100d00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009990 tx=0x7fc1a0009ca0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 -- 172.21.15.99:0/3858648466 shutdown_connections 2023-08-25T09:01:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/3858648466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac101240 0x7fc1ac10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/3858648466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1ac106890 0x7fc1ac106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/3858648466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac100880 0x7fc1ac100d00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 -- 172.21.15.99:0/3858648466 >> 172.21.15.99:0/3858648466 conn(0x7fc1ac0fc370 msgr2=0x7fc1ac0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 -- 172.21.15.99:0/3858648466 shutdown_connections 2023-08-25T09:01:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.688+0000 7fc1b1dff700 1 -- 172.21.15.99:0/3858648466 wait complete. 2023-08-25T09:01:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.689+0000 7fc1b1dff700 1 Processor -- start 2023-08-25T09:01:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1b1dff700 1 -- start start 2023-08-25T09:01:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1b1dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1ac100880 0x7fc1ac19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1b1dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac101240 0x7fc1ac19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1b1dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac106890 0x7fc1ac1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1b1dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac199030 con 0x7fc1ac101240 2023-08-25T09:01:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1b1dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1991a0 con 0x7fc1ac106890 2023-08-25T09:01:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1b1dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac199360 con 0x7fc1ac100880 2023-08-25T09:01:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1abfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac106890 0x7fc1ac1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac101240 0x7fc1ac19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1abfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac106890 0x7fc1ac1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:55128/0 (socket says 172.21.15.99:55128) 2023-08-25T09:01:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac101240 0x7fc1ac19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36400/0 (socket says 172.21.15.99:36400) 2023-08-25T09:01:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1abfff700 1 -- 172.21.15.99:0/214884645 learned_addr learned my addr 172.21.15.99:0/214884645 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.690+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1ac100880 0x7fc1ac19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1abfff700 1 -- 172.21.15.99:0/214884645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1ac100880 msgr2=0x7fc1ac19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1abfff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1ac100880 0x7fc1ac19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1abfff700 1 -- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac101240 msgr2=0x7fc1ac19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1abfff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac101240 0x7fc1ac19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1abfff700 1 -- 172.21.15.99:0/214884645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00095f0 con 0x7fc1ac106890 2023-08-25T09:01:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac101240 0x7fc1ac19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1abfff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac106890 0x7fc1ac1989c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400ef30 tx=0x7fc1a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1a8ff9700 1 -- 172.21.15.99:0/214884645 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4018070 con 0x7fc1ac106890 2023-08-25T09:01:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1b1dff700 1 -- 172.21.15.99:0/214884645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac1995e0 con 0x7fc1ac106890 2023-08-25T09:01:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1a8ff9700 1 -- 172.21.15.99:0/214884645 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a400f040 con 0x7fc1ac106890 2023-08-25T09:01:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1a8ff9700 1 -- 172.21.15.99:0/214884645 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400cd00 con 0x7fc1ac106890 2023-08-25T09:01:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.691+0000 7fc1b1dff700 1 -- 172.21.15.99:0/214884645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac1a5b50 con 0x7fc1ac106890 2023-08-25T09:01:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.694+0000 7fc1a8ff9700 1 -- 172.21.15.99:0/214884645 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fc1a4003a00 con 0x7fc1ac106890 2023-08-25T09:01:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.695+0000 7fc1c5729b80 1 Processor -- start 2023-08-25T09:01:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1c5729b80 1 -- start start 2023-08-25T09:01:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1a8ff9700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc1940a00c0 0x7fc1940a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc5767c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1a8ff9700 1 -- 172.21.15.99:0/214884645 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc1a40cb1f0 con 0x7fc1ac106890 2023-08-25T09:01:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57676450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57676dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc575de750 con 0x55bc57676990 2023-08-25T09:01:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc575cf720 con 0x55bc5767c860 2023-08-25T09:01:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc575cd040 con 0x55bc5767bf40 2023-08-25T09:01:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc1940a00c0 0x7fc1940a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1aaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57676450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc5767c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc5767c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34214/0 (socket says 172.21.15.99:34214) 2023-08-25T09:01:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/417798230 learned_addr learned my addr 172.21.15.99:0/417798230 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.696+0000 7fc1aaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57676450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:55136/0 (socket says 172.21.15.99:55136) 2023-08-25T09:01:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1abfff700 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57676dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/417798230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 msgr2=0x55bc57676450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57676450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/417798230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 msgr2=0x55bc57676dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57676dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/417798230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc574485d0 con 0x55bc5767bf40 2023-08-25T09:01:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57676450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc5767c320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009d00 tx=0x7fc19c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34eae5e6ff8c635a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.697+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc1940a00c0 0x7fc1940a2580 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b6f0 tx=0x7fc1a001a040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc191ffb700 1 -- 172.21.15.99:0/417798230 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c004230 con 0x55bc5767bf40 2023-08-25T09:01:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc191ffb700 1 -- 172.21.15.99:0/417798230 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c004390 con 0x55bc5767bf40 2023-08-25T09:01:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc191ffb700 1 -- 172.21.15.99:0/417798230 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c019710 con 0x55bc5767bf40 2023-08-25T09:01:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc1c5729b80 1 -- 172.21.15.99:0/417798230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 msgr2=0x55bc5767c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc5767c320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009d00 tx=0x7fc19c011720 comp rx=0 tx=0).stop 2023-08-25T09:01:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc1c5729b80 1 -- 172.21.15.99:0/417798230 shutdown_connections 2023-08-25T09:01:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57676dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc5767c320 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/417798230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57676450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.698+0000 7fc1c5729b80 1 -- 172.21.15.99:0/417798230 >> 172.21.15.99:0/417798230 conn(0x55bc575eb080 msgr2=0x55bc57670d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.699+0000 7fc1c5729b80 1 -- 172.21.15.99:0/417798230 shutdown_connections 2023-08-25T09:01:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.699+0000 7fc1c5729b80 1 -- 172.21.15.99:0/417798230 wait complete. 2023-08-25T09:01:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.699+0000 7fc1c5729b80 1 -- start start 2023-08-25T09:01:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.699+0000 7fc1c5729b80 1 Processor -- start 2023-08-25T09:01:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc57676990 0x55bc57682560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc5767f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5767c860 0x55bc5767fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc575c8930 con 0x55bc5767c860 2023-08-25T09:01:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc57566b50 con 0x55bc5767bf40 2023-08-25T09:01:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc57566850 con 0x55bc57676990 2023-08-25T09:01:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1c5729b80 10 client.0 _ll_register_callbacks cb 0x55bc574b0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1abfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5767c860 0x55bc5767fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc57676990 0x55bc57682560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1aaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc5767f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1abfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5767c860 0x55bc5767fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36422/0 (socket says 172.21.15.99:36422) 2023-08-25T09:01:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1abfff700 1 -- 172.21.15.99:0/1561778113 learned_addr learned my addr 172.21.15.99:0/1561778113 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1abfff700 1 -- 172.21.15.99:0/1561778113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc57676990 msgr2=0x55bc57682560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.700+0000 7fc1abfff700 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc57676990 0x55bc57682560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc1abfff700 1 -- 172.21.15.99:0/1561778113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 msgr2=0x55bc5767f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc1abfff700 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc5767f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc1abfff700 1 -- 172.21.15.99:0/1561778113 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a0020040 con 0x55bc5767c860 2023-08-25T09:01:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc1abfff700 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5767c860 0x55bc5767fd40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40090a0 tx=0x7fc1a4098510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719997dd6f1dfb4c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc18affd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:01:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc18affd700 1 -- 172.21.15.99:0/1561778113 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a401b320 con 0x55bc5767c860 2023-08-25T09:01:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc18affd700 1 -- 172.21.15.99:0/1561778113 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc1a4099050 con 0x55bc5767c860 2023-08-25T09:01:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc1c5729b80 1 -- 172.21.15.99:0/1561778113 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc57646160 con 0x55bc5767c860 2023-08-25T09:01:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc18affd700 1 -- 172.21.15.99:0/1561778113 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a400ce60 con 0x55bc5767c860 2023-08-25T09:01:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc1c5729b80 10 client.15369 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc1c5729b80 1 -- 172.21.15.99:0/1561778113 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fc19c009960 con 0x55bc5767c860 2023-08-25T09:01:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.701+0000 7fc18affd700 1 -- 172.21.15.99:0/1561778113 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc57646160 con 0x55bc5767c860 2023-08-25T09:01:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.702+0000 7fc18affd700 1 -- 172.21.15.99:0/1561778113 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc1a4097f90 con 0x55bc5767c860 2023-08-25T09:01:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.702+0000 7fc1c5729b80 10 client.15369 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.702+0000 7fc1c5729b80 10 client.15369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.702+0000 7fc1c5729b80 20 client.15369 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.702+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bc576900f0 0x55bc57692510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.702+0000 7fc1c5729b80 4 client.15369 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:01:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.702+0000 7fc1c5729b80 1 -- 172.21.15.99:0/1561778113 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc5756eb40 con 0x55bc576900f0 2023-08-25T09:01:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.702+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bc576900f0 0x55bc57692510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.703+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bc576900f0 0x55bc57692510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5164ff0a366185 server_cookie=66afc58820e562e3 in_seq=0 out_seq=0 2023-08-25T09:01:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.703+0000 7fc18affd700 10 client.15369 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.703+0000 7fc18affd700 1 -- 172.21.15.99:0/1561778113 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc5756eb40 con 0x55bc576900f0 2023-08-25T09:01:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.703+0000 7fc18affd700 10 client.15369 handle_command_reply: tid=0 2023-08-25T09:01:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 client.15369 shutdown 2023-08-25T09:01:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 -- 172.21.15.99:0/1561778113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5767c860 msgr2=0x55bc5767fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5767c860 0x55bc5767fd40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40090a0 tx=0x7fc1a4098510 comp rx=0 tx=0).stop 2023-08-25T09:01:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 -- 172.21.15.99:0/1561778113 shutdown_connections 2023-08-25T09:01:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5767c860 0x55bc5767fd40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc57676990 0x55bc57682560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bc576900f0 0x55bc57692510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/1561778113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc5767f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 -- 172.21.15.99:0/1561778113 >> 172.21.15.99:0/1561778113 conn(0x55bc575eb080 msgr2=0x55bc5766b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc18affd700 0 client.15369 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 -- 172.21.15.99:0/1561778113 shutdown_connections 2023-08-25T09:01:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 1 -- 172.21.15.99:0/1561778113 wait complete. 2023-08-25T09:01:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.704+0000 7fc1c5729b80 20 client.15369 trim_cache size 0 max 16384 2023-08-25T09:01:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.719+0000 7fc1c5729b80 1 Processor -- start 2023-08-25T09:01:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.719+0000 7fc1c5729b80 1 -- start start 2023-08-25T09:01:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.719+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc5767d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.719+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc57687270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.719+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57682100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.719+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5744a8c0 con 0x55bc57676990 2023-08-25T09:01:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5768d730 con 0x55bc5767c860 2023-08-25T09:01:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc57671250 con 0x55bc5767bf40 2023-08-25T09:01:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc5767d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc5767d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36436/0 (socket says 172.21.15.99:36436) 2023-08-25T09:01:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/3778079509 learned_addr learned my addr 172.21.15.99:0/3778079509 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc57687270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1abfff700 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57682100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/3778079509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 msgr2=0x55bc57687270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc57687270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/3778079509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 msgr2=0x55bc57682100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57682100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/3778079509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc573f4d80 con 0x55bc57676990 2023-08-25T09:01:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.720+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc5767d350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0092a0 tx=0x7fc19c003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535a66bb5ab500cc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1927fc700 1 -- 172.21.15.99:0/3778079509 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c003b60 con 0x55bc57676990 2023-08-25T09:01:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1927fc700 1 -- 172.21.15.99:0/3778079509 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c003680 con 0x55bc57676990 2023-08-25T09:01:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1927fc700 1 -- 172.21.15.99:0/3778079509 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c005680 con 0x55bc57676990 2023-08-25T09:01:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3778079509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 msgr2=0x55bc5767d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc5767d350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0092a0 tx=0x7fc19c003ee0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3778079509 shutdown_connections 2023-08-25T09:01:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767c860 0x55bc57682100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc5767d350 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3778079509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767bf40 0x55bc57687270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3778079509 >> 172.21.15.99:0/3778079509 conn(0x55bc575eb080 msgr2=0x55bc5767cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3778079509 shutdown_connections 2023-08-25T09:01:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.721+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3778079509 wait complete. 2023-08-25T09:01:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.722+0000 7fc1c5729b80 1 -- start start 2023-08-25T09:01:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.722+0000 7fc1c5729b80 1 Processor -- start 2023-08-25T09:01:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.722+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57685450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.722+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc57685990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.722+0000 7fc1c5729b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767c860 0x55bc576e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.722+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc57686110 con 0x55bc57676990 2023-08-25T09:01:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.722+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc576e3490 con 0x55bc5767bf40 2023-08-25T09:01:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.722+0000 7fc1c5729b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc576e3650 con 0x55bc5767c860 2023-08-25T09:01:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1c5729b80 10 client.0 _ll_register_callbacks cb 0x7fc1a40cb0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57685450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57685450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36452/0 (socket says 172.21.15.99:36452) 2023-08-25T09:01:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/3027054233 learned_addr learned my addr 172.21.15.99:0/3027054233 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc57685990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1abfff700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767c860 0x55bc576e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/3027054233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767c860 msgr2=0x55bc576e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767c860 0x55bc576e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 msgr2=0x55bc57685990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc57685990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 -- 172.21.15.99:0/3027054233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc574485d0 con 0x55bc57676990 2023-08-25T09:01:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc57685990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1abfff700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767c860 0x55bc576e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc1ab7fe700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57685450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc19c030040 tx=0x7fc19c005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea67fcc2f86a9bf2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.723+0000 7fc18affd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:01:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc18affd700 1 -- 172.21.15.99:0/3027054233 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c003d70 con 0x55bc57676990 2023-08-25T09:01:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3027054233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc57669600 con 0x55bc57676990 2023-08-25T09:01:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc18affd700 1 -- 172.21.15.99:0/3027054233 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc19c003f50 con 0x55bc57676990 2023-08-25T09:01:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc18affd700 1 -- 172.21.15.99:0/3027054233 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19c004110 con 0x55bc57676990 2023-08-25T09:01:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 10 client.15375 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3027054233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bc573ddc00 con 0x55bc57676990 2023-08-25T09:01:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc18affd700 1 -- 172.21.15.99:0/3027054233 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc57669600 con 0x55bc57676990 2023-08-25T09:01:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc18affd700 1 -- 172.21.15.99:0/3027054233 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc19c011440 con 0x55bc57676990 2023-08-25T09:01:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 10 client.15375 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 20 client.15375 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bc576e4960 0x55bc576e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 4 client.15375 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:01:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.724+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3027054233 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc575c1910 con 0x55bc576e4960 2023-08-25T09:01:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.725+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bc576e4960 0x55bc576e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.725+0000 7fc1aaffd700 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bc576e4960 0x55bc576e6d50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183e39c0598a8daa server_cookie=57502e82c4fbdb07 in_seq=0 out_seq=0 2023-08-25T09:01:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.725+0000 7fc18affd700 10 client.15375 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc18affd700 1 -- 172.21.15.99:0/3027054233 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc575c1910 con 0x55bc576e4960 2023-08-25T09:01:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc18affd700 10 client.15375 handle_command_reply: tid=0 2023-08-25T09:01:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 client.15375 shutdown 2023-08-25T09:01:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3027054233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 msgr2=0x55bc57685450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57685450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc19c030040 tx=0x7fc19c005360 comp rx=0 tx=0).stop 2023-08-25T09:01:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3027054233 shutdown_connections 2023-08-25T09:01:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc5767c860 0x55bc576e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc57676990 0x55bc57685450 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bc576e4960 0x55bc576e6d50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 --2- 172.21.15.99:0/3027054233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5767bf40 0x55bc57685990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc18affd700 0 client.15375 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3027054233 >> 172.21.15.99:0/3027054233 conn(0x55bc575eb080 msgr2=0x55bc5766af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3027054233 shutdown_connections 2023-08-25T09:01:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.768+0000 7fc1c5729b80 1 -- 172.21.15.99:0/3027054233 wait complete. 2023-08-25T09:01:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1c5729b80 20 client.15375 trim_cache size 0 max 16384 2023-08-25T09:01:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:01:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:01:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:01:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:01:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 -- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc1940a00c0 msgr2=0x7fc1940a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc1940a00c0 0x7fc1940a2580 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b6f0 tx=0x7fc1a001a040 comp rx=0 tx=0).stop 2023-08-25T09:01:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 -- 172.21.15.99:0/214884645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac106890 msgr2=0x7fc1ac1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac106890 0x7fc1ac1989c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400ef30 tx=0x7fc1a400c3a0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 -- 172.21.15.99:0/214884645 shutdown_connections 2023-08-25T09:01:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc1940a00c0 0x7fc1940a2580 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac106890 0x7fc1ac1989c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1ac100880 0x7fc1ac19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 --2- 172.21.15.99:0/214884645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac101240 0x7fc1ac19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 -- 172.21.15.99:0/214884645 >> 172.21.15.99:0/214884645 conn(0x7fc1ac0fc370 msgr2=0x7fc1ac10bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 -- 172.21.15.99:0/214884645 shutdown_connections 2023-08-25T09:01:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:05.769+0000 7fc1b1dff700 1 -- 172.21.15.99:0/214884645 wait complete. 2023-08-25T09:01:05.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39722 3.26 MB/sec warmup 71 sec latency 1219.700 ms 2023-08-25T09:01:06.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39722 3.21 MB/sec warmup 72 sec latency 2219.835 ms 2023-08-25T09:01:07.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:07 smithi099 ceph-mon[97641]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 915 KiB/s wr, 151 op/s 2023-08-25T09:01:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:07 smithi157 ceph-mon[130879]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 915 KiB/s wr, 151 op/s 2023-08-25T09:01:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:07 smithi181 ceph-mon[131296]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 915 KiB/s wr, 151 op/s 2023-08-25T09:01:07.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39836 3.17 MB/sec warmup 73 sec latency 3141.383 ms 2023-08-25T09:01:08.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39846 3.13 MB/sec warmup 74 sec latency 978.536 ms 2023-08-25T09:01:09.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:09 smithi099 ceph-mon[97641]: pgmap v62: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 118 op/s 2023-08-25T09:01:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:09 smithi157 ceph-mon[130879]: pgmap v62: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 118 op/s 2023-08-25T09:01:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:09 smithi181 ceph-mon[131296]: pgmap v62: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 118 op/s 2023-08-25T09:01:09.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39846 3.09 MB/sec warmup 75 sec latency 1978.624 ms 2023-08-25T09:01:10.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39846 3.05 MB/sec warmup 76 sec latency 2978.673 ms 2023-08-25T09:01:11.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:11 smithi099 ceph-mon[97641]: pgmap v63: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 118 op/s 2023-08-25T09:01:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:11 smithi157 ceph-mon[130879]: pgmap v63: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 118 op/s 2023-08-25T09:01:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:11 smithi181 ceph-mon[131296]: pgmap v63: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 118 op/s 2023-08-25T09:01:11.891 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39846 3.01 MB/sec warmup 77 sec latency 3978.747 ms 2023-08-25T09:01:12.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39968 2.97 MB/sec warmup 78 sec latency 4903.144 ms 2023-08-25T09:01:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:13 smithi099 ceph-mon[97641]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 698 KiB/s wr, 131 op/s 2023-08-25T09:01:13.665 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:13 smithi157 ceph-mon[130879]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 698 KiB/s wr, 131 op/s 2023-08-25T09:01:13.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:13 smithi181 ceph-mon[131296]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 698 KiB/s wr, 131 op/s 2023-08-25T09:01:13.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 42574 3.15 MB/sec warmup 79 sec latency 14.804 ms 2023-08-25T09:01:14.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 43388 3.14 MB/sec warmup 80 sec latency 198.105 ms 2023-08-25T09:01:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:15 smithi157 ceph-mon[130879]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 506 KiB/s wr, 103 op/s 2023-08-25T09:01:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:15 smithi181 ceph-mon[131296]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 506 KiB/s wr, 103 op/s 2023-08-25T09:01:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:15 smithi099 ceph-mon[97641]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 506 KiB/s wr, 103 op/s 2023-08-25T09:01:15.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 43388 3.10 MB/sec warmup 81 sec latency 1198.242 ms 2023-08-25T09:01:16.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 43388 3.06 MB/sec warmup 82 sec latency 2198.350 ms 2023-08-25T09:01:17.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:17 smithi099 ceph-mon[97641]: pgmap v66: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 736 KiB/s wr, 141 op/s 2023-08-25T09:01:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:17 smithi157 ceph-mon[130879]: pgmap v66: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 736 KiB/s wr, 141 op/s 2023-08-25T09:01:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:17 smithi181 ceph-mon[131296]: pgmap v66: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 736 KiB/s wr, 141 op/s 2023-08-25T09:01:17.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 43510 3.03 MB/sec warmup 83 sec latency 3118.163 ms 2023-08-25T09:01:18.893 INFO:tasks.workunit.client.0.smithi099.stdout: 1 45986 3.15 MB/sec warmup 84 sec latency 14.592 ms 2023-08-25T09:01:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 -- 172.21.15.99:0/1324267054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14108ba0 msgr2=0x7f9e14108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 --2- 172.21.15.99:0/1324267054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14108ba0 0x7f9e14108f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009990 tx=0x7f9e04009ca0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 -- 172.21.15.99:0/1324267054 shutdown_connections 2023-08-25T09:01:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 --2- 172.21.15.99:0/1324267054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e141034c0 0x7f9e1410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 --2- 172.21.15.99:0/1324267054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14108ba0 0x7f9e14108f80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 --2- 172.21.15.99:0/1324267054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e14102b00 0x7f9e14102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 -- 172.21.15.99:0/1324267054 >> 172.21.15.99:0/1324267054 conn(0x7f9e140fe5d0 msgr2=0x7f9e141009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 -- 172.21.15.99:0/1324267054 shutdown_connections 2023-08-25T09:01:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.922+0000 7f9e195ff700 1 -- 172.21.15.99:0/1324267054 wait complete. 2023-08-25T09:01:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e195ff700 1 Processor -- start 2023-08-25T09:01:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e195ff700 1 -- start start 2023-08-25T09:01:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e195ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14102b00 0x7f9e1419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e195ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e141034c0 0x7f9e1419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e195ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e14108ba0 0x7f9e14198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e195ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e14199070 con 0x7f9e14108ba0 2023-08-25T09:01:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e195ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141991e0 con 0x7f9e14102b00 2023-08-25T09:01:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e195ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141993a0 con 0x7f9e141034c0 2023-08-25T09:01:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.923+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14102b00 0x7f9e1419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14102b00 0x7f9e1419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:41288/0 (socket says 172.21.15.99:41288) 2023-08-25T09:01:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e141034c0 0x7f9e1419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e12ffd700 1 -- 172.21.15.99:0/974709000 learned_addr learned my addr 172.21.15.99:0/974709000 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e137fe700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e14108ba0 0x7f9e14198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e12ffd700 1 -- 172.21.15.99:0/974709000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e141034c0 msgr2=0x7f9e1419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e141034c0 0x7f9e1419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e12ffd700 1 -- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e14108ba0 msgr2=0x7f9e14198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e14108ba0 0x7f9e14198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e12ffd700 1 -- 172.21.15.99:0/974709000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040095f0 con 0x7f9e14102b00 2023-08-25T09:01:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e127fc700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e141034c0 0x7f9e1419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14102b00 0x7f9e1419eba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000c00 tx=0x7f9e04004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9dfbfff700 1 -- 172.21.15.99:0/974709000 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0401d070 con 0x7f9e14102b00 2023-08-25T09:01:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9dfbfff700 1 -- 172.21.15.99:0/974709000 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e04021410 con 0x7f9e14102b00 2023-08-25T09:01:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9e195ff700 1 -- 172.21.15.99:0/974709000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e14199680 con 0x7f9e14102b00 2023-08-25T09:01:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.924+0000 7f9dfbfff700 1 -- 172.21.15.99:0/974709000 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04017650 con 0x7f9e14102b00 2023-08-25T09:01:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.925+0000 7f9e195ff700 1 -- 172.21.15.99:0/974709000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e141a5b90 con 0x7f9e14102b00 2023-08-25T09:01:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.928+0000 7f9dfbfff700 1 -- 172.21.15.99:0/974709000 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9e04021580 con 0x7f9e14102b00 2023-08-25T09:01:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.929+0000 7f9e2cea8b80 1 Processor -- start 2023-08-25T09:01:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.929+0000 7f9e2cea8b80 1 -- start start 2023-08-25T09:01:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.929+0000 7f9dfbfff700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9dfc0a00c0 0x7f9dfc0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.929+0000 7f9dfbfff700 1 -- 172.21.15.99:0/974709000 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e040cc9c0 con 0x7f9e14102b00 2023-08-25T09:01:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435cdf40 0x5583435ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435c8990 0x5583435c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558343530750 con 0x5583435c8990 2023-08-25T09:01:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558343521720 con 0x5583435ce860 2023-08-25T09:01:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834351f040 con 0x5583435cdf40 2023-08-25T09:01:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e137fe700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9dfc0a00c0 0x7f9dfc0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435cdf40 0x5583435ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435cdf40 0x5583435ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58848/0 (socket says 172.21.15.99:58848) 2023-08-25T09:01:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e12ffd700 1 -- 172.21.15.99:0/69236041 learned_addr learned my addr 172.21.15.99:0/69236041 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e127fc700 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e137fe700 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435c8990 0x5583435c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e12ffd700 1 -- 172.21.15.99:0/69236041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 msgr2=0x5583435c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e12ffd700 1 -- 172.21.15.99:0/69236041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435c8990 msgr2=0x5583435c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435c8990 0x5583435c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e12ffd700 1 -- 172.21.15.99:0/69236041 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55834339a5d0 con 0x5583435cdf40 2023-08-25T09:01:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.930+0000 7f9e127fc700 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.931+0000 7f9e137fe700 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435c8990 0x5583435c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.931+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435cdf40 0x5583435ce320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e04096b30 tx=0x7f9e04099f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26085d474626f5d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.931+0000 7f9e137fe700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9dfc0a00c0 0x7f9dfc0a2580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e1419f8a0 tx=0x7f9e0c009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.931+0000 7f9df97fa700 1 -- 172.21.15.99:0/69236041 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0409d070 con 0x5583435cdf40 2023-08-25T09:01:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.931+0000 7f9df97fa700 1 -- 172.21.15.99:0/69236041 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e0409a7c0 con 0x5583435cdf40 2023-08-25T09:01:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.931+0000 7f9df97fa700 1 -- 172.21.15.99:0/69236041 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0409a320 con 0x5583435cdf40 2023-08-25T09:01:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/69236041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435cdf40 msgr2=0x5583435ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435cdf40 0x5583435ce320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e04096b30 tx=0x7f9e04099f10 comp rx=0 tx=0).stop 2023-08-25T09:01:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/69236041 shutdown_connections 2023-08-25T09:01:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435c8990 0x5583435c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435cdf40 0x5583435ce320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/69236041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/69236041 >> 172.21.15.99:0/69236041 conn(0x55834353d080 msgr2=0x5583435c2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/69236041 shutdown_connections 2023-08-25T09:01:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/69236041 wait complete. 2023-08-25T09:01:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.932+0000 7f9e2cea8b80 1 -- start start 2023-08-25T09:01:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.933+0000 7f9e2cea8b80 1 Processor -- start 2023-08-25T09:01:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834351a930 con 0x5583435cdf40 2023-08-25T09:01:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583434b8b50 con 0x5583435ce860 2023-08-25T09:01:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583434b8850 con 0x5583435c8990 2023-08-25T09:01:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e2cea8b80 10 client.0 _ll_register_callbacks cb 0x558343402a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58860/0 (socket says 172.21.15.99:58860) 2023-08-25T09:01:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 -- 172.21.15.99:0/573089587 learned_addr learned my addr 172.21.15.99:0/573089587 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e127fc700 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e137fe700 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 -- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 msgr2=0x5583435d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 -- 172.21.15.99:0/573089587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 msgr2=0x5583435d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 -- 172.21.15.99:0/573089587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e08009c90 con 0x5583435c8990 2023-08-25T09:01:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e137fe700 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d4560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e04099dc0 tx=0x7f9e04021c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adfc0e08c60d279c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.934+0000 7f9df6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:01:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9df6ff5700 1 -- 172.21.15.99:0/573089587 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0409d050 con 0x5583435c8990 2023-08-25T09:01:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9df6ff5700 1 -- 172.21.15.99:0/573089587 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e0409a0c0 con 0x5583435c8990 2023-08-25T09:01:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/573089587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558343598160 con 0x5583435c8990 2023-08-25T09:01:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9df6ff5700 1 -- 172.21.15.99:0/573089587 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e0409a7c0 con 0x5583435c8990 2023-08-25T09:01:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 10 client.25426 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/573089587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f9e141a5b90 con 0x5583435c8990 2023-08-25T09:01:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9df6ff5700 1 -- 172.21.15.99:0/573089587 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e0407c080 con 0x5583435c8990 2023-08-25T09:01:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9df6ff5700 1 -- 172.21.15.99:0/573089587 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9e040c6030 con 0x5583435c8990 2023-08-25T09:01:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 10 client.25426 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 10 client.25426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 20 client.25426 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5583435e1a60 0x5583435e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 4 client.25426 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:01:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.935+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/573089587 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583434c0b40 con 0x5583435e1a60 2023-08-25T09:01:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.936+0000 7f9e127fc700 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5583435e1a60 0x5583435e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.936+0000 7f9df6ff5700 10 client.25426 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.936+0000 7f9e127fc700 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5583435e1a60 0x5583435e3e80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf34a9bbd8278fa server_cookie=85453826f1c08485 in_seq=0 out_seq=0 2023-08-25T09:01:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9df6ff5700 1 -- 172.21.15.99:0/573089587 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5583434c0b40 con 0x5583435e1a60 2023-08-25T09:01:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9df6ff5700 10 client.25426 handle_command_reply: tid=0 2023-08-25T09:01:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 client.25426 shutdown 2023-08-25T09:01:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/573089587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 msgr2=0x5583435d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d4560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e04099dc0 tx=0x7f9e04021c00 comp rx=0 tx=0).stop 2023-08-25T09:01:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/573089587 shutdown_connections 2023-08-25T09:01:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5583435e1a60 0x5583435e3e80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d4560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/573089587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9df6ff5700 0 client.25426 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/573089587 >> 172.21.15.99:0/573089587 conn(0x55834353d080 msgr2=0x5583435bd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/573089587 shutdown_connections 2023-08-25T09:01:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/573089587 wait complete. 2023-08-25T09:01:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.937+0000 7f9e2cea8b80 20 client.25426 trim_cache size 0 max 16384 2023-08-25T09:01:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e2cea8b80 1 Processor -- start 2023-08-25T09:01:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e2cea8b80 1 -- start start 2023-08-25T09:01:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834339c8c0 con 0x5583435cdf40 2023-08-25T09:01:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583435d2e60 con 0x5583435ce860 2023-08-25T09:01:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583435c39b0 con 0x5583435c8990 2023-08-25T09:01:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e127fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e137fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e127fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51912/0 (socket says 172.21.15.99:51912) 2023-08-25T09:01:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e127fc700 1 -- 172.21.15.99:0/2644208699 learned_addr learned my addr 172.21.15.99:0/2644208699 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.954+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58870/0 (socket says 172.21.15.99:58870) 2023-08-25T09:01:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9e12ffd700 1 -- 172.21.15.99:0/2644208699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 msgr2=0x5583435d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9e12ffd700 1 -- 172.21.15.99:0/2644208699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 msgr2=0x5583435d8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9e12ffd700 1 -- 172.21.15.99:0/2644208699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558343346d80 con 0x5583435c8990 2023-08-25T09:01:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9e137fe700 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435c8450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e0409e0b0 tx=0x7f9e04021b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9451301c6b50458 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9df9ffb700 1 -- 172.21.15.99:0/2644208699 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0409d070 con 0x5583435c8990 2023-08-25T09:01:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9df9ffb700 1 -- 172.21.15.99:0/2644208699 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558343598160 con 0x5583435c8990 2023-08-25T09:01:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.955+0000 7f9df9ffb700 1 -- 172.21.15.99:0/2644208699 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04098510 con 0x5583435c8990 2023-08-25T09:01:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/2644208699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 msgr2=0x5583435c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435c8450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e0409e0b0 tx=0x7f9e04021b70 comp rx=0 tx=0).stop 2023-08-25T09:01:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/2644208699 shutdown_connections 2023-08-25T09:01:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435ce860 0x5583435d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435c8450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/2644208699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435cdf40 0x5583435d8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/2644208699 >> 172.21.15.99:0/2644208699 conn(0x55834353d080 msgr2=0x5583435de0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/2644208699 shutdown_connections 2023-08-25T09:01:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/2644208699 wait complete. 2023-08-25T09:01:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.956+0000 7f9e2cea8b80 1 -- start start 2023-08-25T09:01:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e2cea8b80 1 Processor -- start 2023-08-25T09:01:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435cdf40 0x5583435d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e2cea8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435ce860 0x558343634880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558343634dc0 con 0x5583435ce860 2023-08-25T09:01:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558343634f00 con 0x5583435cdf40 2023-08-25T09:01:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e2cea8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583436350c0 con 0x5583435c8990 2023-08-25T09:01:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e2cea8b80 10 client.0 _ll_register_callbacks cb 0x7f9e08009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e127fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435cdf40 0x5583435d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58874/0 (socket says 172.21.15.99:58874) 2023-08-25T09:01:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e12ffd700 1 -- 172.21.15.99:0/1486034354 learned_addr learned my addr 172.21.15.99:0/1486034354 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.957+0000 7f9e137fe700 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435ce860 0x558343634880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9e127fc700 1 -- 172.21.15.99:0/1486034354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 msgr2=0x5583435d5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9e127fc700 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9e127fc700 1 -- 172.21.15.99:0/1486034354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435ce860 msgr2=0x558343634880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9e127fc700 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435ce860 0x558343634880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9e127fc700 1 -- 172.21.15.99:0/1486034354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c009030 con 0x5583435cdf40 2023-08-25T09:01:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9e137fe700 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435ce860 0x558343634880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9e127fc700 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435cdf40 0x5583435d56d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e08004790 tx=0x7f9e08004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f259a227ebef98b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9df6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:01:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9df6ff5700 1 -- 172.21.15.99:0/1486034354 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08017040 con 0x5583435cdf40 2023-08-25T09:01:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/1486034354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583435bb600 con 0x5583435cdf40 2023-08-25T09:01:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9df6ff5700 1 -- 172.21.15.99:0/1486034354 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e08013e60 con 0x5583435cdf40 2023-08-25T09:01:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.958+0000 7f9df6ff5700 1 -- 172.21.15.99:0/1486034354 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e0801e980 con 0x5583435cdf40 2023-08-25T09:01:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e2cea8b80 10 client.25448 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/1486034354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f9e141a5b90 con 0x5583435cdf40 2023-08-25T09:01:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9df6ff5700 1 -- 172.21.15.99:0/1486034354 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e08032080 con 0x5583435cdf40 2023-08-25T09:01:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9df6ff5700 1 -- 172.21.15.99:0/1486034354 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9e08010430 con 0x5583435cdf40 2023-08-25T09:01:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e2cea8b80 10 client.25448 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e2cea8b80 10 client.25448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e2cea8b80 20 client.25448 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5583436368f0 0x558343638d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e2cea8b80 4 client.25448 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:01:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/1486034354 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558343513910 con 0x5583436368f0 2023-08-25T09:01:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.959+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5583436368f0 0x558343638d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.960+0000 7f9df6ff5700 10 client.25448 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.960+0000 7f9e12ffd700 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5583436368f0 0x558343638d10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f2fbfe71317f29 server_cookie=747dd1110e5be1a6 in_seq=0 out_seq=0 2023-08-25T09:01:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9df6ff5700 1 -- 172.21.15.99:0/1486034354 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558343513910 con 0x5583436368f0 2023-08-25T09:01:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9df6ff5700 10 client.25448 handle_command_reply: tid=0 2023-08-25T09:01:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 client.25448 shutdown 2023-08-25T09:01:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/1486034354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435cdf40 msgr2=0x5583435d56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435cdf40 0x5583435d56d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e08004790 tx=0x7f9e08004320 comp rx=0 tx=0).stop 2023-08-25T09:01:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/1486034354 shutdown_connections 2023-08-25T09:01:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583435ce860 0x558343634880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583435c8990 0x5583435d5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5583436368f0 0x558343638d10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 --2- 172.21.15.99:0/1486034354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5583435cdf40 0x5583435d56d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9df6ff5700 0 client.25448 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.992+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/1486034354 >> 172.21.15.99:0/1486034354 conn(0x55834353d080 msgr2=0x5583435cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/1486034354 shutdown_connections 2023-08-25T09:01:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e2cea8b80 1 -- 172.21.15.99:0/1486034354 wait complete. 2023-08-25T09:01:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e2cea8b80 20 client.25448 trim_cache size 0 max 16384 2023-08-25T09:01:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:01:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:01:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:01:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:01:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 -- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9dfc0a00c0 msgr2=0x7f9dfc0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9dfc0a00c0 0x7f9dfc0a2580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e1419f8a0 tx=0x7f9e0c009920 comp rx=0 tx=0).stop 2023-08-25T09:01:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 -- 172.21.15.99:0/974709000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14102b00 msgr2=0x7f9e1419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14102b00 0x7f9e1419eba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000c00 tx=0x7f9e04004790 comp rx=0 tx=0).stop 2023-08-25T09:01:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 -- 172.21.15.99:0/974709000 shutdown_connections 2023-08-25T09:01:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9dfc0a00c0 0x7f9dfc0a2580 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e14108ba0 0x7f9e14198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e14102b00 0x7f9e1419eba0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 --2- 172.21.15.99:0/974709000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e141034c0 0x7f9e1419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.993+0000 7f9e195ff700 1 -- 172.21.15.99:0/974709000 >> 172.21.15.99:0/974709000 conn(0x7f9e140fe5d0 msgr2=0x7f9e140feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.994+0000 7f9e195ff700 1 -- 172.21.15.99:0/974709000 shutdown_connections 2023-08-25T09:01:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:18.994+0000 7f9e195ff700 1 -- 172.21.15.99:0/974709000 wait complete. 2023-08-25T09:01:19.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:19 smithi099 ceph-mon[97641]: pgmap v67: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 336 KiB/s wr, 51 op/s 2023-08-25T09:01:19.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:19 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:01:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:19 smithi157 ceph-mon[130879]: pgmap v67: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 336 KiB/s wr, 51 op/s 2023-08-25T09:01:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:19 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:01:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:19 smithi181 ceph-mon[131296]: pgmap v67: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 336 KiB/s wr, 51 op/s 2023-08-25T09:01:19.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:19 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:01:19.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 47050 3.18 MB/sec warmup 85 sec latency 17.441 ms 2023-08-25T09:01:20.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:20 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:20.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:20 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:20.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:20 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:20.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:20 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:20.892 INFO:tasks.workunit.client.0.smithi099.stdout: 1 49333 3.29 MB/sec warmup 86 sec latency 18.237 ms 2023-08-25T09:01:20.992 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:20 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:20.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:20 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:20.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:20 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:20.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:20 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:21.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:20 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:21.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:20 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:21.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:20 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:21.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:20 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:21 smithi157 ceph-mon[130879]: pgmap v68: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 336 KiB/s wr, 51 op/s 2023-08-25T09:01:21.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:21 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:01:21.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:21 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:01:21.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:21 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:21.893 INFO:tasks.workunit.client.0.smithi099.stdout: 1 50355 3.32 MB/sec warmup 87 sec latency 152.431 ms 2023-08-25T09:01:21.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:21 smithi181 ceph-mon[131296]: pgmap v68: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 336 KiB/s wr, 51 op/s 2023-08-25T09:01:21.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:21 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:01:21.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:21 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:01:21.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:21 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:22.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:21 smithi099 ceph-mon[97641]: pgmap v68: 97 pgs: 97 active+clean; 36 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 336 KiB/s wr, 51 op/s 2023-08-25T09:01:22.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:21 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:01:22.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:21 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:01:22.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:21 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:01:22.893 INFO:tasks.workunit.client.0.smithi099.stdout: 1 50411 3.28 MB/sec warmup 88 sec latency 1074.058 ms 2023-08-25T09:01:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:23 smithi157 ceph-mon[130879]: pgmap v69: 97 pgs: 97 active+clean; 36 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 KiB/s wr, 126 op/s 2023-08-25T09:01:23.893 INFO:tasks.workunit.client.0.smithi099.stdout: 1 50411 3.25 MB/sec warmup 89 sec latency 1037.087 ms 2023-08-25T09:01:23.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:23 smithi181 ceph-mon[131296]: pgmap v69: 97 pgs: 97 active+clean; 36 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 KiB/s wr, 126 op/s 2023-08-25T09:01:24.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:23 smithi099 ceph-mon[97641]: pgmap v69: 97 pgs: 97 active+clean; 36 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 KiB/s wr, 126 op/s 2023-08-25T09:01:24.893 INFO:tasks.workunit.client.0.smithi099.stdout: 1 50411 3.21 MB/sec warmup 90 sec latency 2037.212 ms 2023-08-25T09:01:25.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:25 smithi157 ceph-mon[130879]: pgmap v70: 97 pgs: 97 active+clean; 36 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 720 KiB/s wr, 113 op/s 2023-08-25T09:01:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:25 smithi181 ceph-mon[131296]: pgmap v70: 97 pgs: 97 active+clean; 36 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 720 KiB/s wr, 113 op/s 2023-08-25T09:01:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:25 smithi099 ceph-mon[97641]: pgmap v70: 97 pgs: 97 active+clean; 36 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 720 KiB/s wr, 113 op/s 2023-08-25T09:01:25.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 50411 3.17 MB/sec warmup 91 sec latency 3037.379 ms 2023-08-25T09:01:26.893 INFO:tasks.workunit.client.0.smithi099.stdout: 1 50411 3.14 MB/sec warmup 92 sec latency 4037.521 ms 2023-08-25T09:01:27.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:27 smithi099 ceph-mon[97641]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 KiB/s wr, 130 op/s 2023-08-25T09:01:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:27 smithi157 ceph-mon[130879]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 KiB/s wr, 130 op/s 2023-08-25T09:01:27.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:27 smithi181 ceph-mon[131296]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 KiB/s wr, 130 op/s 2023-08-25T09:01:27.893 INFO:tasks.workunit.client.0.smithi099.stdout: 1 50515 3.11 MB/sec warmup 93 sec latency 4956.623 ms 2023-08-25T09:01:28.893 INFO:tasks.workunit.client.0.smithi099.stdout: 1 53065 3.23 MB/sec warmup 94 sec latency 9.410 ms 2023-08-25T09:01:29.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:29 smithi099 ceph-mon[97641]: pgmap v72: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 564 KiB/s wr, 92 op/s 2023-08-25T09:01:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:29 smithi157 ceph-mon[130879]: pgmap v72: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 564 KiB/s wr, 92 op/s 2023-08-25T09:01:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:29 smithi181 ceph-mon[131296]: pgmap v72: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 564 KiB/s wr, 92 op/s 2023-08-25T09:01:29.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 53903 3.24 MB/sec warmup 95 sec latency 295.347 ms 2023-08-25T09:01:30.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 53903 3.21 MB/sec warmup 96 sec latency 1295.506 ms 2023-08-25T09:01:31.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:31 smithi099 ceph-mon[97641]: pgmap v73: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 564 KiB/s wr, 92 op/s 2023-08-25T09:01:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:31 smithi157 ceph-mon[130879]: pgmap v73: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 564 KiB/s wr, 92 op/s 2023-08-25T09:01:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:31 smithi181 ceph-mon[131296]: pgmap v73: 97 pgs: 97 active+clean; 36 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 564 KiB/s wr, 92 op/s 2023-08-25T09:01:31.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 53903 3.17 MB/sec warmup 97 sec latency 2295.619 ms 2023-08-25T09:01:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 -- 172.21.15.99:0/3253358254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 msgr2=0x7f9ae8106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 --2- 172.21.15.99:0/3253358254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 0x7f9ae8106ca0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009990 tx=0x7f9ad8009ca0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 -- 172.21.15.99:0/3253358254 shutdown_connections 2023-08-25T09:01:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 --2- 172.21.15.99:0/3253358254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae8101270 0x7f9ae810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 --2- 172.21.15.99:0/3253358254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 0x7f9ae8106ca0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 --2- 172.21.15.99:0/3253358254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ae81008b0 0x7f9ae8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 -- 172.21.15.99:0/3253358254 >> 172.21.15.99:0/3253358254 conn(0x7f9ae80fc380 msgr2=0x7f9ae80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 -- 172.21.15.99:0/3253358254 shutdown_connections 2023-08-25T09:01:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.149+0000 7f9aedbff700 1 -- 172.21.15.99:0/3253358254 wait complete. 2023-08-25T09:01:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.150+0000 7f9aedbff700 1 Processor -- start 2023-08-25T09:01:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9aedbff700 1 -- start start 2023-08-25T09:01:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9aedbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ae81008b0 0x7f9ae819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9aedbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae8101270 0x7f9ae819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9aedbff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 0x7f9ae81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9aedbff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8199180 con 0x7f9ae81068c0 2023-08-25T09:01:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9aedbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae81992f0 con 0x7f9ae81008b0 2023-08-25T09:01:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9aedbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae81994b0 con 0x7f9ae8101270 2023-08-25T09:01:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ae81008b0 0x7f9ae819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae8101270 0x7f9ae819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 0x7f9ae81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae8101270 0x7f9ae819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:32804/0 (socket says 172.21.15.99:32804) 2023-08-25T09:01:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/897395204 learned_addr learned my addr 172.21.15.99:0/897395204 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.151+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ae81008b0 0x7f9ae819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47256/0 (socket says 172.21.15.99:47256) 2023-08-25T09:01:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/897395204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ae81008b0 msgr2=0x7f9ae819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ae81008b0 0x7f9ae819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 msgr2=0x7f9ae81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 0x7f9ae81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/897395204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80095f0 con 0x7f9ae8101270 2023-08-25T09:01:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae7fff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 0x7f9ae81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae8101270 0x7f9ae819f120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00d980 tx=0x7f9adc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae4ff9700 1 -- 172.21.15.99:0/897395204 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc004220 con 0x7f9ae8101270 2023-08-25T09:01:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9aedbff700 1 -- 172.21.15.99:0/897395204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae8199790 con 0x7f9ae8101270 2023-08-25T09:01:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae4ff9700 1 -- 172.21.15.99:0/897395204 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9adc007640 con 0x7f9ae8101270 2023-08-25T09:01:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9ae4ff9700 1 -- 172.21.15.99:0/897395204 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc005690 con 0x7f9ae8101270 2023-08-25T09:01:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.152+0000 7f9aedbff700 1 -- 172.21.15.99:0/897395204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae81a5bb0 con 0x7f9ae8101270 2023-08-25T09:01:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.156+0000 7f9ae4ff9700 1 -- 172.21.15.99:0/897395204 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9adc004750 con 0x7f9ae8101270 2023-08-25T09:01:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.157+0000 7f9b014e5b80 1 Processor -- start 2023-08-25T09:01:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.157+0000 7f9b014e5b80 1 -- start start 2023-08-25T09:01:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.157+0000 7f9ae4ff9700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ad00a00c0 0x7f9ad00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.157+0000 7f9ae4ff9700 1 -- 172.21.15.99:0/897395204 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9adc0cb450 con 0x7f9ae8101270 2023-08-25T09:01:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.157+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280023f40 0x556280024320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.157+0000 7f9ae7fff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ad00a00c0 0x7f9ad00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.157+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628001e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55628001e990 0x55628001edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627ff86750 con 0x556280024860 2023-08-25T09:01:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627ff77720 con 0x55628001e990 2023-08-25T09:01:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627ff75040 con 0x556280023f40 2023-08-25T09:01:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628001e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628001e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51014/0 (socket says 172.21.15.99:51014) 2023-08-25T09:01:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/250006182 learned_addr learned my addr 172.21.15.99:0/250006182 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae77fe700 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280023f40 0x556280024320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae7fff700 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55628001e990 0x55628001edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/250006182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280023f40 msgr2=0x556280024320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280023f40 0x556280024320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/250006182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55628001e990 msgr2=0x55628001edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55628001e990 0x55628001edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/250006182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55627fdf05d0 con 0x556280024860 2023-08-25T09:01:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae77fe700 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280023f40 0x556280024320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae7fff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ad00a00c0 0x7f9ad00a2580 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009550 tx=0x7f9ae00079a0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.158+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628001e450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9adc098ab0 tx=0x7f9adc0992f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeeef7d1fa1f961 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9acdffb700 1 -- 172.21.15.99:0/250006182 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc09c950 con 0x556280024860 2023-08-25T09:01:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9acdffb700 1 -- 172.21.15.99:0/250006182 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9adc0a0930 con 0x556280024860 2023-08-25T09:01:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9acdffb700 1 -- 172.21.15.99:0/250006182 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc098fe0 con 0x556280024860 2023-08-25T09:01:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9b014e5b80 1 -- 172.21.15.99:0/250006182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 msgr2=0x55628001e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628001e450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9adc098ab0 tx=0x7f9adc0992f0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9b014e5b80 1 -- 172.21.15.99:0/250006182 shutdown_connections 2023-08-25T09:01:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55628001e990 0x55628001edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280023f40 0x556280024320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/250006182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628001e450 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9b014e5b80 1 -- 172.21.15.99:0/250006182 >> 172.21.15.99:0/250006182 conn(0x55627ff93080 msgr2=0x556280018d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.159+0000 7f9b014e5b80 1 -- 172.21.15.99:0/250006182 shutdown_connections 2023-08-25T09:01:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.160+0000 7f9b014e5b80 1 -- 172.21.15.99:0/250006182 wait complete. 2023-08-25T09:01:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.160+0000 7f9b014e5b80 1 -- start start 2023-08-25T09:01:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.160+0000 7f9b014e5b80 1 Processor -- start 2023-08-25T09:01:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280023f40 0x556280027970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280024860 0x556280027ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627ff70930 con 0x556280023f40 2023-08-25T09:01:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627ff0eb50 con 0x556280024860 2023-08-25T09:01:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627ff0e850 con 0x55628001e990 2023-08-25T09:01:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9b014e5b80 10 client.0 _ll_register_callbacks cb 0x55627fe58a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280023f40 0x556280027970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280023f40 0x556280027970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51028/0 (socket says 172.21.15.99:51028) 2023-08-25T09:01:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/2556015613 learned_addr learned my addr 172.21.15.99:0/2556015613 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae77fe700 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae7fff700 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280024860 0x556280027ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/2556015613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 msgr2=0x55628002a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/2556015613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280024860 msgr2=0x556280027ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280024860 0x556280027ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.161+0000 7f9ae6ffd700 1 -- 172.21.15.99:0/2556015613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae81a5bb0 con 0x556280023f40 2023-08-25T09:01:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280023f40 0x556280027970 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0a0400 tx=0x7f9adc098760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82385ca17bb3bc4d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9ac6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:01:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/2556015613 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc09c950 con 0x556280023f40 2023-08-25T09:01:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/2556015613 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9adc095230 con 0x556280023f40 2023-08-25T09:01:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9b014e5b80 1 -- 172.21.15.99:0/2556015613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55627ffee160 con 0x556280023f40 2023-08-25T09:01:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/2556015613 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9adc0953f0 con 0x556280023f40 2023-08-25T09:01:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9b014e5b80 10 client.15405 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9b014e5b80 1 -- 172.21.15.99:0/2556015613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55627fd9cd80 con 0x556280023f40 2023-08-25T09:01:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/2556015613 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55627ffee160 con 0x556280023f40 2023-08-25T09:01:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.162+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/2556015613 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9adc098e60 con 0x556280023f40 2023-08-25T09:01:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.163+0000 7f9b014e5b80 10 client.15405 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.163+0000 7f9b014e5b80 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.163+0000 7f9b014e5b80 20 client.15405 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.163+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x556280037d90 0x55628003a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.163+0000 7f9b014e5b80 4 client.15405 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:01:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.163+0000 7f9b014e5b80 1 -- 172.21.15.99:0/2556015613 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55627ff16b40 con 0x556280037d90 2023-08-25T09:01:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.163+0000 7f9ae77fe700 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x556280037d90 0x55628003a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.164+0000 7f9ae77fe700 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x556280037d90 0x55628003a1b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc559585fc443d3b server_cookie=23fdae2a37747e5b in_seq=0 out_seq=0 2023-08-25T09:01:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.164+0000 7f9ac6ffd700 10 client.15405 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/2556015613 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55627ff16b40 con 0x556280037d90 2023-08-25T09:01:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9ac6ffd700 10 client.15405 handle_command_reply: tid=0 2023-08-25T09:01:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 client.15405 shutdown 2023-08-25T09:01:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 -- 172.21.15.99:0/2556015613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280023f40 msgr2=0x556280027970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280023f40 0x556280027970 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0a0400 tx=0x7f9adc098760 comp rx=0 tx=0).stop 2023-08-25T09:01:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 -- 172.21.15.99:0/2556015613 shutdown_connections 2023-08-25T09:01:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x556280037d90 0x55628003a1b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280024860 0x556280027ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/2556015613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280023f40 0x556280027970 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9ac6ffd700 0 client.15405 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 -- 172.21.15.99:0/2556015613 >> 172.21.15.99:0/2556015613 conn(0x55627ff93080 msgr2=0x556280013820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 -- 172.21.15.99:0/2556015613 shutdown_connections 2023-08-25T09:01:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 1 -- 172.21.15.99:0/2556015613 wait complete. 2023-08-25T09:01:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.165+0000 7f9b014e5b80 20 client.15405 trim_cache size 0 max 16384 2023-08-25T09:01:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.180+0000 7f9b014e5b80 1 Processor -- start 2023-08-25T09:01:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.180+0000 7f9b014e5b80 1 -- start start 2023-08-25T09:01:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.180+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55628001e990 0x556280019810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.180+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 0x5562800339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.180+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280024860 0x556280021af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.180+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627fdf28c0 con 0x55628001e990 2023-08-25T09:01:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.180+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556280015700 con 0x556280023f40 2023-08-25T09:01:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.180+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556280015960 con 0x556280024860 2023-08-25T09:01:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55628001e990 0x556280019810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280024860 0x556280021af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55628001e990 0x556280019810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51042/0 (socket says 172.21.15.99:51042) 2023-08-25T09:01:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae77fe700 1 -- 172.21.15.99:0/3284757473 learned_addr learned my addr 172.21.15.99:0/3284757473 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/3284757473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 0x5562800339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae7fff700 1 -- 172.21.15.99:0/3284757473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 msgr2=0x5562800339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae7fff700 1 --2- 172.21.15.99:0/3284757473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 0x5562800339a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae7fff700 1 -- 172.21.15.99:0/3284757473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55628001e990 msgr2=0x556280019810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae7fff700 1 --2- 172.21.15.99:0/3284757473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55628001e990 0x556280019810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae7fff700 1 -- 172.21.15.99:0/3284757473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55627fd85c00 con 0x556280024860 2023-08-25T09:01:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.181+0000 7f9ae7fff700 1 --2- 172.21.15.99:0/3284757473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280024860 0x556280021af0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009fd0 tx=0x7f9ae0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7843435b2f1830b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9ace7fc700 1 -- 172.21.15.99:0/3284757473 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0023800 con 0x556280024860 2023-08-25T09:01:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9ace7fc700 1 -- 172.21.15.99:0/3284757473 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ae000f040 con 0x556280024860 2023-08-25T09:01:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9ace7fc700 1 -- 172.21.15.99:0/3284757473 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0017d00 con 0x556280024860 2023-08-25T09:01:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 -- 172.21.15.99:0/3284757473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280024860 msgr2=0x556280021af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/3284757473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280024860 0x556280021af0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009fd0 tx=0x7f9ae0015000 comp rx=0 tx=0).stop 2023-08-25T09:01:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 -- 172.21.15.99:0/3284757473 shutdown_connections 2023-08-25T09:01:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/3284757473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556280024860 0x556280021af0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/3284757473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55628001e990 0x556280019810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/3284757473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 0x5562800339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 -- 172.21.15.99:0/3284757473 >> 172.21.15.99:0/3284757473 conn(0x55627ff93080 msgr2=0x556280018bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 -- 172.21.15.99:0/3284757473 shutdown_connections 2023-08-25T09:01:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.182+0000 7f9b014e5b80 1 -- 172.21.15.99:0/3284757473 wait complete. 2023-08-25T09:01:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 1 -- start start 2023-08-25T09:01:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 1 Processor -- start 2023-08-25T09:01:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 0x55628002d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628008ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55628008b0c0 con 0x556280024860 2023-08-25T09:01:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55628008b200 con 0x556280023f40 2023-08-25T09:01:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55628008b340 con 0x55628001e990 2023-08-25T09:01:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.183+0000 7f9b014e5b80 10 client.0 _ll_register_callbacks cb 0x7f9adc098530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628008ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:32842/0 (socket says 172.21.15.99:32842) 2023-08-25T09:01:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 -- 172.21.15.99:0/1247389596 learned_addr learned my addr 172.21.15.99:0/1247389596 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 0x55628002d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 -- 172.21.15.99:0/1247389596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 msgr2=0x55628002d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 0x55628002d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 -- 172.21.15.99:0/1247389596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 msgr2=0x55628008ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628008ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 -- 172.21.15.99:0/1247389596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae0018580 con 0x55628001e990 2023-08-25T09:01:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ae77fe700 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002d420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800b8f0 tx=0x7f9ad800b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db77c4a93a45071b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ac6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:01:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/1247389596 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad801d040 con 0x55628001e990 2023-08-25T09:01:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.184+0000 7f9b014e5b80 1 -- 172.21.15.99:0/1247389596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556280011600 con 0x55628001e990 2023-08-25T09:01:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/1247389596 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9ad8027110 con 0x55628001e990 2023-08-25T09:01:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/1247389596 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ad8027e80 con 0x55628001e990 2023-08-25T09:01:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9b014e5b80 10 client.25450 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9b014e5b80 1 -- 172.21.15.99:0/1247389596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55627fdc31f0 con 0x55628001e990 2023-08-25T09:01:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/1247389596 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556280011600 con 0x55628001e990 2023-08-25T09:01:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/1247389596 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9ad80051f0 con 0x55628001e990 2023-08-25T09:01:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9b014e5b80 10 client.25450 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9b014e5b80 10 client.25450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9b014e5b80 20 client.25450 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55628008cb60 0x55628008ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9b014e5b80 4 client.25450 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:01:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.185+0000 7f9b014e5b80 1 -- 172.21.15.99:0/1247389596 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55627ff69910 con 0x55628008cb60 2023-08-25T09:01:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.186+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55628008cb60 0x55628008ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.186+0000 7f9ac6ffd700 10 client.25450 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.186+0000 7f9ae6ffd700 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55628008cb60 0x55628008ef80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45978a5aa99b9ba4 server_cookie=eefc6e5541712ad8 in_seq=0 out_seq=0 2023-08-25T09:01:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.222+0000 7f9ac6ffd700 1 -- 172.21.15.99:0/1247389596 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55627ff69910 con 0x55628008cb60 2023-08-25T09:01:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.222+0000 7f9ac6ffd700 10 client.25450 handle_command_reply: tid=0 2023-08-25T09:01:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.222+0000 7f9b014e5b80 1 client.25450 shutdown 2023-08-25T09:01:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 -- 172.21.15.99:0/1247389596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 msgr2=0x55628002d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002d420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800b8f0 tx=0x7f9ad800b920 comp rx=0 tx=0).stop 2023-08-25T09:01:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 -- 172.21.15.99:0/1247389596 shutdown_connections 2023-08-25T09:01:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556280024860 0x55628008ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55628001e990 0x55628002d420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55628008cb60 0x55628008ef80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 --2- 172.21.15.99:0/1247389596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556280023f40 0x55628002d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9ac6ffd700 0 client.25450 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 -- 172.21.15.99:0/1247389596 >> 172.21.15.99:0/1247389596 conn(0x55627ff93080 msgr2=0x5562800339a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 -- 172.21.15.99:0/1247389596 shutdown_connections 2023-08-25T09:01:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 1 -- 172.21.15.99:0/1247389596 wait complete. 2023-08-25T09:01:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.223+0000 7f9b014e5b80 20 client.25450 trim_cache size 0 max 16384 2023-08-25T09:01:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:01:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:01:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:01:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:01:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 -- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ad00a00c0 msgr2=0x7f9ad00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ad00a00c0 0x7f9ad00a2580 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009550 tx=0x7f9ae00079a0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 -- 172.21.15.99:0/897395204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae8101270 msgr2=0x7f9ae819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae8101270 0x7f9ae819f120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00d980 tx=0x7f9adc00dc90 comp rx=0 tx=0).stop 2023-08-25T09:01:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 -- 172.21.15.99:0/897395204 shutdown_connections 2023-08-25T09:01:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ad00a00c0 0x7f9ad00a2580 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae81068c0 0x7f9ae81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ae81008b0 0x7f9ae819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 --2- 172.21.15.99:0/897395204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae8101270 0x7f9ae819f120 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 -- 172.21.15.99:0/897395204 >> 172.21.15.99:0/897395204 conn(0x7f9ae80fc380 msgr2=0x7f9ae810bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 -- 172.21.15.99:0/897395204 shutdown_connections 2023-08-25T09:01:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:32.224+0000 7f9aedbff700 1 -- 172.21.15.99:0/897395204 wait complete. 2023-08-25T09:01:32.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 54017 3.14 MB/sec warmup 98 sec latency 3217.474 ms 2023-08-25T09:01:33.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:33 smithi099 ceph-mon[97641]: pgmap v74: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 850 KiB/s wr, 130 op/s 2023-08-25T09:01:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:33 smithi157 ceph-mon[130879]: pgmap v74: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 850 KiB/s wr, 130 op/s 2023-08-25T09:01:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:33 smithi181 ceph-mon[131296]: pgmap v74: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 850 KiB/s wr, 130 op/s 2023-08-25T09:01:33.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 54027 3.11 MB/sec warmup 99 sec latency 986.507 ms 2023-08-25T09:01:34.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 54027 3.08 MB/sec warmup 100 sec latency 1986.613 ms 2023-08-25T09:01:35.380 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:35 smithi157 ceph-mon[130879]: pgmap v75: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 359 KiB/s wr, 54 op/s 2023-08-25T09:01:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:35 smithi181 ceph-mon[131296]: pgmap v75: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 359 KiB/s wr, 54 op/s 2023-08-25T09:01:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:35 smithi099 ceph-mon[97641]: pgmap v75: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 359 KiB/s wr, 54 op/s 2023-08-25T09:01:35.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 54027 3.05 MB/sec warmup 101 sec latency 2986.734 ms 2023-08-25T09:01:36.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 54027 3.02 MB/sec warmup 102 sec latency 3986.873 ms 2023-08-25T09:01:37.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:37 smithi099 ceph-mon[97641]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 706 KiB/s wr, 89 op/s 2023-08-25T09:01:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:37 smithi157 ceph-mon[130879]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 706 KiB/s wr, 89 op/s 2023-08-25T09:01:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:37 smithi181 ceph-mon[131296]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 706 KiB/s wr, 89 op/s 2023-08-25T09:01:37.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 54150 3.00 MB/sec warmup 103 sec latency 4905.045 ms 2023-08-25T09:01:38.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 56746 3.13 MB/sec warmup 104 sec latency 16.178 ms 2023-08-25T09:01:39.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:39 smithi099 ceph-mon[97641]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 633 KiB/s wr, 73 op/s 2023-08-25T09:01:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:39 smithi157 ceph-mon[130879]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 633 KiB/s wr, 73 op/s 2023-08-25T09:01:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:39 smithi181 ceph-mon[131296]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 633 KiB/s wr, 73 op/s 2023-08-25T09:01:39.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 57447 3.11 MB/sec warmup 105 sec latency 246.187 ms 2023-08-25T09:01:40.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 59280 3.16 MB/sec warmup 106 sec latency 393.990 ms 2023-08-25T09:01:41.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:41 smithi099 ceph-mon[97641]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 633 KiB/s wr, 73 op/s 2023-08-25T09:01:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:41 smithi157 ceph-mon[130879]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 633 KiB/s wr, 73 op/s 2023-08-25T09:01:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:41 smithi181 ceph-mon[131296]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 633 KiB/s wr, 73 op/s 2023-08-25T09:01:41.894 INFO:tasks.workunit.client.0.smithi099.stdout: 1 60886 3.23 MB/sec warmup 107 sec latency 11.144 ms 2023-08-25T09:01:42.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 61119 3.21 MB/sec warmup 108 sec latency 805.647 ms 2023-08-25T09:01:43.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:43 smithi099 ceph-mon[97641]: pgmap v79: 97 pgs: 97 active+clean; 38 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 138 op/s 2023-08-25T09:01:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:43 smithi157 ceph-mon[130879]: pgmap v79: 97 pgs: 97 active+clean; 38 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 138 op/s 2023-08-25T09:01:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:43 smithi181 ceph-mon[131296]: pgmap v79: 97 pgs: 97 active+clean; 38 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 138 op/s 2023-08-25T09:01:43.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 61119 3.18 MB/sec warmup 109 sec latency 1033.012 ms 2023-08-25T09:01:44.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 61119 3.15 MB/sec warmup 110 sec latency 2033.167 ms 2023-08-25T09:01:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 -- 172.21.15.99:0/305534341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0314101260 msgr2=0x7f031410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 --2- 172.21.15.99:0/305534341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0314101260 0x7f031410d770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f030c00b0a0 tx=0x7f030c00b3b0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 -- 172.21.15.99:0/305534341 shutdown_connections 2023-08-25T09:01:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 --2- 172.21.15.99:0/305534341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0314101260 0x7f031410d770 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 --2- 172.21.15.99:0/305534341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03141068b0 0x7f0314106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 --2- 172.21.15.99:0/305534341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03141008a0 0x7f0314100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 -- 172.21.15.99:0/305534341 >> 172.21.15.99:0/305534341 conn(0x7f03140fc370 msgr2=0x7f03140fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 -- 172.21.15.99:0/305534341 shutdown_connections 2023-08-25T09:01:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.378+0000 7f0318dff700 1 -- 172.21.15.99:0/305534341 wait complete. 2023-08-25T09:01:45.382 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:45 smithi157 ceph-mon[130879]: pgmap v80: 97 pgs: 97 active+clean; 38 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 980 KiB/s wr, 100 op/s 2023-08-25T09:01:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.379+0000 7f0318dff700 1 Processor -- start 2023-08-25T09:01:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.379+0000 7f0318dff700 1 -- start start 2023-08-25T09:01:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f0318dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03141008a0 0x7f031419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f0318dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0314101260 0x7f031419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f0318dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03141068b0 0x7f0314196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f0318dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0314196d80 con 0x7f03141008a0 2023-08-25T09:01:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f0318dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0314196ef0 con 0x7f03141068b0 2023-08-25T09:01:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f0318dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03141970b0 con 0x7f0314101260 2023-08-25T09:01:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f031259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03141008a0 0x7f031419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f031259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03141008a0 0x7f031419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54562/0 (socket says 172.21.15.99:54562) 2023-08-25T09:01:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f031259c700 1 -- 172.21.15.99:0/3043729139 learned_addr learned my addr 172.21.15.99:0/3043729139 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f0311d9b700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0314101260 0x7f031419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f0312d9d700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03141068b0 0x7f0314196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f031259c700 1 -- 172.21.15.99:0/3043729139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0314101260 msgr2=0x7f031419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f031259c700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0314101260 0x7f031419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f031259c700 1 -- 172.21.15.99:0/3043729139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03141068b0 msgr2=0x7f0314196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f031259c700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03141068b0 0x7f0314196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.380+0000 7f031259c700 1 -- 172.21.15.99:0/3043729139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c009d00 con 0x7f03141008a0 2023-08-25T09:01:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.381+0000 7f031259c700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03141008a0 0x7f031419c8b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9a0 tx=0x7f030400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.381+0000 7f03037fe700 1 -- 172.21.15.99:0/3043729139 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304009790 con 0x7f03141008a0 2023-08-25T09:01:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.381+0000 7f03037fe700 1 -- 172.21.15.99:0/3043729139 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0304010430 con 0x7f03141008a0 2023-08-25T09:01:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.381+0000 7f0318dff700 1 -- 172.21.15.99:0/3043729139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0314197390 con 0x7f03141008a0 2023-08-25T09:01:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.381+0000 7f03037fe700 1 -- 172.21.15.99:0/3043729139 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304018680 con 0x7f03141008a0 2023-08-25T09:01:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.381+0000 7f0318dff700 1 -- 172.21.15.99:0/3043729139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03141a38a0 con 0x7f03141008a0 2023-08-25T09:01:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.384+0000 7f03037fe700 1 -- 172.21.15.99:0/3043729139 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f03040105a0 con 0x7f03141008a0 2023-08-25T09:01:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.385+0000 7f032c67fb80 1 Processor -- start 2023-08-25T09:01:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.385+0000 7f03037fe700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f02fc0a00c0 0x7f02fc0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f032c67fb80 1 -- start start 2023-08-25T09:01:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f03037fe700 1 -- 172.21.15.99:0/3043729139 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0304014070 con 0x7f03141008a0 2023-08-25T09:01:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f0312d9d700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f02fc0a00c0 0x7f02fc0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f61320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f61860 0x55b853f5b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f5b990 0x55b853f5bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f032c67fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853ec3750 con 0x55b853f5b990 2023-08-25T09:01:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f032c67fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853eb4720 con 0x55b853f60f40 2023-08-25T09:01:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f032c67fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853eb2040 con 0x55b853f61860 2023-08-25T09:01:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f031259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f61320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f031259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f61320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:34976/0 (socket says 172.21.15.99:34976) 2023-08-25T09:01:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f031259c700 1 -- 172.21.15.99:0/3662740517 learned_addr learned my addr 172.21.15.99:0/3662740517 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f0312d9d700 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f5b990 0x55b853f5bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.386+0000 7f0311d9b700 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f61860 0x55b853f5b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f031259c700 1 -- 172.21.15.99:0/3662740517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f61860 msgr2=0x55b853f5b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f031259c700 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f61860 0x55b853f5b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f031259c700 1 -- 172.21.15.99:0/3662740517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f5b990 msgr2=0x55b853f5bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f031259c700 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f5b990 0x55b853f5bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f031259c700 1 -- 172.21.15.99:0/3662740517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b853d2d5d0 con 0x55b853f60f40 2023-08-25T09:01:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f0311d9b700 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f61860 0x55b853f5b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f0312d9d700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f02fc0a00c0 0x7f02fc0a2580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f030c00b780 tx=0x7f030c00bed0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f031259c700 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f61320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0304009230 tx=0x7f0304099590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8790e9d25854ab server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f0300ff9700 1 -- 172.21.15.99:0/3662740517 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304099ea0 con 0x55b853f60f40 2023-08-25T09:01:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f0300ff9700 1 -- 172.21.15.99:0/3662740517 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03040a0ca0 con 0x55b853f60f40 2023-08-25T09:01:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.387+0000 7f0300ff9700 1 -- 172.21.15.99:0/3662740517 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040985a0 con 0x55b853f60f40 2023-08-25T09:01:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 -- 172.21.15.99:0/3662740517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 msgr2=0x55b853f61320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f61320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0304009230 tx=0x7f0304099590 comp rx=0 tx=0).stop 2023-08-25T09:01:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 -- 172.21.15.99:0/3662740517 shutdown_connections 2023-08-25T09:01:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f5b990 0x55b853f5bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f61320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3662740517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f61860 0x55b853f5b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 -- 172.21.15.99:0/3662740517 >> 172.21.15.99:0/3662740517 conn(0x55b853ed0080 msgr2=0x55b853f55d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 -- 172.21.15.99:0/3662740517 shutdown_connections 2023-08-25T09:01:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 -- 172.21.15.99:0/3662740517 wait complete. 2023-08-25T09:01:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.388+0000 7f032c67fb80 1 -- start start 2023-08-25T09:01:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f032c67fb80 1 Processor -- start 2023-08-25T09:01:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f67730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f032c67fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853ead930 con 0x55b853f61860 2023-08-25T09:01:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f032c67fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853e4bb50 con 0x55b853f60f40 2023-08-25T09:01:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f032c67fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853e4b850 con 0x55b853f5b990 2023-08-25T09:01:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f032c67fb80 10 client.0 _ll_register_callbacks cb 0x55b853d95a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.389+0000 7f0311d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0311d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f649b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:34984/0 (socket says 172.21.15.99:34984) 2023-08-25T09:01:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0312d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0311d9b700 1 -- 172.21.15.99:0/3097028532 learned_addr learned my addr 172.21.15.99:0/3097028532 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f031259c700 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f67730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0311d9b700 1 -- 172.21.15.99:0/3097028532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 msgr2=0x55b853f67730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0311d9b700 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f67730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0311d9b700 1 -- 172.21.15.99:0/3097028532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 msgr2=0x55b853f650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0311d9b700 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0311d9b700 1 -- 172.21.15.99:0/3097028532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03141a38a0 con 0x55b853f60f40 2023-08-25T09:01:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0312d9d700 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f0311d9b700 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f649b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0308012700 tx=0x7f0308012a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e5554e7cf1fa09b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f02f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:01:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.390+0000 7f02f9ffb700 1 -- 172.21.15.99:0/3097028532 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308017070 con 0x55b853f60f40 2023-08-25T09:01:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 1 -- 172.21.15.99:0/3097028532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b853f2b160 con 0x55b853f60f40 2023-08-25T09:01:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f02f9ffb700 1 -- 172.21.15.99:0/3097028532 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0308004830 con 0x55b853f60f40 2023-08-25T09:01:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f02f9ffb700 1 -- 172.21.15.99:0/3097028532 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03080049f0 con 0x55b853f60f40 2023-08-25T09:01:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 10 client.25478 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 1 -- 172.21.15.99:0/3097028532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b853cd9d80 con 0x55b853f60f40 2023-08-25T09:01:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f02f9ffb700 1 -- 172.21.15.99:0/3097028532 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b853f2b160 con 0x55b853f60f40 2023-08-25T09:01:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f02f9ffb700 1 -- 172.21.15.99:0/3097028532 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0308022070 con 0x55b853f60f40 2023-08-25T09:01:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 10 client.25478 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 10 client.25478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 20 client.25478 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b853f752c0 0x55b853f776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 4 client.25478 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:01:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.391+0000 7f032c67fb80 1 -- 172.21.15.99:0/3097028532 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b853e53b40 con 0x55b853f752c0 2023-08-25T09:01:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.392+0000 7f031259c700 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b853f752c0 0x55b853f776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.392+0000 7f02f9ffb700 10 client.25478 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.392+0000 7f031259c700 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b853f752c0 0x55b853f776e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f037b2c5f0a13e3 server_cookie=bc3605128c38bd18 in_seq=0 out_seq=0 2023-08-25T09:01:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f02f9ffb700 1 -- 172.21.15.99:0/3097028532 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b853e53b40 con 0x55b853f752c0 2023-08-25T09:01:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f02f9ffb700 10 client.25478 handle_command_reply: tid=0 2023-08-25T09:01:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 client.25478 shutdown 2023-08-25T09:01:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 -- 172.21.15.99:0/3097028532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 msgr2=0x55b853f649b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f649b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0308012700 tx=0x7f0308012a10 comp rx=0 tx=0).stop 2023-08-25T09:01:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 -- 172.21.15.99:0/3097028532 shutdown_connections 2023-08-25T09:01:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f67730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b853f752c0 0x55b853f776e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3097028532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f649b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f02f9ffb700 0 client.25478 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 -- 172.21.15.99:0/3097028532 >> 172.21.15.99:0/3097028532 conn(0x55b853ed0080 msgr2=0x55b853f50820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 -- 172.21.15.99:0/3097028532 shutdown_connections 2023-08-25T09:01:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.393+0000 7f032c67fb80 1 -- 172.21.15.99:0/3097028532 wait complete. 2023-08-25T09:01:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.394+0000 7f032c67fb80 20 client.25478 trim_cache size 0 max 16384 2023-08-25T09:01:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f032c67fb80 1 Processor -- start 2023-08-25T09:01:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f032c67fb80 1 -- start start 2023-08-25T09:01:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f6c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f55bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f032c67fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853d2f8c0 con 0x55b853f61860 2023-08-25T09:01:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f032c67fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853f6c800 con 0x55b853f60f40 2023-08-25T09:01:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f032c67fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853f5f6b0 con 0x55b853f5b990 2023-08-25T09:01:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f0312d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f55bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f0311d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f6c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f0311d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f6c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:35000/0 (socket says 172.21.15.99:35000) 2023-08-25T09:01:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f0311d9b700 1 -- 172.21.15.99:0/3129768944 learned_addr learned my addr 172.21.15.99:0/3129768944 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.409+0000 7f031259c700 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f0312d9d700 1 -- 172.21.15.99:0/3129768944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 msgr2=0x55b853f5b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f0312d9d700 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f0312d9d700 1 -- 172.21.15.99:0/3129768944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 msgr2=0x55b853f6c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f0312d9d700 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f6c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f0312d9d700 1 -- 172.21.15.99:0/3129768944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b853cc2c00 con 0x55b853f61860 2023-08-25T09:01:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f0311d9b700 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f6c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f0312d9d700 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f55bd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f030c009fd0 tx=0x7f030c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461bbe96f68dab29 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f03017fa700 1 -- 172.21.15.99:0/3129768944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00e040 con 0x55b853f61860 2023-08-25T09:01:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f03017fa700 1 -- 172.21.15.99:0/3129768944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f030c007760 con 0x55b853f61860 2023-08-25T09:01:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.410+0000 7f03017fa700 1 -- 172.21.15.99:0/3129768944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c005040 con 0x55b853f61860 2023-08-25T09:01:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 -- 172.21.15.99:0/3129768944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 msgr2=0x55b853f55bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f55bd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f030c009fd0 tx=0x7f030c023000 comp rx=0 tx=0).stop 2023-08-25T09:01:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 -- 172.21.15.99:0/3129768944 shutdown_connections 2023-08-25T09:01:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f55bd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 --2- 172.21.15.99:0/3129768944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f6c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 -- 172.21.15.99:0/3129768944 >> 172.21.15.99:0/3129768944 conn(0x55b853ed0080 msgr2=0x55b853f7c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 -- 172.21.15.99:0/3129768944 shutdown_connections 2023-08-25T09:01:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 -- 172.21.15.99:0/3129768944 wait complete. 2023-08-25T09:01:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 -- start start 2023-08-25T09:01:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.411+0000 7f032c67fb80 1 Processor -- start 2023-08-25T09:01:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f5ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f032c67fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f5d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f032c67fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853f5dc90 con 0x55b853f61860 2023-08-25T09:01:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f032c67fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853f5de00 con 0x55b853f60f40 2023-08-25T09:01:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f032c67fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853f5dfc0 con 0x55b853f5b990 2023-08-25T09:01:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f032c67fb80 10 client.0 _ll_register_callbacks cb 0x55b853d95a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f0311d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f5ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f0311d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f5ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:35002/0 (socket says 172.21.15.99:35002) 2023-08-25T09:01:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f0311d9b700 1 -- 172.21.15.99:0/4280040307 learned_addr learned my addr 172.21.15.99:0/4280040307 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f031259c700 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.412+0000 7f0312d9d700 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f5d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f0311d9b700 1 -- 172.21.15.99:0/4280040307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 msgr2=0x55b853f5c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f0311d9b700 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f0311d9b700 1 -- 172.21.15.99:0/4280040307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 msgr2=0x55b853f5d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f0311d9b700 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f5d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f0311d9b700 1 -- 172.21.15.99:0/4280040307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b853cd9d80 con 0x55b853f60f40 2023-08-25T09:01:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f031259c700 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f0311d9b700 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f5ce90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0308012420 tx=0x7f0308005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c426b5b8023108e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f02f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:01:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f02f9ffb700 1 -- 172.21.15.99:0/4280040307 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308017040 con 0x55b853f60f40 2023-08-25T09:01:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f032c67fb80 1 -- 172.21.15.99:0/4280040307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b853f4e600 con 0x55b853f60f40 2023-08-25T09:01:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f02f9ffb700 1 -- 172.21.15.99:0/4280040307 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f030801c470 con 0x55b853f60f40 2023-08-25T09:01:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f02f9ffb700 1 -- 172.21.15.99:0/4280040307 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03080054c0 con 0x55b853f60f40 2023-08-25T09:01:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f032c67fb80 10 client.25484 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.413+0000 7f032c67fb80 1 -- 172.21.15.99:0/4280040307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f030c026050 con 0x55b853f60f40 2023-08-25T09:01:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f02f9ffb700 1 -- 172.21.15.99:0/4280040307 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b853f4e600 con 0x55b853f60f40 2023-08-25T09:01:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f02f9ffb700 1 -- 172.21.15.99:0/4280040307 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f030802a070 con 0x55b853f60f40 2023-08-25T09:01:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f032c67fb80 10 client.25484 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f032c67fb80 10 client.25484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f032c67fb80 20 client.25484 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f032c67fb80 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b853fce2d0 0x55b853fd06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f032c67fb80 4 client.25484 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:01:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f032c67fb80 1 -- 172.21.15.99:0/4280040307 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b853ea6910 con 0x55b853fce2d0 2023-08-25T09:01:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.414+0000 7f031259c700 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b853fce2d0 0x55b853fd06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.415+0000 7f031259c700 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b853fce2d0 0x55b853fd06f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a6c01ec08579fa server_cookie=f681854008a4137d in_seq=0 out_seq=0 2023-08-25T09:01:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.415+0000 7f02f9ffb700 10 client.25484 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.455+0000 7f02f9ffb700 1 -- 172.21.15.99:0/4280040307 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b853ea6910 con 0x55b853fce2d0 2023-08-25T09:01:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.455+0000 7f02f9ffb700 10 client.25484 handle_command_reply: tid=0 2023-08-25T09:01:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.455+0000 7f032c67fb80 1 client.25484 shutdown 2023-08-25T09:01:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.455+0000 7f032c67fb80 1 -- 172.21.15.99:0/4280040307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 msgr2=0x55b853f5ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.455+0000 7f032c67fb80 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f5ce90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0308012420 tx=0x7f0308005e30 comp rx=0 tx=0).stop 2023-08-25T09:01:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 1 -- 172.21.15.99:0/4280040307 shutdown_connections 2023-08-25T09:01:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b853f61860 0x55b853f5d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b853f5b990 0x55b853f5c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b853fce2d0 0x55b853fd06f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 1 --2- 172.21.15.99:0/4280040307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b853f60f40 0x55b853f5ce90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f02f9ffb700 0 client.25484 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 1 -- 172.21.15.99:0/4280040307 >> 172.21.15.99:0/4280040307 conn(0x55b853ed0080 msgr2=0x55b853f55bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 1 -- 172.21.15.99:0/4280040307 shutdown_connections 2023-08-25T09:01:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 1 -- 172.21.15.99:0/4280040307 wait complete. 2023-08-25T09:01:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f032c67fb80 20 client.25484 trim_cache size 0 max 16384 2023-08-25T09:01:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:01:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:01:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:01:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:01:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f0318dff700 1 -- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f02fc0a00c0 msgr2=0x7f02fc0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f0318dff700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f02fc0a00c0 0x7f02fc0a2580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f030c00b780 tx=0x7f030c00bed0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f0318dff700 1 -- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03141008a0 msgr2=0x7f031419c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.456+0000 7f0318dff700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03141008a0 0x7f031419c8b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9a0 tx=0x7f030400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.457+0000 7f0318dff700 1 -- 172.21.15.99:0/3043729139 shutdown_connections 2023-08-25T09:01:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.457+0000 7f0318dff700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f02fc0a00c0 0x7f02fc0a2580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.457+0000 7f0318dff700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03141068b0 0x7f0314196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.457+0000 7f0318dff700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03141008a0 0x7f031419c8b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.457+0000 7f0318dff700 1 --2- 172.21.15.99:0/3043729139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0314101260 0x7f031419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.457+0000 7f0318dff700 1 -- 172.21.15.99:0/3043729139 >> 172.21.15.99:0/3043729139 conn(0x7f03140fc370 msgr2=0x7f03140fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.457+0000 7f0318dff700 1 -- 172.21.15.99:0/3043729139 shutdown_connections 2023-08-25T09:01:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:45.457+0000 7f0318dff700 1 -- 172.21.15.99:0/3043729139 wait complete. 2023-08-25T09:01:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:45 smithi181 ceph-mon[131296]: pgmap v80: 97 pgs: 97 active+clean; 38 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 980 KiB/s wr, 100 op/s 2023-08-25T09:01:45.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:45 smithi099 ceph-mon[97641]: pgmap v80: 97 pgs: 97 active+clean; 38 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 980 KiB/s wr, 100 op/s 2023-08-25T09:01:45.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 61119 3.12 MB/sec warmup 111 sec latency 3033.319 ms 2023-08-25T09:01:46.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 61119 3.09 MB/sec warmup 112 sec latency 4033.475 ms 2023-08-25T09:01:47.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:47 smithi099 ceph-mon[97641]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-08-25T09:01:47.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:47 smithi157 ceph-mon[130879]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-08-25T09:01:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:47 smithi181 ceph-mon[131296]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-08-25T09:01:47.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 61242 3.07 MB/sec warmup 113 sec latency 4952.245 ms 2023-08-25T09:01:48.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 63854 3.19 MB/sec warmup 114 sec latency 18.327 ms 2023-08-25T09:01:49.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:49 smithi099 ceph-mon[97641]: pgmap v82: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 821 KiB/s wr, 99 op/s 2023-08-25T09:01:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:49 smithi157 ceph-mon[130879]: pgmap v82: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 821 KiB/s wr, 99 op/s 2023-08-25T09:01:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:49 smithi181 ceph-mon[131296]: pgmap v82: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 821 KiB/s wr, 99 op/s 2023-08-25T09:01:49.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 65039 3.19 MB/sec warmup 115 sec latency 10.748 ms 2023-08-25T09:01:50.895 INFO:tasks.workunit.client.0.smithi099.stdout: 1 67563 3.30 MB/sec warmup 116 sec latency 13.202 ms 2023-08-25T09:01:51.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:51 smithi099 ceph-mon[97641]: pgmap v83: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 821 KiB/s wr, 99 op/s 2023-08-25T09:01:51.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:51 smithi157 ceph-mon[130879]: pgmap v83: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 821 KiB/s wr, 99 op/s 2023-08-25T09:01:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:51 smithi181 ceph-mon[131296]: pgmap v83: 97 pgs: 97 active+clean; 39 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 821 KiB/s wr, 99 op/s 2023-08-25T09:01:51.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68087 3.29 MB/sec warmup 117 sec latency 383.223 ms 2023-08-25T09:01:52.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68142 3.26 MB/sec warmup 118 sec latency 1311.391 ms 2023-08-25T09:01:53.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:53 smithi099 ceph-mon[97641]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2023-08-25T09:01:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:53 smithi157 ceph-mon[130879]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2023-08-25T09:01:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:53 smithi181 ceph-mon[131296]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2023-08-25T09:01:53.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68143 3.23 MB/sec warmup 119 sec latency 992.820 ms 2023-08-25T09:01:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:55 smithi157 ceph-mon[130879]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 816 KiB/s wr, 104 op/s 2023-08-25T09:01:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:55 smithi181 ceph-mon[131296]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 816 KiB/s wr, 104 op/s 2023-08-25T09:01:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:55 smithi099 ceph-mon[97641]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 816 KiB/s wr, 104 op/s 2023-08-25T09:01:55.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68143 0.00 MB/sec execute 1 sec latency 2993.063 ms 2023-08-25T09:01:56.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68143 0.00 MB/sec execute 2 sec latency 3993.202 ms 2023-08-25T09:01:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:57 smithi099 ceph-mon[97641]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-08-25T09:01:57.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:57 smithi157 ceph-mon[130879]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-08-25T09:01:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:57 smithi181 ceph-mon[131296]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-08-25T09:01:57.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68211 0.05 MB/sec execute 3 sec latency 4910.666 ms 2023-08-25T09:01:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 -- 172.21.15.99:0/2855865327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc103480 msgr2=0x7f4edc10f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/2855865327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc103480 0x7f4edc10f990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b0a0 tx=0x7f4ed400b3b0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 -- 172.21.15.99:0/2855865327 shutdown_connections 2023-08-25T09:01:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/2855865327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc103480 0x7f4edc10f990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/2855865327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc108ad0 0x7f4edc108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/2855865327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc102ac0 0x7f4edc102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 -- 172.21.15.99:0/2855865327 >> 172.21.15.99:0/2855865327 conn(0x7f4edc0fe570 msgr2=0x7f4edc100990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 -- 172.21.15.99:0/2855865327 shutdown_connections 2023-08-25T09:01:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.612+0000 7f4ee31ff700 1 -- 172.21.15.99:0/2855865327 wait complete. 2023-08-25T09:01:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.613+0000 7f4ee31ff700 1 Processor -- start 2023-08-25T09:01:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.613+0000 7f4ee31ff700 1 -- start start 2023-08-25T09:01:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.613+0000 7f4ee31ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc102ac0 0x7f4edc19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.613+0000 7f4ee31ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc103480 0x7f4edc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.613+0000 7f4ee31ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc108ad0 0x7f4edc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.613+0000 7f4ee31ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc198fe0 con 0x7f4edc108ad0 2023-08-25T09:01:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.613+0000 7f4ee31ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc199150 con 0x7f4edc103480 2023-08-25T09:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.613+0000 7f4ee31ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc199310 con 0x7f4edc102ac0 2023-08-25T09:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc108ad0 0x7f4edc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc108ad0 0x7f4edc198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36390/0 (socket says 172.21.15.99:36390) 2023-08-25T09:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 -- 172.21.15.99:0/3143430244 learned_addr learned my addr 172.21.15.99:0/3143430244 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc102ac0 0x7f4edc19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4edbfff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc103480 0x7f4edc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 -- 172.21.15.99:0/3143430244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc102ac0 msgr2=0x7f4edc19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc102ac0 0x7f4edc19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 -- 172.21.15.99:0/3143430244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc103480 msgr2=0x7f4edc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc103480 0x7f4edc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 -- 172.21.15.99:0/3143430244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4009d00 con 0x7f4edc108ad0 2023-08-25T09:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.614+0000 7f4ee179c700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc108ad0 0x7f4edc198970 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40062a0 tx=0x7f4ed4007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.615+0000 7f4ed9ffb700 1 -- 172.21.15.99:0/3143430244 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400e040 con 0x7f4edc108ad0 2023-08-25T09:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.615+0000 7f4ed9ffb700 1 -- 172.21.15.99:0/3143430244 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed40094c0 con 0x7f4edc108ad0 2023-08-25T09:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.615+0000 7f4ee31ff700 1 -- 172.21.15.99:0/3143430244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc199590 con 0x7f4edc108ad0 2023-08-25T09:01:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.615+0000 7f4ed9ffb700 1 -- 172.21.15.99:0/3143430244 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40052a0 con 0x7f4edc108ad0 2023-08-25T09:01:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.615+0000 7f4ee31ff700 1 -- 172.21.15.99:0/3143430244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1a5b00 con 0x7f4edc108ad0 2023-08-25T09:01:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.618+0000 7f4ed9ffb700 1 -- 172.21.15.99:0/3143430244 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f4ed401d470 con 0x7f4edc108ad0 2023-08-25T09:01:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.619+0000 7f4ef6b0ab80 1 Processor -- start 2023-08-25T09:01:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ef6b0ab80 1 -- start start 2023-08-25T09:01:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ed9ffb700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4ecc0a0140 0x7f4ecc0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ed9ffb700 1 -- 172.21.15.99:0/3143430244 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4ed40ccc30 con 0x7f4edc108ad0 2023-08-25T09:01:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4edbfff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4ecc0a0140 0x7f4ecc0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b038750 con 0x55dc2b0d6860 2023-08-25T09:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b029720 con 0x55dc2b0d5f40 2023-08-25T09:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b027040 con 0x55dc2b0d0990 2023-08-25T09:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ee0f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ee0f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48894/0 (socket says 172.21.15.99:48894) 2023-08-25T09:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/3880337950 learned_addr learned my addr 172.21.15.99:0/3880337950 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4edbfff700 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.620+0000 7f4ee179c700 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/3880337950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 msgr2=0x55dc2b0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/3880337950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 msgr2=0x55dc2b0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/3880337950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc2aea25d0 con 0x55dc2b0d5f40 2023-08-25T09:01:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ee179c700 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d6320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009d00 tx=0x7f4ed0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b575c20f365ace4e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4edbfff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4ecc0a0140 0x7f4ecc0a2600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009fd0 tx=0x7f4ec8009b40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ec6ffd700 1 -- 172.21.15.99:0/3880337950 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0004230 con 0x55dc2b0d5f40 2023-08-25T09:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ec6ffd700 1 -- 172.21.15.99:0/3880337950 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed0004390 con 0x55dc2b0d5f40 2023-08-25T09:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.621+0000 7f4ec6ffd700 1 -- 172.21.15.99:0/3880337950 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0019750 con 0x55dc2b0d5f40 2023-08-25T09:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/3880337950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 msgr2=0x55dc2b0d6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d6320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009d00 tx=0x7f4ed0011720 comp rx=0 tx=0).stop 2023-08-25T09:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/3880337950 shutdown_connections 2023-08-25T09:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d6320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/3880337950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/3880337950 >> 172.21.15.99:0/3880337950 conn(0x55dc2b045080 msgr2=0x55dc2b0cad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/3880337950 shutdown_connections 2023-08-25T09:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.622+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/3880337950 wait complete. 2023-08-25T09:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 1 -- start start 2023-08-25T09:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 1 Processor -- start 2023-08-25T09:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0dc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b022930 con 0x55dc2b0d5f40 2023-08-25T09:01:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2afc0b50 con 0x55dc2b0d6860 2023-08-25T09:01:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2afc0850 con 0x55dc2b0d0990 2023-08-25T09:01:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.623+0000 7f4ef6b0ab80 10 client.0 _ll_register_callbacks cb 0x55dc2af0aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4ee0f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0dc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4edbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4ee0f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0dc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:60112/0 (socket says 172.21.15.99:60112) 2023-08-25T09:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/1387548202 learned_addr learned my addr 172.21.15.99:0/1387548202 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4ee179c700 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4edbfff700 1 -- 172.21.15.99:0/1387548202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 msgr2=0x55dc2b0dc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4edbfff700 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0dc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4edbfff700 1 -- 172.21.15.99:0/1387548202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d6860 msgr2=0x55dc2b0d9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4edbfff700 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4edbfff700 1 -- 172.21.15.99:0/1387548202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed0009960 con 0x55dc2b0d5f40 2023-08-25T09:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4edbfff700 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d97d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80042b0 tx=0x7f4ec80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbd90ac45d5de89 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4ebbfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.624+0000 7f4ebbfff700 1 -- 172.21.15.99:0/1387548202 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8009850 con 0x55dc2b0d5f40 2023-08-25T09:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1387548202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc2b0a0160 con 0x55dc2b0d5f40 2023-08-25T09:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ebbfff700 1 -- 172.21.15.99:0/1387548202 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4ec8011e40 con 0x55dc2b0d5f40 2023-08-25T09:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ebbfff700 1 -- 172.21.15.99:0/1387548202 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ec8003680 con 0x55dc2b0d5f40 2023-08-25T09:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 10 client.15441 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1387548202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dc2ae4ed80 con 0x55dc2b0d5f40 2023-08-25T09:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ebbfff700 1 -- 172.21.15.99:0/1387548202 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc2b0a0160 con 0x55dc2b0d5f40 2023-08-25T09:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ebbfff700 1 -- 172.21.15.99:0/1387548202 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4ec8030070 con 0x55dc2b0d5f40 2023-08-25T09:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 10 client.15441 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 20 client.15441 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55dc2b0ea1b0 0x55dc2b0ec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 4 client.15441 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.625+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1387548202 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc2afc8b40 con 0x55dc2b0ea1b0 2023-08-25T09:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.626+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55dc2b0ea1b0 0x55dc2b0ec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.626+0000 7f4ebbfff700 10 client.15441 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.626+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55dc2b0ea1b0 0x55dc2b0ec5d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2174b45ce4081dda server_cookie=84607eb3a4bd511 in_seq=0 out_seq=0 2023-08-25T09:01:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ebbfff700 1 -- 172.21.15.99:0/1387548202 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc2afc8b40 con 0x55dc2b0ea1b0 2023-08-25T09:01:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ebbfff700 10 client.15441 handle_command_reply: tid=0 2023-08-25T09:01:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 client.15441 shutdown 2023-08-25T09:01:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1387548202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 msgr2=0x55dc2b0d97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d97d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80042b0 tx=0x7f4ec80047c0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1387548202 shutdown_connections 2023-08-25T09:01:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55dc2b0ea1b0 0x55dc2b0ec5d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1387548202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d97d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ebbfff700 0 client.15441 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1387548202 >> 172.21.15.99:0/1387548202 conn(0x55dc2b045080 msgr2=0x55dc2b0c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1387548202 shutdown_connections 2023-08-25T09:01:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1387548202 wait complete. 2023-08-25T09:01:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.627+0000 7f4ef6b0ab80 20 client.15441 trim_cache size 0 max 16384 2023-08-25T09:01:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.642+0000 7f4ef6b0ab80 1 Processor -- start 2023-08-25T09:01:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.642+0000 7f4ef6b0ab80 1 -- start start 2023-08-25T09:01:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.642+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.642+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.642+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.642+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b0d1290 con 0x55dc2b0d5f40 2023-08-25T09:01:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.642+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2aea48c0 con 0x55dc2b0d0990 2023-08-25T09:01:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.642+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b0c7700 con 0x55dc2b0d6860 2023-08-25T09:01:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36426/0 (socket says 172.21.15.99:36426) 2023-08-25T09:01:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 -- 172.21.15.99:0/1454481712 learned_addr learned my addr 172.21.15.99:0/1454481712 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4ee179c700 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 -- 172.21.15.99:0/1454481712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d6860 msgr2=0x55dc2b0d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 -- 172.21.15.99:0/1454481712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d0990 msgr2=0x55dc2b0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 -- 172.21.15.99:0/1454481712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc2ae37c00 con 0x55dc2b0d5f40 2023-08-25T09:01:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:01:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.643+0000 7f4edbfff700 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d3950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4ec802a9c0 tx=0x7f4ec8004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8026bf90e034dc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ec77fe700 1 -- 172.21.15.99:0/1454481712 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8011e40 con 0x55dc2b0d5f40 2023-08-25T09:01:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ec77fe700 1 -- 172.21.15.99:0/1454481712 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec8009830 con 0x55dc2b0d5f40 2023-08-25T09:01:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ec77fe700 1 -- 172.21.15.99:0/1454481712 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8003b40 con 0x55dc2b0d5f40 2023-08-25T09:01:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1454481712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 msgr2=0x55dc2b0d3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d3950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4ec802a9c0 tx=0x7f4ec8004720 comp rx=0 tx=0).stop 2023-08-25T09:01:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1454481712 shutdown_connections 2023-08-25T09:01:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d6860 0x55dc2b0d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d0990 0x55dc2b0d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/1454481712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0d3950 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1454481712 >> 172.21.15.99:0/1454481712 conn(0x55dc2b045080 msgr2=0x55dc2b0e59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1454481712 shutdown_connections 2023-08-25T09:01:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.644+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/1454481712 wait complete. 2023-08-25T09:01:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 1 -- start start 2023-08-25T09:01:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 1 Processor -- start 2023-08-25T09:01:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b13d520 con 0x55dc2b0d6860 2023-08-25T09:01:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b13d660 con 0x55dc2b0d5f40 2023-08-25T09:01:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2b13d820 con 0x55dc2b0d0990 2023-08-25T09:01:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.645+0000 7f4ef6b0ab80 10 client.0 _ll_register_callbacks cb 0x55dc2af0aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:01:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0df450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:60128/0 (socket says 172.21.15.99:60128) 2023-08-25T09:01:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/2386569204 learned_addr learned my addr 172.21.15.99:0/2386569204 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:01:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee179c700 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4edbfff700 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/2386569204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 msgr2=0x55dc2b0df990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0df990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/2386569204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 msgr2=0x55dc2b13cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:01:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b13cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 -- 172.21.15.99:0/2386569204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc2aea25d0 con 0x55dc2b0d0990 2023-08-25T09:01:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee179c700 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b13cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:01:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ee0f9b700 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0df450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0003e10 tx=0x7f4ed0003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a322d80afca2c8b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:01:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ebbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:01:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ebbfff700 1 -- 172.21.15.99:0/2386569204 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0029040 con 0x55dc2b0d0990 2023-08-25T09:01:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/2386569204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc2b0c3600 con 0x55dc2b0d0990 2023-08-25T09:01:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.646+0000 7f4ebbfff700 1 -- 172.21.15.99:0/2386569204 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4ed00040f0 con 0x55dc2b0d0990 2023-08-25T09:01:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ebbfff700 1 -- 172.21.15.99:0/2386569204 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed00042b0 con 0x55dc2b0d0990 2023-08-25T09:01:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ef6b0ab80 10 client.25486 fetch_fsmap learned FSMap version 25 2023-08-25T09:01:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/2386569204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dc2ae4ed80 con 0x55dc2b0d0990 2023-08-25T09:01:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ebbfff700 1 -- 172.21.15.99:0/2386569204 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc2b0c3600 con 0x55dc2b0d0990 2023-08-25T09:01:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ebbfff700 1 -- 172.21.15.99:0/2386569204 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4ed002c070 con 0x55dc2b0d0990 2023-08-25T09:01:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ef6b0ab80 10 client.25486 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:01:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ef6b0ab80 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:01:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ef6b0ab80 20 client.25486 populate_metadata read hostname 'smithi099' 2023-08-25T09:01:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55dc2b13ef10 0x55dc2b141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:01:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ef6b0ab80 4 client.25486 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:01:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.647+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/2386569204 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc2b01b910 con 0x55dc2b13ef10 2023-08-25T09:01:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.648+0000 7f4edbfff700 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55dc2b13ef10 0x55dc2b141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:01:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.648+0000 7f4edbfff700 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55dc2b13ef10 0x55dc2b141330 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f4395d5df6e7d7 server_cookie=99b548f2b2bdd874 in_seq=0 out_seq=0 2023-08-25T09:01:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.648+0000 7f4ebbfff700 10 client.25486 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.681+0000 7f4ebbfff700 1 -- 172.21.15.99:0/2386569204 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc2b01b910 con 0x55dc2b13ef10 2023-08-25T09:01:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.681+0000 7f4ebbfff700 10 client.25486 handle_command_reply: tid=0 2023-08-25T09:01:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.681+0000 7f4ef6b0ab80 1 client.25486 shutdown 2023-08-25T09:01:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.681+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/2386569204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 msgr2=0x55dc2b0df450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.681+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0df450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0003e10 tx=0x7f4ed0003730 comp rx=0 tx=0).stop 2023-08-25T09:01:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/2386569204 shutdown_connections 2023-08-25T09:01:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc2b0d6860 0x55dc2b13cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc2b0d0990 0x55dc2b0df450 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55dc2b13ef10 0x55dc2b141330 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 1 --2- 172.21.15.99:0/2386569204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc2b0d5f40 0x55dc2b0df990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ebbfff700 0 client.25486 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:01:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/2386569204 >> 172.21.15.99:0/2386569204 conn(0x55dc2b045080 msgr2=0x55dc2b0d3950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/2386569204 shutdown_connections 2023-08-25T09:01:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 1 -- 172.21.15.99:0/2386569204 wait complete. 2023-08-25T09:01:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.682+0000 7f4ef6b0ab80 20 client.25486 trim_cache size 0 max 16384 2023-08-25T09:01:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:01:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:01:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:01:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:01:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 -- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4ecc0a0140 msgr2=0x7f4ecc0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4ecc0a0140 0x7f4ecc0a2600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009fd0 tx=0x7f4ec8009b40 comp rx=0 tx=0).stop 2023-08-25T09:01:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 -- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc108ad0 msgr2=0x7f4edc198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:01:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc108ad0 0x7f4edc198970 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40062a0 tx=0x7f4ed4007870 comp rx=0 tx=0).stop 2023-08-25T09:01:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 -- 172.21.15.99:0/3143430244 shutdown_connections 2023-08-25T09:01:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4ecc0a0140 0x7f4ecc0a2600 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4edc108ad0 0x7f4edc198970 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4edc102ac0 0x7f4edc19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 --2- 172.21.15.99:0/3143430244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4edc103480 0x7f4edc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:01:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 -- 172.21.15.99:0/3143430244 >> 172.21.15.99:0/3143430244 conn(0x7f4edc0fe570 msgr2=0x7f4edc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:01:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 -- 172.21.15.99:0/3143430244 shutdown_connections 2023-08-25T09:01:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:01:58.683+0000 7f4ee31ff700 1 -- 172.21.15.99:0/3143430244 wait complete. 2023-08-25T09:01:58.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68211 0.04 MB/sec execute 4 sec latency 1033.611 ms 2023-08-25T09:01:59.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:01:59 smithi099 ceph-mon[97641]: pgmap v87: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2023-08-25T09:01:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:01:59 smithi157 ceph-mon[130879]: pgmap v87: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2023-08-25T09:01:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:01:59 smithi181 ceph-mon[131296]: pgmap v87: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2023-08-25T09:01:59.896 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68211 0.03 MB/sec execute 5 sec latency 2033.704 ms 2023-08-25T09:02:00.897 INFO:tasks.workunit.client.0.smithi099.stdout: 1 68211 0.03 MB/sec execute 6 sec latency 3033.826 ms 2023-08-25T09:02:01.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:01 smithi099 ceph-mon[97641]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2023-08-25T09:02:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:01 smithi157 ceph-mon[130879]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2023-08-25T09:02:01.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:01 smithi181 ceph-mon[131296]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2023-08-25T09:02:01.897 INFO:tasks.workunit.client.0.smithi099.stdout: 1 69755 0.95 MB/sec execute 7 sec latency 3099.325 ms 2023-08-25T09:02:02.897 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71363 2.36 MB/sec execute 8 sec latency 17.385 ms 2023-08-25T09:02:03.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:03 smithi099 ceph-mon[97641]: pgmap v89: 97 pgs: 97 active+clean; 39 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 142 op/s 2023-08-25T09:02:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:03 smithi157 ceph-mon[130879]: pgmap v89: 97 pgs: 97 active+clean; 39 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 142 op/s 2023-08-25T09:02:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:03 smithi181 ceph-mon[131296]: pgmap v89: 97 pgs: 97 active+clean; 39 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 142 op/s 2023-08-25T09:02:03.897 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71629 2.11 MB/sec execute 9 sec latency 733.377 ms 2023-08-25T09:02:04.897 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71629 1.90 MB/sec execute 10 sec latency 1733.510 ms 2023-08-25T09:02:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:05 smithi157 ceph-mon[130879]: pgmap v90: 97 pgs: 97 active+clean; 39 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 681 KiB/s wr, 73 op/s 2023-08-25T09:02:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:05 smithi181 ceph-mon[131296]: pgmap v90: 97 pgs: 97 active+clean; 39 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 681 KiB/s wr, 73 op/s 2023-08-25T09:02:05.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:05 smithi099 ceph-mon[97641]: pgmap v90: 97 pgs: 97 active+clean; 39 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 681 KiB/s wr, 73 op/s 2023-08-25T09:02:05.897 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71629 1.73 MB/sec execute 11 sec latency 2733.639 ms 2023-08-25T09:02:06.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71629 1.58 MB/sec execute 12 sec latency 3733.796 ms 2023-08-25T09:02:07.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:07 smithi099 ceph-mon[97641]: pgmap v91: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-08-25T09:02:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:07 smithi157 ceph-mon[130879]: pgmap v91: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-08-25T09:02:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:07 smithi181 ceph-mon[131296]: pgmap v91: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-08-25T09:02:07.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71685 1.47 MB/sec execute 13 sec latency 4652.375 ms 2023-08-25T09:02:08.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71685 1.37 MB/sec execute 14 sec latency 1041.064 ms 2023-08-25T09:02:09.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:09 smithi099 ceph-mon[97641]: pgmap v92: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 97 op/s 2023-08-25T09:02:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:09 smithi157 ceph-mon[130879]: pgmap v92: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 97 op/s 2023-08-25T09:02:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:09 smithi181 ceph-mon[131296]: pgmap v92: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 97 op/s 2023-08-25T09:02:09.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71685 1.27 MB/sec execute 15 sec latency 2041.113 ms 2023-08-25T09:02:10.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71685 1.19 MB/sec execute 16 sec latency 3041.238 ms 2023-08-25T09:02:11.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:11 smithi099 ceph-mon[97641]: pgmap v93: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 97 op/s 2023-08-25T09:02:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:11 smithi157 ceph-mon[130879]: pgmap v93: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 97 op/s 2023-08-25T09:02:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:11 smithi181 ceph-mon[131296]: pgmap v93: 97 pgs: 97 active+clean; 40 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 97 op/s 2023-08-25T09:02:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 -- 172.21.15.99:0/2249014308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc501008a0 msgr2=0x7fcc50100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2249014308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc501008a0 0x7fcc50100d20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009990 tx=0x7fcc3c009ca0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 -- 172.21.15.99:0/2249014308 shutdown_connections 2023-08-25T09:02:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2249014308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc50101260 0x7fcc5010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2249014308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc501068b0 0x7fcc50106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2249014308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc501008a0 0x7fcc50100d20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 -- 172.21.15.99:0/2249014308 >> 172.21.15.99:0/2249014308 conn(0x7fcc500fc370 msgr2=0x7fcc500fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 -- 172.21.15.99:0/2249014308 shutdown_connections 2023-08-25T09:02:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.838+0000 7fcc56bff700 1 -- 172.21.15.99:0/2249014308 wait complete. 2023-08-25T09:02:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.839+0000 7fcc56bff700 1 Processor -- start 2023-08-25T09:02:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.839+0000 7fcc56bff700 1 -- start start 2023-08-25T09:02:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.839+0000 7fcc56bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc501008a0 0x7fcc5019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.839+0000 7fcc56bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50101260 0x7fcc5019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.839+0000 7fcc56bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc501068b0 0x7fcc501989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.839+0000 7fcc56bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50199050 con 0x7fcc50101260 2023-08-25T09:02:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.839+0000 7fcc56bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501991c0 con 0x7fcc501068b0 2023-08-25T09:02:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.839+0000 7fcc56bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50199380 con 0x7fcc501008a0 2023-08-25T09:02:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc4ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50101260 0x7fcc5019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc501008a0 0x7fcc5019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc501008a0 0x7fcc5019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:44292/0 (socket says 172.21.15.99:44292) 2023-08-25T09:02:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 -- 172.21.15.99:0/2757256835 learned_addr learned my addr 172.21.15.99:0/2757256835 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5519c700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc501068b0 0x7fcc501989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 -- 172.21.15.99:0/2757256835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc501068b0 msgr2=0x7fcc501989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc501068b0 0x7fcc501989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 -- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50101260 msgr2=0x7fcc5019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50101260 0x7fcc5019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 -- 172.21.15.99:0/2757256835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c0095f0 con 0x7fcc501008a0 2023-08-25T09:02:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5519c700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc501068b0 0x7fcc501989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:02:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.840+0000 7fcc5499b700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc501008a0 0x7fcc5019eb80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400e9a0 tx=0x7fcc4400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.841+0000 7fcc4dffb700 1 -- 172.21.15.99:0/2757256835 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44009790 con 0x7fcc501008a0 2023-08-25T09:02:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.841+0000 7fcc56bff700 1 -- 172.21.15.99:0/2757256835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc50199660 con 0x7fcc501008a0 2023-08-25T09:02:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.841+0000 7fcc4dffb700 1 -- 172.21.15.99:0/2757256835 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc4400cbd0 con 0x7fcc501008a0 2023-08-25T09:02:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.841+0000 7fcc4dffb700 1 -- 172.21.15.99:0/2757256835 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44018680 con 0x7fcc501008a0 2023-08-25T09:02:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.841+0000 7fcc56bff700 1 -- 172.21.15.99:0/2757256835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc501a5b70 con 0x7fcc501008a0 2023-08-25T09:02:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.844+0000 7fcc4dffb700 1 -- 172.21.15.99:0/2757256835 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fcc44010430 con 0x7fcc501008a0 2023-08-25T09:02:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.845+0000 7fcc6a5c0b80 1 Processor -- start 2023-08-25T09:02:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.845+0000 7fcc6a5c0b80 1 -- start start 2023-08-25T09:02:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5df40 0x55f59fa5e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 0x55f59fa58450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59f9c0750 con 0x55f59fa5df40 2023-08-25T09:02:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59f9b1720 con 0x55f59fa58990 2023-08-25T09:02:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59f9af040 con 0x55f59fa5e860 2023-08-25T09:02:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc5519c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5df40 0x55f59fa5e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc5499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 0x55f59fa58450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc5519c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5df40 0x55f59fa5e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33698/0 (socket says 172.21.15.99:33698) 2023-08-25T09:02:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc5519c700 1 -- 172.21.15.99:0/3203558395 learned_addr learned my addr 172.21.15.99:0/3203558395 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc4dffb700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fcc400a0140 0x7fcc400a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc4ffff700 1 -- 172.21.15.99:0/3203558395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 msgr2=0x55f59fa58450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 0x55f59fa58450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc4ffff700 1 -- 172.21.15.99:0/3203558395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5df40 msgr2=0x55f59fa5e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5df40 0x55f59fa5e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc4ffff700 1 -- 172.21.15.99:0/3203558395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f59f82a5d0 con 0x55f59fa58990 2023-08-25T09:02:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.846+0000 7fcc5519c700 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5df40 0x55f59fa5e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:02:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.847+0000 7fcc4dffb700 1 -- 172.21.15.99:0/2757256835 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcc44014070 con 0x7fcc501008a0 2023-08-25T09:02:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.847+0000 7fcc5519c700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fcc400a0140 0x7fcc400a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.847+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58dd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00b2e0 tx=0x7fcc3c005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6993a2dd91062111 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.847+0000 7fcc3affd700 1 -- 172.21.15.99:0/3203558395 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c01d070 con 0x55f59fa58990 2023-08-25T09:02:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.847+0000 7fcc3affd700 1 -- 172.21.15.99:0/3203558395 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc3c01e040 con 0x55f59fa58990 2023-08-25T09:02:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.847+0000 7fcc3affd700 1 -- 172.21.15.99:0/3203558395 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0047f0 con 0x55f59fa58990 2023-08-25T09:02:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/3203558395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 msgr2=0x55f59fa58dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc5519c700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fcc400a0140 0x7fcc400a2600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcc480094f0 tx=0x7fcc48004ad0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58dd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00b2e0 tx=0x7fcc3c005ea0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/3203558395 shutdown_connections 2023-08-25T09:02:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58dd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5df40 0x55f59fa5e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/3203558395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 0x55f59fa58450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/3203558395 >> 172.21.15.99:0/3203558395 conn(0x55f59f9cd080 msgr2=0x55f59fa52d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/3203558395 shutdown_connections 2023-08-25T09:02:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/3203558395 wait complete. 2023-08-25T09:02:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.848+0000 7fcc6a5c0b80 1 -- start start 2023-08-25T09:02:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc6a5c0b80 1 Processor -- start 2023-08-25T09:02:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa64690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5df40 0x55f59fa618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 0x55f59fa61eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59f9aa930 con 0x55f59fa58990 2023-08-25T09:02:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59f948b50 con 0x55f59fa5df40 2023-08-25T09:02:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59f948850 con 0x55f59fa5e860 2023-08-25T09:02:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc6a5c0b80 10 client.0 _ll_register_callbacks cb 0x55f59f892a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:02:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc5499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa64690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc4ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5df40 0x55f59fa618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc5499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa64690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33712/0 (socket says 172.21.15.99:33712) 2023-08-25T09:02:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.849+0000 7fcc5499b700 1 -- 172.21.15.99:0/337830653 learned_addr learned my addr 172.21.15.99:0/337830653 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc5519c700 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 0x55f59fa61eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc5499b700 1 -- 172.21.15.99:0/337830653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 msgr2=0x55f59fa61eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc5499b700 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 0x55f59fa61eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc5499b700 1 -- 172.21.15.99:0/337830653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5df40 msgr2=0x55f59fa618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc5499b700 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5df40 0x55f59fa618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc5499b700 1 -- 172.21.15.99:0/337830653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc480090e0 con 0x55f59fa58990 2023-08-25T09:02:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5df40 0x55f59fa618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:02:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc5499b700 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa64690 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcc440953f0 tx=0x7fcc44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc329c6f6cb1a8ad server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:02:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc2ffff700 1 -- 172.21.15.99:0/337830653 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440a0a70 con 0x55f59fa58990 2023-08-25T09:02:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/337830653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f59fa28160 con 0x55f59fa58990 2023-08-25T09:02:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc2ffff700 1 -- 172.21.15.99:0/337830653 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcc440995f0 con 0x55f59fa58990 2023-08-25T09:02:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc2ffff700 1 -- 172.21.15.99:0/337830653 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc44098ef0 con 0x55f59fa58990 2023-08-25T09:02:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc6a5c0b80 10 client.15459 fetch_fsmap learned FSMap version 25 2023-08-25T09:02:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.850+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/337830653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fcc5010e070 con 0x55f59fa58990 2023-08-25T09:02:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc2ffff700 1 -- 172.21.15.99:0/337830653 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc440b0070 con 0x55f59fa58990 2023-08-25T09:02:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc2ffff700 1 -- 172.21.15.99:0/337830653 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcc440cdbb0 con 0x55f59fa58990 2023-08-25T09:02:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc6a5c0b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:02:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc6a5c0b80 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:02:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc6a5c0b80 20 client.15459 populate_metadata read hostname 'smithi099' 2023-08-25T09:02:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f59fa72130 0x55f59fa74550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc6a5c0b80 4 client.15459 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:02:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/337830653 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f59f950b40 con 0x55f59fa72130 2023-08-25T09:02:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.851+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f59fa72130 0x55f59fa74550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.852+0000 7fcc2ffff700 10 client.15459 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.852+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f59fa72130 0x55f59fa74550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c67f20066e2946f server_cookie=240a7e69c9b92f43 in_seq=0 out_seq=0 2023-08-25T09:02:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.852+0000 7fcc2ffff700 1 -- 172.21.15.99:0/337830653 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f59f950b40 con 0x55f59fa72130 2023-08-25T09:02:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc2ffff700 10 client.15459 handle_command_reply: tid=0 2023-08-25T09:02:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 client.15459 shutdown 2023-08-25T09:02:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/337830653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 msgr2=0x55f59fa64690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa64690 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcc440953f0 tx=0x7fcc44000bc0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/337830653 shutdown_connections 2023-08-25T09:02:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5e860 0x55f59fa61eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa64690 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f59fa72130 0x55f59fa74550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/337830653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5df40 0x55f59fa618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/337830653 >> 172.21.15.99:0/337830653 conn(0x55f59f9cd080 msgr2=0x55f59fa4d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc2ffff700 0 client.15459 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/337830653 shutdown_connections 2023-08-25T09:02:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/337830653 wait complete. 2023-08-25T09:02:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.853+0000 7fcc6a5c0b80 20 client.15459 trim_cache size 0 max 16384 2023-08-25T09:02:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.868+0000 7fcc6a5c0b80 1 Processor -- start 2023-08-25T09:02:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.868+0000 7fcc6a5c0b80 1 -- start start 2023-08-25T09:02:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.868+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.868+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.868+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5e860 0x55f59fa641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59f82c8c0 con 0x55f59fa5e860 2023-08-25T09:02:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59fa53250 con 0x55f59fa58990 2023-08-25T09:02:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59fa69780 con 0x55f59fa5df40 2023-08-25T09:02:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc4ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc4ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa69240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:44312/0 (socket says 172.21.15.99:44312) 2023-08-25T09:02:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc4ffff700 1 -- 172.21.15.99:0/1388186450 learned_addr learned my addr 172.21.15.99:0/1388186450 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc5499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc5519c700 1 --2- 172.21.15.99:0/1388186450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5e860 0x55f59fa641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc4ffff700 1 -- 172.21.15.99:0/1388186450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 msgr2=0x55f59fa58450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/1388186450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc4ffff700 1 -- 172.21.15.99:0/1388186450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5e860 msgr2=0x55f59fa641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/1388186450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5e860 0x55f59fa641c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.869+0000 7fcc4ffff700 1 -- 172.21.15.99:0/1388186450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f59f7d6d80 con 0x55f59fa5df40 2023-08-25T09:02:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/1388186450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa69240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c005b20 tx=0x7fcc3c01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83882d20b476ff5b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc3b7fe700 1 -- 172.21.15.99:0/1388186450 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c01d040 con 0x55f59fa5df40 2023-08-25T09:02:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc3b7fe700 1 -- 172.21.15.99:0/1388186450 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc3c01e040 con 0x55f59fa5df40 2023-08-25T09:02:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc3b7fe700 1 -- 172.21.15.99:0/1388186450 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c033bb0 con 0x55f59fa5df40 2023-08-25T09:02:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/1388186450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 msgr2=0x55f59fa69240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/1388186450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa69240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c005b20 tx=0x7fcc3c01a040 comp rx=0 tx=0).stop 2023-08-25T09:02:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/1388186450 shutdown_connections 2023-08-25T09:02:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/1388186450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa5e860 0x55f59fa641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/1388186450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa58990 0x55f59fa58450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/1388186450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa69240 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/1388186450 >> 172.21.15.99:0/1388186450 conn(0x55f59f9cd080 msgr2=0x55f59fa5b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.870+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/1388186450 shutdown_connections 2023-08-25T09:02:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.871+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/1388186450 wait complete. 2023-08-25T09:02:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.871+0000 7fcc6a5c0b80 1 -- start start 2023-08-25T09:02:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.871+0000 7fcc6a5c0b80 1 Processor -- start 2023-08-25T09:02:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.871+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa59880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc6a5c0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5e860 0x55f59fa5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59fa5aa40 con 0x55f59fa58990 2023-08-25T09:02:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59fa5abb0 con 0x55f59fa5e860 2023-08-25T09:02:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc6a5c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59fa5ad70 con 0x55f59fa5df40 2023-08-25T09:02:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc6a5c0b80 10 client.0 _ll_register_callbacks cb 0x55f59f892a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:02:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa59880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5519c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5e860 0x55f59fa5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa59880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33722/0 (socket says 172.21.15.99:33722) 2023-08-25T09:02:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 -- 172.21.15.99:0/516587669 learned_addr learned my addr 172.21.15.99:0/516587669 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 -- 172.21.15.99:0/516587669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 msgr2=0x55f59fa59dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa59dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 -- 172.21.15.99:0/516587669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5e860 msgr2=0x55f59fa5a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5e860 0x55f59fa5a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 -- 172.21.15.99:0/516587669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f59f82a5d0 con 0x55f59fa58990 2023-08-25T09:02:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc5499b700 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa59880 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcc440cdee0 tx=0x7fcc44005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c43ca95fc440210 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.872+0000 7fcc2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:02:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc2ffff700 1 -- 172.21.15.99:0/516587669 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440995f0 con 0x55f59fa58990 2023-08-25T09:02:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc2ffff700 1 -- 172.21.15.99:0/516587669 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcc44098ef0 con 0x55f59fa58990 2023-08-25T09:02:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/516587669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f59fa4b600 con 0x55f59fa58990 2023-08-25T09:02:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc2ffff700 1 -- 172.21.15.99:0/516587669 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc4409cd20 con 0x55f59fa58990 2023-08-25T09:02:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 10 client.15465 fetch_fsmap learned FSMap version 25 2023-08-25T09:02:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/516587669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f59f7bfc00 con 0x55f59fa58990 2023-08-25T09:02:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc2ffff700 1 -- 172.21.15.99:0/516587669 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f59fa4b600 con 0x55f59fa58990 2023-08-25T09:02:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc2ffff700 1 -- 172.21.15.99:0/516587669 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcc44097e50 con 0x55f59fa58990 2023-08-25T09:02:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:02:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:02:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 20 client.15465 populate_metadata read hostname 'smithi099' 2023-08-25T09:02:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f59fac6ab0 0x55f59fac8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 4 client.15465 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:02:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.873+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/516587669 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f59f9a3910 con 0x55f59fac6ab0 2023-08-25T09:02:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.874+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f59fac6ab0 0x55f59fac8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.874+0000 7fcc2ffff700 10 client.15465 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.874+0000 7fcc4ffff700 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f59fac6ab0 0x55f59fac8ed0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d31e7f17a1dffe8 server_cookie=c331ccbc682cbb0d in_seq=0 out_seq=0 2023-08-25T09:02:11.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71685 1.12 MB/sec execute 17 sec latency 4041.310 ms 2023-08-25T09:02:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc2ffff700 1 -- 172.21.15.99:0/516587669 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f59f9a3910 con 0x55f59fac6ab0 2023-08-25T09:02:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc2ffff700 10 client.15465 handle_command_reply: tid=0 2023-08-25T09:02:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 client.15465 shutdown 2023-08-25T09:02:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/516587669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 msgr2=0x55f59fa59880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa59880 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcc440cdee0 tx=0x7fcc44005780 comp rx=0 tx=0).stop 2023-08-25T09:02:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/516587669 shutdown_connections 2023-08-25T09:02:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f59fac6ab0 0x55f59fac8ed0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f59fa5e860 0x55f59fa5a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f59fa58990 0x55f59fa59880 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc2ffff700 0 client.15465 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 --2- 172.21.15.99:0/516587669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59fa5df40 0x55f59fa59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/516587669 >> 172.21.15.99:0/516587669 conn(0x55f59f9cd080 msgr2=0x55f59fa61160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/516587669 shutdown_connections 2023-08-25T09:02:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.903+0000 7fcc6a5c0b80 1 -- 172.21.15.99:0/516587669 wait complete. 2023-08-25T09:02:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc6a5c0b80 20 client.15465 trim_cache size 0 max 16384 2023-08-25T09:02:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:02:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:02:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:02:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:02:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 -- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fcc400a0140 msgr2=0x7fcc400a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fcc400a0140 0x7fcc400a2600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcc480094f0 tx=0x7fcc48004ad0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 -- 172.21.15.99:0/2757256835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc501008a0 msgr2=0x7fcc5019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc501008a0 0x7fcc5019eb80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400e9a0 tx=0x7fcc4400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 -- 172.21.15.99:0/2757256835 shutdown_connections 2023-08-25T09:02:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fcc400a0140 0x7fcc400a2600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc501068b0 0x7fcc501989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc501008a0 0x7fcc5019eb80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 --2- 172.21.15.99:0/2757256835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50101260 0x7fcc5019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 -- 172.21.15.99:0/2757256835 >> 172.21.15.99:0/2757256835 conn(0x7fcc500fc370 msgr2=0x7fcc5010bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 -- 172.21.15.99:0/2757256835 shutdown_connections 2023-08-25T09:02:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:11.904+0000 7fcc56bff700 1 -- 172.21.15.99:0/2757256835 wait complete. 2023-08-25T09:02:12.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71753 1.07 MB/sec execute 18 sec latency 4958.516 ms 2023-08-25T09:02:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:13 smithi099 ceph-mon[97641]: pgmap v94: 97 pgs: 97 active+clean; 41 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 728 KiB/s wr, 108 op/s 2023-08-25T09:02:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:13 smithi157 ceph-mon[130879]: pgmap v94: 97 pgs: 97 active+clean; 41 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 728 KiB/s wr, 108 op/s 2023-08-25T09:02:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:13 smithi181 ceph-mon[131296]: pgmap v94: 97 pgs: 97 active+clean; 41 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 728 KiB/s wr, 108 op/s 2023-08-25T09:02:13.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71753 1.01 MB/sec execute 19 sec latency 1029.896 ms 2023-08-25T09:02:14.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71753 0.96 MB/sec execute 20 sec latency 2030.031 ms 2023-08-25T09:02:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:15 smithi157 ceph-mon[130879]: pgmap v95: 97 pgs: 97 active+clean; 41 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 601 KiB/s wr, 64 op/s 2023-08-25T09:02:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:15 smithi181 ceph-mon[131296]: pgmap v95: 97 pgs: 97 active+clean; 41 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 601 KiB/s wr, 64 op/s 2023-08-25T09:02:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:15 smithi099 ceph-mon[97641]: pgmap v95: 97 pgs: 97 active+clean; 41 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 601 KiB/s wr, 64 op/s 2023-08-25T09:02:15.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71753 0.92 MB/sec execute 21 sec latency 3030.150 ms 2023-08-25T09:02:16.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71753 0.88 MB/sec execute 22 sec latency 4030.257 ms 2023-08-25T09:02:17.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:17 smithi099 ceph-mon[97641]: pgmap v96: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-08-25T09:02:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:17 smithi157 ceph-mon[130879]: pgmap v96: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-08-25T09:02:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:17 smithi181 ceph-mon[131296]: pgmap v96: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-08-25T09:02:17.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71876 0.85 MB/sec execute 23 sec latency 4945.912 ms 2023-08-25T09:02:18.898 INFO:tasks.workunit.client.0.smithi099.stdout: 1 74277 1.39 MB/sec execute 24 sec latency 13.808 ms 2023-08-25T09:02:19.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:19 smithi099 ceph-mon[97641]: pgmap v97: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 625 KiB/s wr, 40 op/s 2023-08-25T09:02:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:19 smithi157 ceph-mon[130879]: pgmap v97: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 625 KiB/s wr, 40 op/s 2023-08-25T09:02:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:19 smithi181 ceph-mon[131296]: pgmap v97: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 625 KiB/s wr, 40 op/s 2023-08-25T09:02:19.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 75210 1.53 MB/sec execute 25 sec latency 140.850 ms 2023-08-25T09:02:20.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 75210 1.47 MB/sec execute 26 sec latency 1140.975 ms 2023-08-25T09:02:21.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:21 smithi099 ceph-mon[97641]: pgmap v98: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 625 KiB/s wr, 40 op/s 2023-08-25T09:02:21.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:21 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:02:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:21 smithi157 ceph-mon[130879]: pgmap v98: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 625 KiB/s wr, 40 op/s 2023-08-25T09:02:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:21 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:02:21.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:21 smithi181 ceph-mon[131296]: pgmap v98: 97 pgs: 97 active+clean; 47 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 625 KiB/s wr, 40 op/s 2023-08-25T09:02:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:21 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:02:21.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 75210 1.41 MB/sec execute 27 sec latency 2141.066 ms 2023-08-25T09:02:22.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:22 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:22 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:22 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:02:22.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:22 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:02:22.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:22 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:22 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:22 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:22 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:02:22.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:22 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:02:22.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:22 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:22 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:22 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:22 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:02:22.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:22 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:02:22.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:22 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:02:22.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 75333 1.37 MB/sec execute 28 sec latency 3056.760 ms 2023-08-25T09:02:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:23 smithi157 ceph-mon[130879]: pgmap v99: 97 pgs: 97 active+clean; 43 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 876 KiB/s wr, 85 op/s 2023-08-25T09:02:23.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:23 smithi181 ceph-mon[131296]: pgmap v99: 97 pgs: 97 active+clean; 43 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 876 KiB/s wr, 85 op/s 2023-08-25T09:02:23.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:23 smithi099 ceph-mon[97641]: pgmap v99: 97 pgs: 97 active+clean; 43 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 876 KiB/s wr, 85 op/s 2023-08-25T09:02:23.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 77717 1.77 MB/sec execute 29 sec latency 41.434 ms 2023-08-25T09:02:24.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78809 1.91 MB/sec execute 30 sec latency 61.783 ms 2023-08-25T09:02:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 -- 172.21.15.99:0/1708863232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1008a0 msgr2=0x7f0c9c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/1708863232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1008a0 0x7f0c9c100d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0098f0 tx=0x7f0c8c009c00 comp rx=0 tx=0).stop 2023-08-25T09:02:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 -- 172.21.15.99:0/1708863232 shutdown_connections 2023-08-25T09:02:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/1708863232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c101260 0x7f0c9c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/1708863232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c9c1068b0 0x7f0c9c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/1708863232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1008a0 0x7f0c9c100d20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 -- 172.21.15.99:0/1708863232 >> 172.21.15.99:0/1708863232 conn(0x7f0c9c0fc370 msgr2=0x7f0c9c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 -- 172.21.15.99:0/1708863232 shutdown_connections 2023-08-25T09:02:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.046+0000 7f0ca33ff700 1 -- 172.21.15.99:0/1708863232 wait complete. 2023-08-25T09:02:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.047+0000 7f0ca33ff700 1 Processor -- start 2023-08-25T09:02:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.047+0000 7f0ca33ff700 1 -- start start 2023-08-25T09:02:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.047+0000 7f0ca33ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c9c1008a0 0x7f0c9c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca33ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c101260 0x7f0c9c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca33ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1068b0 0x7f0c9c196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca33ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c196e70 con 0x7f0c9c1008a0 2023-08-25T09:02:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca33ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c196fe0 con 0x7f0c9c101260 2023-08-25T09:02:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca33ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1971a0 con 0x7f0c9c1068b0 2023-08-25T09:02:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca119b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c9c1008a0 0x7f0c9c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca119b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c9c1008a0 0x7f0c9c19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55262/0 (socket says 172.21.15.99:55262) 2023-08-25T09:02:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca119b700 1 -- 172.21.15.99:0/910815049 learned_addr learned my addr 172.21.15.99:0/910815049 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca199c700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1068b0 0x7f0c9c196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca099a700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c101260 0x7f0c9c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca119b700 1 -- 172.21.15.99:0/910815049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1068b0 msgr2=0x7f0c9c196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca119b700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1068b0 0x7f0c9c196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca119b700 1 -- 172.21.15.99:0/910815049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c101260 msgr2=0x7f0c9c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca119b700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c101260 0x7f0c9c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca119b700 1 -- 172.21.15.99:0/910815049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c009520 con 0x7f0c9c1008a0 2023-08-25T09:02:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.048+0000 7f0ca199c700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1068b0 0x7f0c9c196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:02:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.049+0000 7f0ca119b700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c9c1008a0 0x7f0c9c19c9a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800e9c0 tx=0x7f0c9800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.049+0000 7f0c927fc700 1 -- 172.21.15.99:0/910815049 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98004a30 con 0x7f0c9c1008a0 2023-08-25T09:02:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.049+0000 7f0ca33ff700 1 -- 172.21.15.99:0/910815049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c197480 con 0x7f0c9c1008a0 2023-08-25T09:02:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.049+0000 7f0c927fc700 1 -- 172.21.15.99:0/910815049 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c9800cc80 con 0x7f0c9c1008a0 2023-08-25T09:02:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.049+0000 7f0c927fc700 1 -- 172.21.15.99:0/910815049 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c980054e0 con 0x7f0c9c1008a0 2023-08-25T09:02:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.049+0000 7f0ca33ff700 1 -- 172.21.15.99:0/910815049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c1a3990 con 0x7f0c9c1008a0 2023-08-25T09:02:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.052+0000 7f0c927fc700 1 -- 172.21.15.99:0/910815049 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f0c98004b90 con 0x7f0c9c1008a0 2023-08-25T09:02:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.053+0000 7f0cb6dc4b80 1 Processor -- start 2023-08-25T09:02:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.053+0000 7f0cb6dc4b80 1 -- start start 2023-08-25T09:02:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.053+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508836610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.053+0000 7f0c927fc700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0c880a00c0 0x7f0c880a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.053+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508830860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.053+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c5088311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508794750 con 0x55c508836230 2023-08-25T09:02:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508785720 con 0x55c508836b50 2023-08-25T09:02:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508783040 con 0x55c508830da0 2023-08-25T09:02:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0c927fc700 1 -- 172.21.15.99:0/910815049 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c98014070 con 0x7f0c9c1008a0 2023-08-25T09:02:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca199c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508836610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca119b700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0c880a00c0 0x7f0c880a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca099a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508830860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca199c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508836610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55272/0 (socket says 172.21.15.99:55272) 2023-08-25T09:02:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca199c700 1 -- 172.21.15.99:0/3720889583 learned_addr learned my addr 172.21.15.99:0/3720889583 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca199c700 1 --2- 172.21.15.99:0/3720889583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c5088311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca099a700 1 -- 172.21.15.99:0/3720889583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 msgr2=0x55c5088311e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca099a700 1 --2- 172.21.15.99:0/3720889583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c5088311e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca099a700 1 -- 172.21.15.99:0/3720889583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 msgr2=0x55c508836610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca099a700 1 --2- 172.21.15.99:0/3720889583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508836610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca099a700 1 -- 172.21.15.99:0/3720889583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5085fe5d0 con 0x55c508836b50 2023-08-25T09:02:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.054+0000 7f0ca119b700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0c880a00c0 0x7f0c880a2580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c98095280 tx=0x7f0c98004e40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.055+0000 7f0ca099a700 1 --2- 172.21.15.99:0/3720889583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508830860 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009810 tx=0x7f0c8c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4487ff0766d199ea server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.055+0000 7f0c877fe700 1 -- 172.21.15.99:0/3720889583 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00f720 con 0x55c508836b50 2023-08-25T09:02:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.055+0000 7f0c877fe700 1 -- 172.21.15.99:0/3720889583 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c8c01d040 con 0x55c508836b50 2023-08-25T09:02:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.055+0000 7f0c877fe700 1 -- 172.21.15.99:0/3720889583 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c005440 con 0x55c508836b50 2023-08-25T09:02:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.055+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/3720889583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 msgr2=0x55c508830860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.055+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/3720889583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508830860 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009810 tx=0x7f0c8c005dc0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/3720889583 shutdown_connections 2023-08-25T09:02:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/3720889583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c5088311e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/3720889583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508836610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/3720889583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508830860 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/3720889583 >> 172.21.15.99:0/3720889583 conn(0x55c5087a1080 msgr2=0x55c508826d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/3720889583 shutdown_connections 2023-08-25T09:02:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/3720889583 wait complete. 2023-08-25T09:02:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 -- start start 2023-08-25T09:02:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.056+0000 7f0cb6dc4b80 1 Processor -- start 2023-08-25T09:02:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c50883cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508839da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 0x55c50883a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50877e930 con 0x55c508836230 2023-08-25T09:02:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50871cb50 con 0x55c508830da0 2023-08-25T09:02:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50871c850 con 0x55c508836b50 2023-08-25T09:02:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0cb6dc4b80 10 client.0 _ll_register_callbacks cb 0x55c508666a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:02:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0ca099a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508839da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0ca199c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c50883cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0ca199c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c50883cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48532/0 (socket says 172.21.15.99:48532) 2023-08-25T09:02:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0ca199c700 1 -- 172.21.15.99:0/268806273 learned_addr learned my addr 172.21.15.99:0/268806273 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.057+0000 7f0ca199c700 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 0x55c50883a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0ca199c700 1 -- 172.21.15.99:0/268806273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 msgr2=0x55c50883a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0ca199c700 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 0x55c50883a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0ca199c700 1 -- 172.21.15.99:0/268806273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 msgr2=0x55c508839da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0ca199c700 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508839da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0ca199c700 1 -- 172.21.15.99:0/268806273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c10e070 con 0x55c508830da0 2023-08-25T09:02:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0ca199c700 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c50883cb20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400e800 tx=0x7f0c9400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6677c6ceecd7b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0c84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:02:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0c84ff9700 1 -- 172.21.15.99:0/268806273 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94004bb0 con 0x55c508830da0 2023-08-25T09:02:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/268806273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5087fc160 con 0x55c508830da0 2023-08-25T09:02:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0c84ff9700 1 -- 172.21.15.99:0/268806273 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c94004d10 con 0x55c508830da0 2023-08-25T09:02:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0c84ff9700 1 -- 172.21.15.99:0/268806273 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c9401b510 con 0x55c508830da0 2023-08-25T09:02:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0cb6dc4b80 10 client.25532 fetch_fsmap learned FSMap version 25 2023-08-25T09:02:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.058+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/268806273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c5085aad80 con 0x55c508830da0 2023-08-25T09:02:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0c84ff9700 1 -- 172.21.15.99:0/268806273 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5087fc160 con 0x55c508830da0 2023-08-25T09:02:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0c84ff9700 1 -- 172.21.15.99:0/268806273 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0c9401a570 con 0x55c508830da0 2023-08-25T09:02:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0cb6dc4b80 10 client.25532 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:02:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0cb6dc4b80 10 client.25532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:02:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0cb6dc4b80 20 client.25532 populate_metadata read hostname 'smithi099' 2023-08-25T09:02:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c50884a640 0x55c50884ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0cb6dc4b80 4 client.25532 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:02:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/268806273 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c508724b40 con 0x55c50884a640 2023-08-25T09:02:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.059+0000 7f0ca099a700 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c50884a640 0x55c50884ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.060+0000 7f0ca099a700 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c50884a640 0x55c50884ca60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea4544ff2a7bd65 server_cookie=bc3cadedf68ddf12 in_seq=0 out_seq=0 2023-08-25T09:02:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.060+0000 7f0c84ff9700 10 client.25532 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.060+0000 7f0c84ff9700 1 -- 172.21.15.99:0/268806273 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c508724b40 con 0x55c50884a640 2023-08-25T09:02:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.060+0000 7f0c84ff9700 10 client.25532 handle_command_reply: tid=0 2023-08-25T09:02:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.060+0000 7f0cb6dc4b80 1 client.25532 shutdown 2023-08-25T09:02:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/268806273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 msgr2=0x55c50883cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c50883cb20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400e800 tx=0x7f0c9400b040 comp rx=0 tx=0).stop 2023-08-25T09:02:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/268806273 shutdown_connections 2023-08-25T09:02:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 0x55c50883a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c50884a640 0x55c50884ca60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c50883cb20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0c84ff9700 0 client.25532 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/268806273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508839da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/268806273 >> 172.21.15.99:0/268806273 conn(0x55c5087a1080 msgr2=0x55c508821820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/268806273 shutdown_connections 2023-08-25T09:02:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/268806273 wait complete. 2023-08-25T09:02:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.061+0000 7f0cb6dc4b80 20 client.25532 trim_cache size 0 max 16384 2023-08-25T09:02:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.076+0000 7f0cb6dc4b80 1 Processor -- start 2023-08-25T09:02:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.076+0000 7f0cb6dc4b80 1 -- start start 2023-08-25T09:02:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.076+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c508839d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.076+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508820870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.076+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508841690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.076+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5086008c0 con 0x55c508836230 2023-08-25T09:02:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.076+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508841bd0 con 0x55c508836b50 2023-08-25T09:02:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.076+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5088347d0 con 0x55c508830da0 2023-08-25T09:02:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c508839d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c508839d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:51878/0 (socket says 172.21.15.99:51878) 2023-08-25T09:02:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 -- 172.21.15.99:0/15033294 learned_addr learned my addr 172.21.15.99:0/15033294 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca099a700 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508820870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508841690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 -- 172.21.15.99:0/15033294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 msgr2=0x55c508841690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508841690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 -- 172.21.15.99:0/15033294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 msgr2=0x55c508820870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508820870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 -- 172.21.15.99:0/15033294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c508593c00 con 0x55c508830da0 2023-08-25T09:02:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0ca199c700 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c508839d60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009a00 tx=0x7f0c9401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bfae33bf8cfcee2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0c87fff700 1 -- 172.21.15.99:0/15033294 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94004490 con 0x55c508830da0 2023-08-25T09:02:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0c87fff700 1 -- 172.21.15.99:0/15033294 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c94022070 con 0x55c508830da0 2023-08-25T09:02:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.077+0000 7f0c87fff700 1 -- 172.21.15.99:0/15033294 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9401b510 con 0x55c508830da0 2023-08-25T09:02:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/15033294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 msgr2=0x55c508839d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c508839d60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009a00 tx=0x7f0c9401a000 comp rx=0 tx=0).stop 2023-08-25T09:02:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/15033294 shutdown_connections 2023-08-25T09:02:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508836b50 0x55c508841690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508830da0 0x55c508839d60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/15033294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c508820870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/15033294 >> 172.21.15.99:0/15033294 conn(0x55c5087a1080 msgr2=0x55c5088513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/15033294 shutdown_connections 2023-08-25T09:02:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/15033294 wait complete. 2023-08-25T09:02:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.078+0000 7f0cb6dc4b80 1 -- start start 2023-08-25T09:02:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0cb6dc4b80 1 Processor -- start 2023-08-25T09:02:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c508832d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c5088332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0cb6dc4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 0x55c508831c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508833ad0 con 0x55c508836230 2023-08-25T09:02:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5088321d0 con 0x55c508830da0 2023-08-25T09:02:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0cb6dc4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508832390 con 0x55c508836b50 2023-08-25T09:02:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0cb6dc4b80 10 client.0 _ll_register_callbacks cb 0x55c508666a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:02:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0ca099a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c5088332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0ca199c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c508832d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0ca199c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c508832d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48544/0 (socket says 172.21.15.99:48544) 2023-08-25T09:02:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.079+0000 7f0ca199c700 1 -- 172.21.15.99:0/811852220 learned_addr learned my addr 172.21.15.99:0/811852220 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0ca199c700 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 0x55c508831c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0ca099a700 1 -- 172.21.15.99:0/811852220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 msgr2=0x55c508831c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0ca099a700 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 0x55c508831c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0ca099a700 1 -- 172.21.15.99:0/811852220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 msgr2=0x55c508832d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0ca099a700 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c508832d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0ca099a700 1 -- 172.21.15.99:0/811852220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5085aad80 con 0x55c508836230 2023-08-25T09:02:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0ca199c700 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c508832d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:02:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0ca099a700 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c5088332c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00bd50 tx=0x7f0c8c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7483b58b93fa9c95 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0c84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:02:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0c84ff9700 1 -- 172.21.15.99:0/811852220 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c02fcd0 con 0x55c508836230 2023-08-25T09:02:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/811852220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c50881f600 con 0x55c508836230 2023-08-25T09:02:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.080+0000 7f0c84ff9700 1 -- 172.21.15.99:0/811852220 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c8c02fe30 con 0x55c508836230 2023-08-25T09:02:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0c84ff9700 1 -- 172.21.15.99:0/811852220 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c8c02f0c0 con 0x55c508836230 2023-08-25T09:02:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0cb6dc4b80 10 client.15483 fetch_fsmap learned FSMap version 25 2023-08-25T09:02:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/811852220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c5085d11f0 con 0x55c508836230 2023-08-25T09:02:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0c84ff9700 1 -- 172.21.15.99:0/811852220 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c8c03a080 con 0x55c508836230 2023-08-25T09:02:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0c84ff9700 1 -- 172.21.15.99:0/811852220 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0c8c015020 con 0x55c508836230 2023-08-25T09:02:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0cb6dc4b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 25 2023-08-25T09:02:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0cb6dc4b80 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:02:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0cb6dc4b80 20 client.15483 populate_metadata read hostname 'smithi099' 2023-08-25T09:02:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c50889ee50 0x55c5088a1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0cb6dc4b80 4 client.15483 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:02:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.081+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/811852220 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c508777910 con 0x55c50889ee50 2023-08-25T09:02:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.082+0000 7f0ca199c700 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c50889ee50 0x55c5088a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.082+0000 7f0c84ff9700 10 client.15483 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.082+0000 7f0ca199c700 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c50889ee50 0x55c5088a1270 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bdd6cca3caf2b66 server_cookie=7a87a98656f89a4f in_seq=0 out_seq=0 2023-08-25T09:02:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0c84ff9700 1 -- 172.21.15.99:0/811852220 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c508777910 con 0x55c50889ee50 2023-08-25T09:02:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0c84ff9700 10 client.15483 handle_command_reply: tid=0 2023-08-25T09:02:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 client.15483 shutdown 2023-08-25T09:02:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/811852220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 msgr2=0x55c5088332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c5088332c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00bd50 tx=0x7f0c8c00bf80 comp rx=0 tx=0).stop 2023-08-25T09:02:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/811852220 shutdown_connections 2023-08-25T09:02:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c508836b50 0x55c508831c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c50889ee50 0x55c5088a1270 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c508830da0 0x55c508832d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 --2- 172.21.15.99:0/811852220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c508836230 0x55c5088332c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/811852220 >> 172.21.15.99:0/811852220 conn(0x55c5087a1080 msgr2=0x55c508826bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0c84ff9700 0 client.15483 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/811852220 shutdown_connections 2023-08-25T09:02:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.115+0000 7f0cb6dc4b80 1 -- 172.21.15.99:0/811852220 wait complete. 2023-08-25T09:02:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0cb6dc4b80 20 client.15483 trim_cache size 0 max 16384 2023-08-25T09:02:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:02:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:02:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:02:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:02:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 -- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0c880a00c0 msgr2=0x7f0c880a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0c880a00c0 0x7f0c880a2580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c98095280 tx=0x7f0c98004e40 comp rx=0 tx=0).stop 2023-08-25T09:02:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 -- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c9c1008a0 msgr2=0x7f0c9c19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c9c1008a0 0x7f0c9c19c9a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800e9c0 tx=0x7f0c9800ecd0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 -- 172.21.15.99:0/910815049 shutdown_connections 2023-08-25T09:02:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0c880a00c0 0x7f0c880a2580 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c9c1068b0 0x7f0c9c196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c9c1008a0 0x7f0c9c19c9a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 --2- 172.21.15.99:0/910815049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c101260 0x7f0c9c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.116+0000 7f0ca33ff700 1 -- 172.21.15.99:0/910815049 >> 172.21.15.99:0/910815049 conn(0x7f0c9c0fc370 msgr2=0x7f0c9c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.117+0000 7f0ca33ff700 1 -- 172.21.15.99:0/910815049 shutdown_connections 2023-08-25T09:02:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:25.117+0000 7f0ca33ff700 1 -- 172.21.15.99:0/910815049 wait complete. 2023-08-25T09:02:25.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:25 smithi157 ceph-mon[130879]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 759 KiB/s wr, 74 op/s 2023-08-25T09:02:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:25 smithi181 ceph-mon[131296]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 759 KiB/s wr, 74 op/s 2023-08-25T09:02:25.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:25 smithi099 ceph-mon[97641]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 759 KiB/s wr, 74 op/s 2023-08-25T09:02:25.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78809 1.85 MB/sec execute 31 sec latency 1061.932 ms 2023-08-25T09:02:26.899 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78809 1.79 MB/sec execute 32 sec latency 2062.029 ms 2023-08-25T09:02:27.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:27 smithi099 ceph-mon[97641]: pgmap v101: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2023-08-25T09:02:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:27 smithi157 ceph-mon[130879]: pgmap v101: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2023-08-25T09:02:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:27 smithi181 ceph-mon[131296]: pgmap v101: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2023-08-25T09:02:27.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78877 1.74 MB/sec execute 33 sec latency 2977.368 ms 2023-08-25T09:02:28.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78877 1.69 MB/sec execute 34 sec latency 1031.752 ms 2023-08-25T09:02:29.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:29 smithi099 ceph-mon[97641]: pgmap v102: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 112 op/s 2023-08-25T09:02:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:29 smithi157 ceph-mon[130879]: pgmap v102: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 112 op/s 2023-08-25T09:02:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:29 smithi181 ceph-mon[131296]: pgmap v102: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 112 op/s 2023-08-25T09:02:29.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78877 1.65 MB/sec execute 35 sec latency 2031.882 ms 2023-08-25T09:02:30.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78877 1.60 MB/sec execute 36 sec latency 3031.977 ms 2023-08-25T09:02:31.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:31 smithi099 ceph-mon[97641]: pgmap v103: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 112 op/s 2023-08-25T09:02:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:31 smithi157 ceph-mon[130879]: pgmap v103: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 112 op/s 2023-08-25T09:02:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:31 smithi181 ceph-mon[131296]: pgmap v103: 97 pgs: 97 active+clean; 41 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 112 op/s 2023-08-25T09:02:31.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78877 1.56 MB/sec execute 37 sec latency 4032.141 ms 2023-08-25T09:02:32.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 79000 1.52 MB/sec execute 38 sec latency 4949.843 ms 2023-08-25T09:02:33.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:33 smithi099 ceph-mon[97641]: pgmap v104: 97 pgs: 97 active+clean; 41 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-08-25T09:02:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:33 smithi157 ceph-mon[130879]: pgmap v104: 97 pgs: 97 active+clean; 41 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-08-25T09:02:33.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:33 smithi181 ceph-mon[131296]: pgmap v104: 97 pgs: 97 active+clean; 41 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-08-25T09:02:33.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 81442 1.84 MB/sec execute 39 sec latency 31.130 ms 2023-08-25T09:02:34.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:34 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:34.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:34 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:34.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:34 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:34.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82302 1.91 MB/sec execute 40 sec latency 255.231 ms 2023-08-25T09:02:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:35 smithi099 ceph-mon[97641]: pgmap v105: 97 pgs: 97 active+clean; 41 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-08-25T09:02:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:35 smithi157 ceph-mon[130879]: pgmap v105: 97 pgs: 97 active+clean; 41 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-08-25T09:02:35.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:35 smithi181 ceph-mon[131296]: pgmap v105: 97 pgs: 97 active+clean; 41 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-08-25T09:02:35.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82302 1.86 MB/sec execute 41 sec latency 1255.313 ms 2023-08-25T09:02:36.900 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82302 1.82 MB/sec execute 42 sec latency 2255.454 ms 2023-08-25T09:02:37.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:37 smithi099 ceph-mon[97641]: pgmap v106: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2023-08-25T09:02:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:37 smithi157 ceph-mon[130879]: pgmap v106: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2023-08-25T09:02:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:37 smithi181 ceph-mon[131296]: pgmap v106: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2023-08-25T09:02:37.901 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82358 1.78 MB/sec execute 43 sec latency 3171.755 ms 2023-08-25T09:02:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 -- 172.21.15.99:0/1759505666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424072cd0 msgr2=0x7f64240730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 --2- 172.21.15.99:0/1759505666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424072cd0 0x7f64240730b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f64200098f0 tx=0x7f6420009c00 comp rx=0 tx=0).stop 2023-08-25T09:02:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 -- 172.21.15.99:0/1759505666 shutdown_connections 2023-08-25T09:02:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 --2- 172.21.15.99:0/1759505666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642410d160 0x7f642410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 --2- 172.21.15.99:0/1759505666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424072cd0 0x7f64240730b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 --2- 172.21.15.99:0/1759505666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64240735f0 0x7f642410cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 -- 172.21.15.99:0/1759505666 >> 172.21.15.99:0/1759505666 conn(0x7f64240fc370 msgr2=0x7f64240fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 -- 172.21.15.99:0/1759505666 shutdown_connections 2023-08-25T09:02:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.272+0000 7f642c7ff700 1 -- 172.21.15.99:0/1759505666 wait complete. 2023-08-25T09:02:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.273+0000 7f642c7ff700 1 Processor -- start 2023-08-25T09:02:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.273+0000 7f642c7ff700 1 -- start start 2023-08-25T09:02:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642c7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6424072cd0 0x7f642419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642c7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64240735f0 0x7f642419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642c7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642410d160 0x7f6424198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642c7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241990b0 con 0x7f6424072cd0 2023-08-25T09:02:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642c7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6424199220 con 0x7f642410d160 2023-08-25T09:02:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642c7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241993e0 con 0x7f64240735f0 2023-08-25T09:02:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6424072cd0 0x7f642419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6424072cd0 0x7f642419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57752/0 (socket says 172.21.15.99:57752) 2023-08-25T09:02:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642a59b700 1 -- 172.21.15.99:0/3949324078 learned_addr learned my addr 172.21.15.99:0/3949324078 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f6429d9a700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64240735f0 0x7f642419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642ad9c700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642410d160 0x7f6424198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642a59b700 1 -- 172.21.15.99:0/3949324078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64240735f0 msgr2=0x7f642419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642a59b700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64240735f0 0x7f642419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642a59b700 1 -- 172.21.15.99:0/3949324078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642410d160 msgr2=0x7f6424198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642a59b700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642410d160 0x7f6424198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642a59b700 1 -- 172.21.15.99:0/3949324078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140095f0 con 0x7f6424072cd0 2023-08-25T09:02:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.274+0000 7f642ad9c700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642410d160 0x7f6424198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:02:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.275+0000 7f642a59b700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6424072cd0 0x7f642419ebe0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6420009230 tx=0x7f642000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.275+0000 7f641b7fe700 1 -- 172.21.15.99:0/3949324078 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642001c070 con 0x7f6424072cd0 2023-08-25T09:02:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.275+0000 7f641b7fe700 1 -- 172.21.15.99:0/3949324078 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6420005040 con 0x7f6424072cd0 2023-08-25T09:02:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.275+0000 7f642c7ff700 1 -- 172.21.15.99:0/3949324078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6420009520 con 0x7f6424072cd0 2023-08-25T09:02:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.275+0000 7f641b7fe700 1 -- 172.21.15.99:0/3949324078 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420003920 con 0x7f6424072cd0 2023-08-25T09:02:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.275+0000 7f642c7ff700 1 -- 172.21.15.99:0/3949324078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64241a59f0 con 0x7f6424072cd0 2023-08-25T09:02:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.278+0000 7f641b7fe700 1 -- 172.21.15.99:0/3949324078 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f6420020030 con 0x7f6424072cd0 2023-08-25T09:02:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.279+0000 7f64401a1b80 1 Processor -- start 2023-08-25T09:02:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f64401a1b80 1 -- start start 2023-08-25T09:02:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f641b7fe700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64100a0140 0x7f64100a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f641b7fe700 1 -- 172.21.15.99:0/3949324078 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f64200cc7e0 con 0x7f6424072cd0 2023-08-25T09:02:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f642ad9c700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64100a0140 0x7f64100a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a171a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171f50 0x563a4a16bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a16c1a0 0x563a4a16c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f64401a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a0cb750 con 0x563a4a171630 2023-08-25T09:02:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f64401a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a0bc720 con 0x563a4a16c1a0 2023-08-25T09:02:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f64401a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a0ba040 con 0x563a4a171f50 2023-08-25T09:02:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f642a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a171a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f642a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a171a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57760/0 (socket says 172.21.15.99:57760) 2023-08-25T09:02:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.280+0000 7f642a59b700 1 -- 172.21.15.99:0/2921323346 learned_addr learned my addr 172.21.15.99:0/2921323346 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642ad9c700 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a16c1a0 0x563a4a16c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f6429d9a700 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171f50 0x563a4a16bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642a59b700 1 -- 172.21.15.99:0/2921323346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171f50 msgr2=0x563a4a16bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642a59b700 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171f50 0x563a4a16bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642a59b700 1 -- 172.21.15.99:0/2921323346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a16c1a0 msgr2=0x563a4a16c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642a59b700 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a16c1a0 0x563a4a16c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642a59b700 1 -- 172.21.15.99:0/2921323346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a49f355d0 con 0x563a4a171630 2023-08-25T09:02:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f6429d9a700 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171f50 0x563a4a16bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:02:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642ad9c700 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a16c1a0 0x563a4a16c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:02:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642ad9c700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64100a0140 0x7f64100a2600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f641c009fd0 tx=0x7f641c009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f642a59b700 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a171a10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6420096950 tx=0x7f6420099840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4475d8d41ad99f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f6418ff9700 1 -- 172.21.15.99:0/2921323346 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642009dd10 con 0x563a4a171630 2023-08-25T09:02:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f6418ff9700 1 -- 172.21.15.99:0/2921323346 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64200a1c60 con 0x563a4a171630 2023-08-25T09:02:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.281+0000 7f6418ff9700 1 -- 172.21.15.99:0/2921323346 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642009a140 con 0x563a4a171630 2023-08-25T09:02:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 -- 172.21.15.99:0/2921323346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 msgr2=0x563a4a171a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a171a10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6420096950 tx=0x7f6420099840 comp rx=0 tx=0).stop 2023-08-25T09:02:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 -- 172.21.15.99:0/2921323346 shutdown_connections 2023-08-25T09:02:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a16c1a0 0x563a4a16c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a171a10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 --2- 172.21.15.99:0/2921323346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171f50 0x563a4a16bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 -- 172.21.15.99:0/2921323346 >> 172.21.15.99:0/2921323346 conn(0x563a4a0d8080 msgr2=0x563a4a15dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 -- 172.21.15.99:0/2921323346 shutdown_connections 2023-08-25T09:02:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.282+0000 7f64401a1b80 1 -- 172.21.15.99:0/2921323346 wait complete. 2023-08-25T09:02:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 1 -- start start 2023-08-25T09:02:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 1 Processor -- start 2023-08-25T09:02:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a175110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 0x563a4a1756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a0b5930 con 0x563a4a171630 2023-08-25T09:02:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a053b50 con 0x563a4a171f50 2023-08-25T09:02:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a053850 con 0x563a4a16c1a0 2023-08-25T09:02:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.283+0000 7f64401a1b80 10 client.0 _ll_register_callbacks cb 0x563a49f9da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:02:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f6429d9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a175110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:47748/0 (socket says 172.21.15.99:47748) 2023-08-25T09:02:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 -- 172.21.15.99:0/1671910489 learned_addr learned my addr 172.21.15.99:0/1671910489 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642ad9c700 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 0x563a4a1756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 -- 172.21.15.99:0/1671910489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 msgr2=0x563a4a1756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 0x563a4a1756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 -- 172.21.15.99:0/1671910489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 msgr2=0x563a4a175110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a175110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 -- 172.21.15.99:0/1671910489 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64241a59f0 con 0x563a4a16c1a0 2023-08-25T09:02:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f642a59b700 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177df0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f642000bd50 tx=0x7f6420005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c07a16521d8977 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f640dffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:02:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f640dffb700 1 -- 172.21.15.99:0/1671910489 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642009dd10 con 0x563a4a16c1a0 2023-08-25T09:02:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f64401a1b80 1 -- 172.21.15.99:0/1671910489 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a4a133160 con 0x563a4a16c1a0 2023-08-25T09:02:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.284+0000 7f640dffb700 1 -- 172.21.15.99:0/1671910489 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f64200b7e40 con 0x563a4a16c1a0 2023-08-25T09:02:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f640dffb700 1 -- 172.21.15.99:0/1671910489 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64200ca110 con 0x563a4a16c1a0 2023-08-25T09:02:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f64401a1b80 10 client.25534 fetch_fsmap learned FSMap version 26 2023-08-25T09:02:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f64401a1b80 1 -- 172.21.15.99:0/1671910489 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563a49ee1d80 con 0x563a4a16c1a0 2023-08-25T09:02:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f640dffb700 1 -- 172.21.15.99:0/1671910489 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a4a133160 con 0x563a4a16c1a0 2023-08-25T09:02:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f640dffb700 1 -- 172.21.15.99:0/1671910489 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64200993d0 con 0x563a4a16c1a0 2023-08-25T09:02:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f64401a1b80 10 client.25534 fetch_fsmap finished waiting for FSMap version 26 2023-08-25T09:02:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f64401a1b80 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:02:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f64401a1b80 20 client.25534 populate_metadata read hostname 'smithi099' 2023-08-25T09:02:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563a4a185230 0x563a4a187650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f64401a1b80 4 client.25534 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:02:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.285+0000 7f64401a1b80 1 -- 172.21.15.99:0/1671910489 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a4a05bb40 con 0x563a4a185230 2023-08-25T09:02:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.286+0000 7f6429d9a700 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563a4a185230 0x563a4a187650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.286+0000 7f640dffb700 10 client.25534 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.286+0000 7f6429d9a700 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563a4a185230 0x563a4a187650 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8e5b99500871fa server_cookie=f06b53ab6adabad2 in_seq=0 out_seq=0 2023-08-25T09:02:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f640dffb700 1 -- 172.21.15.99:0/1671910489 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a4a05bb40 con 0x563a4a185230 2023-08-25T09:02:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f640dffb700 10 client.25534 handle_command_reply: tid=0 2023-08-25T09:02:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 client.25534 shutdown 2023-08-25T09:02:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 -- 172.21.15.99:0/1671910489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 msgr2=0x563a4a177df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177df0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f642000bd50 tx=0x7f6420005650 comp rx=0 tx=0).stop 2023-08-25T09:02:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 -- 172.21.15.99:0/1671910489 shutdown_connections 2023-08-25T09:02:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563a4a185230 0x563a4a187650 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 0x563a4a1756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177df0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1671910489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171630 0x563a4a175110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f640dffb700 0 client.25534 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 -- 172.21.15.99:0/1671910489 >> 172.21.15.99:0/1671910489 conn(0x563a4a0d8080 msgr2=0x563a4a158820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 -- 172.21.15.99:0/1671910489 shutdown_connections 2023-08-25T09:02:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 1 -- 172.21.15.99:0/1671910489 wait complete. 2023-08-25T09:02:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.287+0000 7f64401a1b80 20 client.25534 trim_cache size 0 max 16384 2023-08-25T09:02:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f64401a1b80 1 Processor -- start 2023-08-25T09:02:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f64401a1b80 1 -- start start 2023-08-25T09:02:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171630 0x563a4a16e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171f50 0x563a4a16e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f64401a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a49f378c0 con 0x563a4a171f50 2023-08-25T09:02:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f64401a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a15a730 con 0x563a4a171630 2023-08-25T09:02:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f64401a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a16fc50 con 0x563a4a16c1a0 2023-08-25T09:02:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.303+0000 7f642ad9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171f50 0x563a4a16e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642a59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642ad9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171f50 0x563a4a16e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57782/0 (socket says 172.21.15.99:57782) 2023-08-25T09:02:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642ad9c700 1 -- 172.21.15.99:0/3077050511 learned_addr learned my addr 172.21.15.99:0/3077050511 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f6429d9a700 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171630 0x563a4a16e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642a59b700 1 -- 172.21.15.99:0/3077050511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171630 msgr2=0x563a4a16e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642a59b700 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171630 0x563a4a16e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642a59b700 1 -- 172.21.15.99:0/3077050511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171f50 msgr2=0x563a4a16e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642a59b700 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171f50 0x563a4a16e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642a59b700 1 -- 172.21.15.99:0/3077050511 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a49ecac00 con 0x563a4a16c1a0 2023-08-25T09:02:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642ad9c700 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171f50 0x563a4a16e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:02:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f642a59b700 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6420099b90 tx=0x7f6420099840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d874c172d0c132c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f64197fa700 1 -- 172.21.15.99:0/3077050511 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642009a3b0 con 0x563a4a16c1a0 2023-08-25T09:02:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f64197fa700 1 -- 172.21.15.99:0/3077050511 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f642009a230 con 0x563a4a16c1a0 2023-08-25T09:02:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.304+0000 7f64197fa700 1 -- 172.21.15.99:0/3077050511 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64200993d0 con 0x563a4a16c1a0 2023-08-25T09:02:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 -- 172.21.15.99:0/3077050511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 msgr2=0x563a4a177920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6420099b90 tx=0x7f6420099840 comp rx=0 tx=0).stop 2023-08-25T09:02:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 -- 172.21.15.99:0/3077050511 shutdown_connections 2023-08-25T09:02:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a171f50 0x563a4a16e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a16c1a0 0x563a4a177920 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 --2- 172.21.15.99:0/3077050511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171630 0x563a4a16e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 -- 172.21.15.99:0/3077050511 >> 172.21.15.99:0/3077050511 conn(0x563a4a0d8080 msgr2=0x563a4a180f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 -- 172.21.15.99:0/3077050511 shutdown_connections 2023-08-25T09:02:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.305+0000 7f64401a1b80 1 -- 172.21.15.99:0/3077050511 wait complete. 2023-08-25T09:02:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 1 -- start start 2023-08-25T09:02:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 1 Processor -- start 2023-08-25T09:02:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a16c1a0 0x563a4a17a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171630 0x563a4a17ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 0x563a4a17b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a1d8200 con 0x563a4a16c1a0 2023-08-25T09:02:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a1d8370 con 0x563a4a171f50 2023-08-25T09:02:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4a1d8530 con 0x563a4a171630 2023-08-25T09:02:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f64401a1b80 10 client.0 _ll_register_callbacks cb 0x7f6420099a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:02:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f642a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a16c1a0 0x563a4a17a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f642a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a16c1a0 0x563a4a17a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57794/0 (socket says 172.21.15.99:57794) 2023-08-25T09:02:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.306+0000 7f642a59b700 1 -- 172.21.15.99:0/1615176892 learned_addr learned my addr 172.21.15.99:0/1615176892 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f642ad9c700 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 0x563a4a17b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f6429d9a700 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171630 0x563a4a17ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f642a59b700 1 -- 172.21.15.99:0/1615176892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171630 msgr2=0x563a4a17ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f642a59b700 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171630 0x563a4a17ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f642a59b700 1 -- 172.21.15.99:0/1615176892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 msgr2=0x563a4a17b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f642a59b700 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 0x563a4a17b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f642a59b700 1 -- 172.21.15.99:0/1615176892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c007d00 con 0x563a4a16c1a0 2023-08-25T09:02:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f642a59b700 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a16c1a0 0x563a4a17a800 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f64200059d0 tx=0x7f6420009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eef0d4f44ef09e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f640dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:02:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f640dffb700 1 -- 172.21.15.99:0/1615176892 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642009dd10 con 0x563a4a16c1a0 2023-08-25T09:02:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f640dffb700 1 -- 172.21.15.99:0/1615176892 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f64200b7e40 con 0x563a4a16c1a0 2023-08-25T09:02:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f64401a1b80 1 -- 172.21.15.99:0/1615176892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a4a0ae910 con 0x563a4a16c1a0 2023-08-25T09:02:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f640dffb700 1 -- 172.21.15.99:0/1615176892 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64200993d0 con 0x563a4a16c1a0 2023-08-25T09:02:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f64401a1b80 10 client.15501 fetch_fsmap learned FSMap version 26 2023-08-25T09:02:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.307+0000 7f64401a1b80 1 -- 172.21.15.99:0/1615176892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563a49ee1d80 con 0x563a4a16c1a0 2023-08-25T09:02:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f640dffb700 1 -- 172.21.15.99:0/1615176892 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a4a0ae910 con 0x563a4a16c1a0 2023-08-25T09:02:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f640dffb700 1 -- 172.21.15.99:0/1615176892 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64200833f0 con 0x563a4a16c1a0 2023-08-25T09:02:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f64401a1b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 26 2023-08-25T09:02:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f64401a1b80 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:02:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f64401a1b80 20 client.15501 populate_metadata read hostname 'smithi099' 2023-08-25T09:02:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563a4a1d9770 0x563a4a1dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f64401a1b80 4 client.15501 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:02:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f64401a1b80 1 -- 172.21.15.99:0/1615176892 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a4a06eff0 con 0x563a4a1d9770 2023-08-25T09:02:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.308+0000 7f6429d9a700 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563a4a1d9770 0x563a4a1dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.309+0000 7f640dffb700 10 client.15501 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.309+0000 7f6429d9a700 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563a4a1d9770 0x563a4a1dbb60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f8bd3d18beaadc server_cookie=106559e81a4e0788 in_seq=0 out_seq=0 2023-08-25T09:02:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f640dffb700 1 -- 172.21.15.99:0/1615176892 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a4a06eff0 con 0x563a4a1d9770 2023-08-25T09:02:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f640dffb700 10 client.15501 handle_command_reply: tid=0 2023-08-25T09:02:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 client.15501 shutdown 2023-08-25T09:02:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 -- 172.21.15.99:0/1615176892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a16c1a0 msgr2=0x563a4a17a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a16c1a0 0x563a4a17a800 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f64200059d0 tx=0x7f6420009260 comp rx=0 tx=0).stop 2023-08-25T09:02:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 -- 172.21.15.99:0/1615176892 shutdown_connections 2023-08-25T09:02:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563a4a1d9770 0x563a4a1dbb60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4a171f50 0x563a4a17b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a4a16c1a0 0x563a4a17a800 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f640dffb700 0 client.15501 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 --2- 172.21.15.99:0/1615176892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a4a171630 0x563a4a17ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.337+0000 7f64401a1b80 1 -- 172.21.15.99:0/1615176892 >> 172.21.15.99:0/1615176892 conn(0x563a4a0d8080 msgr2=0x563a4a172400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f64401a1b80 1 -- 172.21.15.99:0/1615176892 shutdown_connections 2023-08-25T09:02:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f64401a1b80 1 -- 172.21.15.99:0/1615176892 wait complete. 2023-08-25T09:02:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f64401a1b80 20 client.15501 trim_cache size 0 max 16384 2023-08-25T09:02:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:02:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:02:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:02:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:02:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 -- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64100a0140 msgr2=0x7f64100a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64100a0140 0x7f64100a2600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f641c009fd0 tx=0x7f641c009920 comp rx=0 tx=0).stop 2023-08-25T09:02:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 -- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6424072cd0 msgr2=0x7f642419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6424072cd0 0x7f642419ebe0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6420009230 tx=0x7f642000fa00 comp rx=0 tx=0).stop 2023-08-25T09:02:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 -- 172.21.15.99:0/3949324078 shutdown_connections 2023-08-25T09:02:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64100a0140 0x7f64100a2600 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642410d160 0x7f6424198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6424072cd0 0x7f642419ebe0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 --2- 172.21.15.99:0/3949324078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64240735f0 0x7f642419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.338+0000 7f642c7ff700 1 -- 172.21.15.99:0/3949324078 >> 172.21.15.99:0/3949324078 conn(0x7f64240fc370 msgr2=0x7f642410f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.339+0000 7f642c7ff700 1 -- 172.21.15.99:0/3949324078 shutdown_connections 2023-08-25T09:02:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:38.339+0000 7f642c7ff700 1 -- 172.21.15.99:0/3949324078 wait complete. 2023-08-25T09:02:38.901 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82358 1.74 MB/sec execute 44 sec latency 1039.790 ms 2023-08-25T09:02:39.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:39 smithi099 ceph-mon[97641]: pgmap v107: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 476 KiB/s wr, 47 op/s 2023-08-25T09:02:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:39 smithi157 ceph-mon[130879]: pgmap v107: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 476 KiB/s wr, 47 op/s 2023-08-25T09:02:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:39 smithi181 ceph-mon[131296]: pgmap v107: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 476 KiB/s wr, 47 op/s 2023-08-25T09:02:39.901 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82358 1.70 MB/sec execute 45 sec latency 2039.902 ms 2023-08-25T09:02:40.901 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82358 1.66 MB/sec execute 46 sec latency 3040.035 ms 2023-08-25T09:02:41.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:41 smithi099 ceph-mon[97641]: pgmap v108: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 476 KiB/s wr, 47 op/s 2023-08-25T09:02:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:41 smithi157 ceph-mon[130879]: pgmap v108: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 476 KiB/s wr, 47 op/s 2023-08-25T09:02:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:41 smithi181 ceph-mon[131296]: pgmap v108: 97 pgs: 97 active+clean; 43 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 476 KiB/s wr, 47 op/s 2023-08-25T09:02:41.901 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82358 1.63 MB/sec execute 47 sec latency 4040.154 ms 2023-08-25T09:02:42.901 INFO:tasks.workunit.client.0.smithi099.stdout: 1 82462 1.60 MB/sec execute 48 sec latency 4955.878 ms 2023-08-25T09:02:43.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:43 smithi099 ceph-mon[97641]: pgmap v109: 97 pgs: 97 active+clean; 43 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 527 KiB/s wr, 65 op/s 2023-08-25T09:02:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:43 smithi157 ceph-mon[130879]: pgmap v109: 97 pgs: 97 active+clean; 43 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 527 KiB/s wr, 65 op/s 2023-08-25T09:02:43.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:43 smithi181 ceph-mon[131296]: pgmap v109: 97 pgs: 97 active+clean; 43 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 527 KiB/s wr, 65 op/s 2023-08-25T09:02:43.901 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85010 1.85 MB/sec execute 49 sec latency 11.950 ms 2023-08-25T09:02:44.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:44 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:44.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:44 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:44.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:44 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:44.901 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85843 1.91 MB/sec execute 50 sec latency 225.048 ms 2023-08-25T09:02:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:45 smithi157 ceph-mon[130879]: pgmap v110: 97 pgs: 97 active+clean; 43 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 507 KiB/s wr, 59 op/s 2023-08-25T09:02:45.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:45 smithi181 ceph-mon[131296]: pgmap v110: 97 pgs: 97 active+clean; 43 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 507 KiB/s wr, 59 op/s 2023-08-25T09:02:45.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:45 smithi099 ceph-mon[97641]: pgmap v110: 97 pgs: 97 active+clean; 43 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 507 KiB/s wr, 59 op/s 2023-08-25T09:02:45.902 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85843 1.87 MB/sec execute 51 sec latency 1225.181 ms 2023-08-25T09:02:46.902 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85843 1.84 MB/sec execute 52 sec latency 2225.311 ms 2023-08-25T09:02:47.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:47 smithi157 ceph-mon[130879]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 719 KiB/s wr, 93 op/s 2023-08-25T09:02:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:47 smithi181 ceph-mon[131296]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 719 KiB/s wr, 93 op/s 2023-08-25T09:02:47.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:47 smithi099 ceph-mon[97641]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 719 KiB/s wr, 93 op/s 2023-08-25T09:02:47.902 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85966 1.81 MB/sec execute 53 sec latency 3138.688 ms 2023-08-25T09:02:48.902 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85967 1.77 MB/sec execute 54 sec latency 990.537 ms 2023-08-25T09:02:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:49 smithi157 ceph-mon[130879]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 263 KiB/s wr, 51 op/s 2023-08-25T09:02:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:49 smithi181 ceph-mon[131296]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 263 KiB/s wr, 51 op/s 2023-08-25T09:02:49.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:49 smithi099 ceph-mon[97641]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 263 KiB/s wr, 51 op/s 2023-08-25T09:02:49.902 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85967 1.74 MB/sec execute 55 sec latency 1990.657 ms 2023-08-25T09:02:50.903 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85967 1.71 MB/sec execute 56 sec latency 2990.786 ms 2023-08-25T09:02:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.493+0000 7fb21e9ff700 1 -- 172.21.15.99:0/2953444926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2181068c0 msgr2=0x7fb218106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.493+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/2953444926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2181068c0 0x7fb218106ca0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb2080098f0 tx=0x7fb208009c00 comp rx=0 tx=0).stop 2023-08-25T09:02:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.493+0000 7fb21e9ff700 1 -- 172.21.15.99:0/2953444926 shutdown_connections 2023-08-25T09:02:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.493+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/2953444926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb218069620 0x7fb21810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.493+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/2953444926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2181068c0 0x7fb218106ca0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.493+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/2953444926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb218068c80 0x7fb2180690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.493+0000 7fb21e9ff700 1 -- 172.21.15.99:0/2953444926 >> 172.21.15.99:0/2953444926 conn(0x7fb218076920 msgr2=0x7fb218076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.494+0000 7fb21e9ff700 1 -- 172.21.15.99:0/2953444926 shutdown_connections 2023-08-25T09:02:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.494+0000 7fb21e9ff700 1 -- 172.21.15.99:0/2953444926 wait complete. 2023-08-25T09:02:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21e9ff700 1 Processor -- start 2023-08-25T09:02:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21e9ff700 1 -- start start 2023-08-25T09:02:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21e9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218068c80 0x7fb2180ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21e9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb218069620 0x7fb2180ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21e9ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2181068c0 0x7fb2181002e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21e9ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181a9520 con 0x7fb2181068c0 2023-08-25T09:02:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21e9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181a9690 con 0x7fb218069620 2023-08-25T09:02:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21e9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181a9850 con 0x7fb218068c80 2023-08-25T09:02:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb21cf9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2181068c0 0x7fb2181002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb2177fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb218069620 0x7fb2180ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb2177fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb218069620 0x7fb2180ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:46730/0 (socket says 172.21.15.99:46730) 2023-08-25T09:02:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.495+0000 7fb217fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218068c80 0x7fb2180ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2177fe700 1 -- 172.21.15.99:0/3353158794 learned_addr learned my addr 172.21.15.99:0/3353158794 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2177fe700 1 -- 172.21.15.99:0/3353158794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218068c80 msgr2=0x7fb2180ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2177fe700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218068c80 0x7fb2180ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2177fe700 1 -- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2181068c0 msgr2=0x7fb2181002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2177fe700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2181068c0 0x7fb2181002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2177fe700 1 -- 172.21.15.99:0/3353158794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0095f0 con 0x7fb218069620 2023-08-25T09:02:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb21cf9c700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2181068c0 0x7fb2181002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:02:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2177fe700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb218069620 0x7fb2180ffda0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00ec10 tx=0x7fb20c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2157fa700 1 -- 172.21.15.99:0/3353158794 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00cc90 con 0x7fb218069620 2023-08-25T09:02:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2157fa700 1 -- 172.21.15.99:0/3353158794 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20c0044e0 con 0x7fb218069620 2023-08-25T09:02:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb21e9ff700 1 -- 172.21.15.99:0/3353158794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb208009520 con 0x7fb218069620 2023-08-25T09:02:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.496+0000 7fb2157fa700 1 -- 172.21.15.99:0/3353158794 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c010430 con 0x7fb218069620 2023-08-25T09:02:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.497+0000 7fb21e9ff700 1 -- 172.21.15.99:0/3353158794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2181a9eb0 con 0x7fb218069620 2023-08-25T09:02:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.500+0000 7fb2157fa700 1 -- 172.21.15.99:0/3353158794 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fb20c003fb0 con 0x7fb218069620 2023-08-25T09:02:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.501+0000 7fb23223cb80 1 Processor -- start 2023-08-25T09:02:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.501+0000 7fb23223cb80 1 -- start start 2023-08-25T09:02:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.501+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719adf40 0x55e2719ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.501+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.501+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719a8990 0x55e2719a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.501+0000 7fb23223cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e271910750 con 0x55e2719adf40 2023-08-25T09:02:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.501+0000 7fb23223cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e271901720 con 0x55e2719a8990 2023-08-25T09:02:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.501+0000 7fb23223cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2718ff040 con 0x55e2719ae860 2023-08-25T09:02:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb21cf9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719adf40 0x55e2719ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb2157fa700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb2000a00c0 0x7fb2000a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:59686/0 (socket says 172.21.15.99:59686) 2023-08-25T09:02:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 -- 172.21.15.99:0/1354617664 learned_addr learned my addr 172.21.15.99:0/1354617664 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb2177fe700 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719a8990 0x55e2719a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb2157fa700 1 -- 172.21.15.99:0/3353158794 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb20c014070 con 0x7fb218069620 2023-08-25T09:02:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 -- 172.21.15.99:0/1354617664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719a8990 msgr2=0x55e2719a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb21cf9c700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb2000a00c0 0x7fb2000a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719a8990 0x55e2719a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 -- 172.21.15.99:0/1354617664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719adf40 msgr2=0x55e2719ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719adf40 0x55e2719ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 -- 172.21.15.99:0/1354617664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e27177a5d0 con 0x55e2719ae860 2023-08-25T09:02:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb2177fe700 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719a8990 0x55e2719a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:02:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.502+0000 7fb217fff700 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a8450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb208000c00 tx=0x7fb208005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa25466a41d1545b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb21cf9c700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb2000a00c0 0x7fb2000a2580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb210003fd0 tx=0x7fb210004b40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb1fe7fc700 1 -- 172.21.15.99:0/1354617664 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20800f7a0 con 0x55e2719ae860 2023-08-25T09:02:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb1fe7fc700 1 -- 172.21.15.99:0/1354617664 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20801d040 con 0x55e2719ae860 2023-08-25T09:02:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb1fe7fc700 1 -- 172.21.15.99:0/1354617664 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208005510 con 0x55e2719ae860 2023-08-25T09:02:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb23223cb80 1 -- 172.21.15.99:0/1354617664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 msgr2=0x55e2719a8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb23223cb80 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a8450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb208000c00 tx=0x7fb208005dc0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb23223cb80 1 -- 172.21.15.99:0/1354617664 shutdown_connections 2023-08-25T09:02:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb23223cb80 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719a8990 0x55e2719a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb23223cb80 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719adf40 0x55e2719ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb23223cb80 1 --2- 172.21.15.99:0/1354617664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a8450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.503+0000 7fb23223cb80 1 -- 172.21.15.99:0/1354617664 >> 172.21.15.99:0/1354617664 conn(0x55e27191d080 msgr2=0x55e2719a2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.504+0000 7fb23223cb80 1 -- 172.21.15.99:0/1354617664 shutdown_connections 2023-08-25T09:02:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.504+0000 7fb23223cb80 1 -- 172.21.15.99:0/1354617664 wait complete. 2023-08-25T09:02:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.504+0000 7fb23223cb80 1 -- start start 2023-08-25T09:02:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.504+0000 7fb23223cb80 1 Processor -- start 2023-08-25T09:02:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719adf40 0x55e2719b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719ae860 0x55e2719b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb23223cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2718fa930 con 0x55e2719a8990 2023-08-25T09:02:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb23223cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e271898b50 con 0x55e2719ae860 2023-08-25T09:02:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb23223cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e271898850 con 0x55e2719adf40 2023-08-25T09:02:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb23223cb80 10 client.0 _ll_register_callbacks cb 0x55e2717e2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:02:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb217fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb217fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43958/0 (socket says 172.21.15.99:43958) 2023-08-25T09:02:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb217fff700 1 -- 172.21.15.99:0/3655442249 learned_addr learned my addr 172.21.15.99:0/3655442249 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb21cf9c700 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719ae860 0x55e2719b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb2177fe700 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719adf40 0x55e2719b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb217fff700 1 -- 172.21.15.99:0/3655442249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719adf40 msgr2=0x55e2719b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb217fff700 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719adf40 0x55e2719b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb217fff700 1 -- 172.21.15.99:0/3655442249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719ae860 msgr2=0x55e2719b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb217fff700 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719ae860 0x55e2719b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.505+0000 7fb217fff700 1 -- 172.21.15.99:0/3655442249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21810e080 con 0x55e2719a8990 2023-08-25T09:02:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.506+0000 7fb217fff700 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719b4680 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb208009810 tx=0x7fb20800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d09406020c8ee8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.506+0000 7fb1f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:02:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.506+0000 7fb1f77fe700 1 -- 172.21.15.99:0/3655442249 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2080054d0 con 0x55e2719a8990 2023-08-25T09:02:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.506+0000 7fb23223cb80 1 -- 172.21.15.99:0/3655442249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e271978160 con 0x55e2719a8990 2023-08-25T09:02:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.506+0000 7fb1f77fe700 1 -- 172.21.15.99:0/3655442249 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb208005630 con 0x55e2719a8990 2023-08-25T09:02:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.506+0000 7fb1f77fe700 1 -- 172.21.15.99:0/3655442249 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb20800bc10 con 0x55e2719a8990 2023-08-25T09:02:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.506+0000 7fb23223cb80 10 client.15513 fetch_fsmap learned FSMap version 27 2023-08-25T09:02:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.506+0000 7fb23223cb80 1 -- 172.21.15.99:0/3655442249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e271726d80 con 0x55e2719a8990 2023-08-25T09:02:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb1f77fe700 1 -- 172.21.15.99:0/3655442249 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e271978160 con 0x55e2719a8990 2023-08-25T09:02:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb1f77fe700 1 -- 172.21.15.99:0/3655442249 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb208015020 con 0x55e2719a8990 2023-08-25T09:02:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb23223cb80 10 client.15513 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T09:02:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb23223cb80 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:02:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb23223cb80 20 client.15513 populate_metadata read hostname 'smithi099' 2023-08-25T09:02:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb23223cb80 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e2719c1bd0 0x55e2719c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb23223cb80 4 client.15513 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:02:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb23223cb80 1 -- 172.21.15.99:0/3655442249 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2718a0b40 con 0x55e2719c1bd0 2023-08-25T09:02:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.507+0000 7fb2177fe700 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e2719c1bd0 0x55e2719c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.508+0000 7fb2177fe700 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e2719c1bd0 0x55e2719c3fc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=889eaaa32f3113cc server_cookie=7fe87689eddcad56 in_seq=0 out_seq=0 2023-08-25T09:02:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.508+0000 7fb1f77fe700 10 client.15513 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb1f77fe700 1 -- 172.21.15.99:0/3655442249 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2718a0b40 con 0x55e2719c1bd0 2023-08-25T09:02:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb1f77fe700 10 client.15513 handle_command_reply: tid=0 2023-08-25T09:02:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 client.15513 shutdown 2023-08-25T09:02:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 -- 172.21.15.99:0/3655442249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 msgr2=0x55e2719b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719b4680 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb208009810 tx=0x7fb20800b790 comp rx=0 tx=0).stop 2023-08-25T09:02:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 -- 172.21.15.99:0/3655442249 shutdown_connections 2023-08-25T09:02:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e2719c1bd0 0x55e2719c3fc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719ae860 0x55e2719b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719b4680 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 --2- 172.21.15.99:0/3655442249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719adf40 0x55e2719b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb1f77fe700 0 client.15513 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 -- 172.21.15.99:0/3655442249 >> 172.21.15.99:0/3655442249 conn(0x55e27191d080 msgr2=0x55e27199d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 -- 172.21.15.99:0/3655442249 shutdown_connections 2023-08-25T09:02:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 1 -- 172.21.15.99:0/3655442249 wait complete. 2023-08-25T09:02:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.509+0000 7fb23223cb80 20 client.15513 trim_cache size 0 max 16384 2023-08-25T09:02:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.524+0000 7fb23223cb80 1 Processor -- start 2023-08-25T09:02:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.524+0000 7fb23223cb80 1 -- start start 2023-08-25T09:02:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.524+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.524+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.524+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.524+0000 7fb23223cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27177c8c0 con 0x55e2719a8990 2023-08-25T09:02:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.524+0000 7fb23223cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e271978160 con 0x55e2719adf40 2023-08-25T09:02:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.524+0000 7fb23223cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27199f720 con 0x55e2719ae860 2023-08-25T09:02:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43968/0 (socket says 172.21.15.99:43968) 2023-08-25T09:02:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 -- 172.21.15.99:0/552588128 learned_addr learned my addr 172.21.15.99:0/552588128 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb21cf9c700 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb2177fe700 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 -- 172.21.15.99:0/552588128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 msgr2=0x55e2719b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 -- 172.21.15.99:0/552588128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 msgr2=0x55e2719b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 -- 172.21.15.99:0/552588128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e27170fc00 con 0x55e2719a8990 2023-08-25T09:02:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb217fff700 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719a8450 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb20800fa50 tx=0x7fb208005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcae7687d118aa0b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb1feffd700 1 -- 172.21.15.99:0/552588128 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208021a80 con 0x55e2719a8990 2023-08-25T09:02:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.525+0000 7fb1feffd700 1 -- 172.21.15.99:0/552588128 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20801d070 con 0x55e2719a8990 2023-08-25T09:02:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb1feffd700 1 -- 172.21.15.99:0/552588128 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208003680 con 0x55e2719a8990 2023-08-25T09:02:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 -- 172.21.15.99:0/552588128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 msgr2=0x55e2719a8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719a8450 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb20800fa50 tx=0x7fb208005050 comp rx=0 tx=0).stop 2023-08-25T09:02:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 -- 172.21.15.99:0/552588128 shutdown_connections 2023-08-25T09:02:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719a8450 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 --2- 172.21.15.99:0/552588128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 -- 172.21.15.99:0/552588128 >> 172.21.15.99:0/552588128 conn(0x55e27191d080 msgr2=0x55e2719b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 -- 172.21.15.99:0/552588128 shutdown_connections 2023-08-25T09:02:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 -- 172.21.15.99:0/552588128 wait complete. 2023-08-25T09:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.526+0000 7fb23223cb80 1 -- start start 2023-08-25T09:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb23223cb80 1 Processor -- start 2023-08-25T09:02:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719aa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb23223cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb23223cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2719ab500 con 0x55e2719a8990 2023-08-25T09:02:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb23223cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2719a9dc0 con 0x55e2719adf40 2023-08-25T09:02:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb23223cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2719a9f80 con 0x55e2719ae860 2023-08-25T09:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb23223cb80 10 client.0 _ll_register_callbacks cb 0x7fb208009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb217fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb2177fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb217fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43970/0 (socket says 172.21.15.99:43970) 2023-08-25T09:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.527+0000 7fb217fff700 1 -- 172.21.15.99:0/4219715894 learned_addr learned my addr 172.21.15.99:0/4219715894 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb21cf9c700 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb2177fe700 1 -- 172.21.15.99:0/4219715894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 msgr2=0x55e2719a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb2177fe700 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb2177fe700 1 -- 172.21.15.99:0/4219715894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 msgr2=0x55e2719aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb2177fe700 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb2177fe700 1 -- 172.21.15.99:0/4219715894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e271726d80 con 0x55e2719adf40 2023-08-25T09:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb217fff700 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb2177fe700 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719aaeb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0c92f0 tx=0x7fb20c06c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10bc0a538fa40974 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:02:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb1f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:02:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb1f77fe700 1 -- 172.21.15.99:0/4219715894 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c098c10 con 0x55e2719adf40 2023-08-25T09:02:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb23223cb80 1 -- 172.21.15.99:0/4219715894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e27199b600 con 0x55e2719adf40 2023-08-25T09:02:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb1f77fe700 1 -- 172.21.15.99:0/4219715894 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb20c0999b0 con 0x55e2719adf40 2023-08-25T09:02:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.528+0000 7fb1f77fe700 1 -- 172.21.15.99:0/4219715894 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb20c016330 con 0x55e2719adf40 2023-08-25T09:02:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb23223cb80 10 client.25574 fetch_fsmap learned FSMap version 27 2023-08-25T09:02:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb23223cb80 1 -- 172.21.15.99:0/4219715894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e27174d1f0 con 0x55e2719adf40 2023-08-25T09:02:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb1f77fe700 1 -- 172.21.15.99:0/4219715894 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20c063080 con 0x55e2719adf40 2023-08-25T09:02:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb1f77fe700 1 -- 172.21.15.99:0/4219715894 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb20c065120 con 0x55e2719adf40 2023-08-25T09:02:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb23223cb80 10 client.25574 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T09:02:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb23223cb80 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:02:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb23223cb80 20 client.25574 populate_metadata read hostname 'smithi099' 2023-08-25T09:02:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb23223cb80 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e271a16790 0x55e271a18bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:02:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb23223cb80 4 client.25574 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:02:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb23223cb80 1 -- 172.21.15.99:0/4219715894 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e2718f3910 con 0x55e271a16790 2023-08-25T09:02:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.529+0000 7fb217fff700 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e271a16790 0x55e271a18bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:02:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.530+0000 7fb217fff700 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e271a16790 0x55e271a18bb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d819facdc3b230 server_cookie=2ac84713da4476f7 in_seq=0 out_seq=0 2023-08-25T09:02:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.530+0000 7fb1f77fe700 10 client.25574 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.557+0000 7fb1f77fe700 1 -- 172.21.15.99:0/4219715894 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e2718f3910 con 0x55e271a16790 2023-08-25T09:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.557+0000 7fb1f77fe700 10 client.25574 handle_command_reply: tid=0 2023-08-25T09:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.557+0000 7fb23223cb80 1 client.25574 shutdown 2023-08-25T09:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 -- 172.21.15.99:0/4219715894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 msgr2=0x55e2719aaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719aaeb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0c92f0 tx=0x7fb20c06c3d0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 -- 172.21.15.99:0/4219715894 shutdown_connections 2023-08-25T09:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e2719ae860 0x55e2719a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2719a8990 0x55e2719aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e271a16790 0x55e271a18bb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 --2- 172.21.15.99:0/4219715894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2719adf40 0x55e2719aaeb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb1f77fe700 0 client.25574 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 -- 172.21.15.99:0/4219715894 >> 172.21.15.99:0/4219715894 conn(0x55e27191d080 msgr2=0x55e2719aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 -- 172.21.15.99:0/4219715894 shutdown_connections 2023-08-25T09:02:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 1 -- 172.21.15.99:0/4219715894 wait complete. 2023-08-25T09:02:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.558+0000 7fb23223cb80 20 client.25574 trim_cache size 0 max 16384 2023-08-25T09:02:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:02:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:02:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:02:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:02:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 -- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb2000a00c0 msgr2=0x7fb2000a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb2000a00c0 0x7fb2000a2580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb210003fd0 tx=0x7fb210004b40 comp rx=0 tx=0).stop 2023-08-25T09:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 -- 172.21.15.99:0/3353158794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb218069620 msgr2=0x7fb2180ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb218069620 0x7fb2180ffda0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00ec10 tx=0x7fb20c00ef20 comp rx=0 tx=0).stop 2023-08-25T09:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 -- 172.21.15.99:0/3353158794 shutdown_connections 2023-08-25T09:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb2000a00c0 0x7fb2000a2580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2181068c0 0x7fb2181002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218068c80 0x7fb2180ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 --2- 172.21.15.99:0/3353158794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb218069620 0x7fb2180ffda0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 -- 172.21.15.99:0/3353158794 >> 172.21.15.99:0/3353158794 conn(0x7fb218076920 msgr2=0x7fb2180fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 -- 172.21.15.99:0/3353158794 shutdown_connections 2023-08-25T09:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:02:51.559+0000 7fb21e9ff700 1 -- 172.21.15.99:0/3353158794 wait complete. 2023-08-25T09:02:51.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:51 smithi157 ceph-mon[130879]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 263 KiB/s wr, 51 op/s 2023-08-25T09:02:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:51 smithi181 ceph-mon[131296]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 263 KiB/s wr, 51 op/s 2023-08-25T09:02:51.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:51 smithi099 ceph-mon[97641]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 263 KiB/s wr, 51 op/s 2023-08-25T09:02:51.902 INFO:tasks.workunit.client.0.smithi099.stdout: 1 85967 1.68 MB/sec execute 57 sec latency 3990.937 ms 2023-08-25T09:02:52.902 INFO:tasks.workunit.client.0.smithi099.stdout: 1 86089 1.66 MB/sec execute 58 sec latency 4908.424 ms 2023-08-25T09:02:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:53 smithi157 ceph-mon[130879]: pgmap v114: 97 pgs: 97 active+clean; 45 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 616 KiB/s wr, 70 op/s 2023-08-25T09:02:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:53 smithi181 ceph-mon[131296]: pgmap v114: 97 pgs: 97 active+clean; 45 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 616 KiB/s wr, 70 op/s 2023-08-25T09:02:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:53 smithi099 ceph-mon[97641]: pgmap v114: 97 pgs: 97 active+clean; 45 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 616 KiB/s wr, 70 op/s 2023-08-25T09:02:53.902 INFO:tasks.workunit.client.0.smithi099.stdout: 1 88687 1.91 MB/sec execute 59 sec latency 11.304 ms 2023-08-25T09:02:54.903 INFO:tasks.workunit.client.0.smithi099.stdout: 1 89448 1.91 MB/sec execute 60 sec latency 177.862 ms 2023-08-25T09:02:55.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:55 smithi157 ceph-mon[130879]: pgmap v115: 97 pgs: 97 active+clean; 45 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 565 KiB/s wr, 52 op/s 2023-08-25T09:02:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:55 smithi181 ceph-mon[131296]: pgmap v115: 97 pgs: 97 active+clean; 45 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 565 KiB/s wr, 52 op/s 2023-08-25T09:02:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:55 smithi099 ceph-mon[97641]: pgmap v115: 97 pgs: 97 active+clean; 45 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 565 KiB/s wr, 52 op/s 2023-08-25T09:02:55.903 INFO:tasks.workunit.client.0.smithi099.stdout: 1 89448 1.88 MB/sec execute 61 sec latency 1178.015 ms 2023-08-25T09:02:56.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:56 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:56.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:56 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:56.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:56 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:02:56.903 INFO:tasks.workunit.client.0.smithi099.stdout: 1 89448 1.85 MB/sec execute 62 sec latency 2178.160 ms 2023-08-25T09:02:57.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:57 smithi157 ceph-mon[130879]: pgmap v116: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2023-08-25T09:02:57.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:57 smithi181 ceph-mon[131296]: pgmap v116: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2023-08-25T09:02:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:57 smithi099 ceph-mon[97641]: pgmap v116: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2023-08-25T09:02:57.903 INFO:tasks.workunit.client.0.smithi099.stdout: 1 89516 1.82 MB/sec execute 63 sec latency 3090.702 ms 2023-08-25T09:02:58.903 INFO:tasks.workunit.client.0.smithi099.stdout: 1 89516 1.80 MB/sec execute 64 sec latency 1025.890 ms 2023-08-25T09:02:59.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:02:59 smithi099 ceph-mon[97641]: pgmap v117: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 832 KiB/s wr, 75 op/s 2023-08-25T09:02:59.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:02:59 smithi157 ceph-mon[130879]: pgmap v117: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 832 KiB/s wr, 75 op/s 2023-08-25T09:02:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:02:59 smithi181 ceph-mon[131296]: pgmap v117: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 832 KiB/s wr, 75 op/s 2023-08-25T09:02:59.903 INFO:tasks.workunit.client.0.smithi099.stdout: 1 89516 1.77 MB/sec execute 65 sec latency 2026.035 ms 2023-08-25T09:03:00.903 INFO:tasks.workunit.client.0.smithi099.stdout: 1 90693 1.82 MB/sec execute 66 sec latency 2161.907 ms 2023-08-25T09:03:01.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:01 smithi099 ceph-mon[97641]: pgmap v118: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 832 KiB/s wr, 75 op/s 2023-08-25T09:03:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:01 smithi157 ceph-mon[130879]: pgmap v118: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 832 KiB/s wr, 75 op/s 2023-08-25T09:03:01.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:01 smithi181 ceph-mon[131296]: pgmap v118: 97 pgs: 97 active+clean; 47 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 832 KiB/s wr, 75 op/s 2023-08-25T09:03:01.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 92669 1.99 MB/sec execute 67 sec latency 11.636 ms 2023-08-25T09:03:02.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 93059 1.97 MB/sec execute 68 sec latency 631.140 ms 2023-08-25T09:03:03.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:03 smithi099 ceph-mon[97641]: pgmap v119: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-08-25T09:03:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:03 smithi157 ceph-mon[130879]: pgmap v119: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-08-25T09:03:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:03 smithi181 ceph-mon[131296]: pgmap v119: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-08-25T09:03:03.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 93060 1.94 MB/sec execute 69 sec latency 998.524 ms 2023-08-25T09:03:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 -- 172.21.15.99:0/3540181405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c108b80 msgr2=0x7f5c4c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3540181405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c108b80 0x7f5c4c108f60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009990 tx=0x7f5c48009ca0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 -- 172.21.15.99:0/3540181405 shutdown_connections 2023-08-25T09:03:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3540181405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c1034a0 0x7f5c4c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3540181405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c108b80 0x7f5c4c108f60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3540181405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c102ae0 0x7f5c4c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 -- 172.21.15.99:0/3540181405 >> 172.21.15.99:0/3540181405 conn(0x7f5c4c0fe5b0 msgr2=0x7f5c4c1009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 -- 172.21.15.99:0/3540181405 shutdown_connections 2023-08-25T09:03:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.709+0000 7f5c53fff700 1 -- 172.21.15.99:0/3540181405 wait complete. 2023-08-25T09:03:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.710+0000 7f5c53fff700 1 Processor -- start 2023-08-25T09:03:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c53fff700 1 -- start start 2023-08-25T09:03:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c53fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c102ae0 0x7f5c4c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c53fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c1034a0 0x7f5c4c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c53fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c108b80 0x7f5c4c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c53fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c1990c0 con 0x7f5c4c108b80 2023-08-25T09:03:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c53fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c199230 con 0x7f5c4c102ae0 2023-08-25T09:03:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c53fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c1993f0 con 0x7f5c4c1034a0 2023-08-25T09:03:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c51d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c102ae0 0x7f5c4c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c108b80 0x7f5c4c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c51d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c102ae0 0x7f5c4c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52404/0 (socket says 172.21.15.99:52404) 2023-08-25T09:03:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c51d9b700 1 -- 172.21.15.99:0/3956162823 learned_addr learned my addr 172.21.15.99:0/3956162823 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.711+0000 7f5c5159a700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c1034a0 0x7f5c4c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c51d9b700 1 -- 172.21.15.99:0/3956162823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c1034a0 msgr2=0x7f5c4c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c1034a0 0x7f5c4c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c51d9b700 1 -- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c108b80 msgr2=0x7f5c4c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c108b80 0x7f5c4c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c51d9b700 1 -- 172.21.15.99:0/3956162823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c480095f0 con 0x7f5c4c102ae0 2023-08-25T09:03:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c102ae0 0x7f5c4c19ebf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005910 tx=0x7f5c48005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c42ffd700 1 -- 172.21.15.99:0/3956162823 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4801d070 con 0x7f5c4c102ae0 2023-08-25T09:03:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c53fff700 1 -- 172.21.15.99:0/3956162823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c199670 con 0x7f5c4c102ae0 2023-08-25T09:03:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c42ffd700 1 -- 172.21.15.99:0/3956162823 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c48022470 con 0x7f5c4c102ae0 2023-08-25T09:03:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c42ffd700 1 -- 172.21.15.99:0/3956162823 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48017650 con 0x7f5c4c102ae0 2023-08-25T09:03:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.712+0000 7f5c53fff700 1 -- 172.21.15.99:0/3956162823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c1a5be0 con 0x7f5c4c102ae0 2023-08-25T09:03:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.716+0000 7f5c42ffd700 1 -- 172.21.15.99:0/3956162823 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f5c4800fad0 con 0x7f5c4c102ae0 2023-08-25T09:03:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.717+0000 7f5c6787bb80 1 Processor -- start 2023-08-25T09:03:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.717+0000 7f5c6787bb80 1 -- start start 2023-08-25T09:03:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0badd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b022750 con 0x55bf6b0c0860 2023-08-25T09:03:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b013720 con 0x55bf6b0bff40 2023-08-25T09:03:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b011040 con 0x55bf6b0ba990 2023-08-25T09:03:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c42ffd700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c380a00c0 0x7f5c380a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52410/0 (socket says 172.21.15.99:52410) 2023-08-25T09:03:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 -- 172.21.15.99:0/183141051 learned_addr learned my addr 172.21.15.99:0/183141051 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5159a700 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0badd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c42ffd700 1 -- 172.21.15.99:0/3956162823 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5c480ccb60 con 0x7f5c4c102ae0 2023-08-25T09:03:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c380a00c0 0x7f5c380a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 -- 172.21.15.99:0/183141051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 msgr2=0x55bf6b0badd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0badd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 -- 172.21.15.99:0/183141051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 msgr2=0x55bf6b0ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0ba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.718+0000 7f5c5259c700 1 -- 172.21.15.99:0/183141051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf6ae8c5d0 con 0x55bf6b0bff40 2023-08-25T09:03:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.719+0000 7f5c5259c700 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c0320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009f60 tx=0x7f5c44009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df75c44c5b815951 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.719+0000 7f5c5259c700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c380a00c0 0x7f5c380a2580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c4401b000 tx=0x7f5c4401d000 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.719+0000 7f5c37fff700 1 -- 172.21.15.99:0/183141051 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440098c0 con 0x55bf6b0bff40 2023-08-25T09:03:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.719+0000 7f5c37fff700 1 -- 172.21.15.99:0/183141051 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c4400f040 con 0x55bf6b0bff40 2023-08-25T09:03:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.719+0000 7f5c37fff700 1 -- 172.21.15.99:0/183141051 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4401ab30 con 0x55bf6b0bff40 2023-08-25T09:03:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 -- 172.21.15.99:0/183141051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 msgr2=0x55bf6b0c0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c0320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009f60 tx=0x7f5c44009f90 comp rx=0 tx=0).stop 2023-08-25T09:03:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 -- 172.21.15.99:0/183141051 shutdown_connections 2023-08-25T09:03:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0badd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c0320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/183141051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 -- 172.21.15.99:0/183141051 >> 172.21.15.99:0/183141051 conn(0x55bf6b02f080 msgr2=0x55bf6b0b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 -- 172.21.15.99:0/183141051 shutdown_connections 2023-08-25T09:03:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 -- 172.21.15.99:0/183141051 wait complete. 2023-08-25T09:03:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 -- start start 2023-08-25T09:03:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.720+0000 7f5c6787bb80 1 Processor -- start 2023-08-25T09:03:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0ba990 0x55bf6b0c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0c0860 0x55bf6b0c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b00c930 con 0x55bf6b0ba990 2023-08-25T09:03:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6afaab50 con 0x55bf6b0bff40 2023-08-25T09:03:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6afaa850 con 0x55bf6b0c0860 2023-08-25T09:03:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c6787bb80 10 client.0 _ll_register_callbacks cb 0x55bf6aef4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c51d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0ba990 0x55bf6b0c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c51d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0ba990 0x55bf6b0c39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40232/0 (socket says 172.21.15.99:40232) 2023-08-25T09:03:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c51d9b700 1 -- 172.21.15.99:0/2201434510 learned_addr learned my addr 172.21.15.99:0/2201434510 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c5159a700 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.721+0000 7f5c5259c700 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0c0860 0x55bf6b0c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c51d9b700 1 -- 172.21.15.99:0/2201434510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0c0860 msgr2=0x55bf6b0c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0c0860 0x55bf6b0c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c51d9b700 1 -- 172.21.15.99:0/2201434510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 msgr2=0x55bf6b0c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c51d9b700 1 -- 172.21.15.99:0/2201434510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c44009b40 con 0x55bf6b0ba990 2023-08-25T09:03:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0ba990 0x55bf6b0c39b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ca270 tx=0x7f5c48022c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c637a1ba43da4993 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c357fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:03:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c357fa700 1 -- 172.21.15.99:0/2201434510 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4809a960 con 0x55bf6b0ba990 2023-08-25T09:03:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c357fa700 1 -- 172.21.15.99:0/2201434510 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5c4809a260 con 0x55bf6b0ba990 2023-08-25T09:03:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c6787bb80 1 -- 172.21.15.99:0/2201434510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf6b08a160 con 0x55bf6b0ba990 2023-08-25T09:03:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c357fa700 1 -- 172.21.15.99:0/2201434510 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4809e090 con 0x55bf6b0ba990 2023-08-25T09:03:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c6787bb80 10 client.15531 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c6787bb80 1 -- 172.21.15.99:0/2201434510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bf6ae38d80 con 0x55bf6b0ba990 2023-08-25T09:03:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.722+0000 7f5c357fa700 1 -- 172.21.15.99:0/2201434510 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf6b08a160 con 0x55bf6b0ba990 2023-08-25T09:03:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.723+0000 7f5c357fa700 1 -- 172.21.15.99:0/2201434510 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5c48099750 con 0x55bf6b0ba990 2023-08-25T09:03:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.723+0000 7f5c6787bb80 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.723+0000 7f5c6787bb80 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.723+0000 7f5c6787bb80 20 client.15531 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.723+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bf6b0d3e90 0x55bf6b0d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.723+0000 7f5c6787bb80 4 client.15531 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:03:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.723+0000 7f5c6787bb80 1 -- 172.21.15.99:0/2201434510 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf6afb2b40 con 0x55bf6b0d3e90 2023-08-25T09:03:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.723+0000 7f5c5159a700 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bf6b0d3e90 0x55bf6b0d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.724+0000 7f5c357fa700 10 client.15531 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.724+0000 7f5c5159a700 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bf6b0d3e90 0x55bf6b0d62b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62cc32e00d110256 server_cookie=e3dcf6316fddf907 in_seq=0 out_seq=0 2023-08-25T09:03:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.724+0000 7f5c357fa700 1 -- 172.21.15.99:0/2201434510 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bf6afb2b40 con 0x55bf6b0d3e90 2023-08-25T09:03:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.724+0000 7f5c357fa700 10 client.15531 handle_command_reply: tid=0 2023-08-25T09:03:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.724+0000 7f5c6787bb80 1 client.15531 shutdown 2023-08-25T09:03:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.724+0000 7f5c6787bb80 1 -- 172.21.15.99:0/2201434510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0ba990 msgr2=0x55bf6b0c39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.724+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0ba990 0x55bf6b0c39b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ca270 tx=0x7f5c48022c20 comp rx=0 tx=0).stop 2023-08-25T09:03:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 1 -- 172.21.15.99:0/2201434510 shutdown_connections 2023-08-25T09:03:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0c0860 0x55bf6b0c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0ba990 0x55bf6b0c39b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bf6b0d3e90 0x55bf6b0d62b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/2201434510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 1 -- 172.21.15.99:0/2201434510 >> 172.21.15.99:0/2201434510 conn(0x55bf6b02f080 msgr2=0x55bf6b0af820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c357fa700 0 client.15531 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 1 -- 172.21.15.99:0/2201434510 shutdown_connections 2023-08-25T09:03:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 1 -- 172.21.15.99:0/2201434510 wait complete. 2023-08-25T09:03:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.725+0000 7f5c6787bb80 20 client.15531 trim_cache size 0 max 16384 2023-08-25T09:03:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.740+0000 7f5c6787bb80 1 Processor -- start 2023-08-25T09:03:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.740+0000 7f5c6787bb80 1 -- start start 2023-08-25T09:03:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6ae8e8c0 con 0x55bf6b0c0860 2023-08-25T09:03:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b0b1720 con 0x55bf6b0bff40 2023-08-25T09:03:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b0d3cb0 con 0x55bf6b0ba990 2023-08-25T09:03:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40244/0 (socket says 172.21.15.99:40244) 2023-08-25T09:03:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5159a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5259c700 1 -- 172.21.15.99:0/3572901426 learned_addr learned my addr 172.21.15.99:0/3572901426 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5259c700 1 -- 172.21.15.99:0/3572901426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 msgr2=0x55bf6b0ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5259c700 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0ba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5259c700 1 -- 172.21.15.99:0/3572901426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 msgr2=0x55bf6b0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5259c700 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5259c700 1 -- 172.21.15.99:0/3572901426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf6b0aeb40 con 0x55bf6b0c0860 2023-08-25T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.741+0000 7f5c5159a700 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.742+0000 7f5c5259c700 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bea50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009f60 tx=0x7f5c440091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0583c9d60c3ba7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.742+0000 7f5c40ff9700 1 -- 172.21.15.99:0/3572901426 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4401ae20 con 0x55bf6b0c0860 2023-08-25T09:03:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.742+0000 7f5c40ff9700 1 -- 172.21.15.99:0/3572901426 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c440097e0 con 0x55bf6b0c0860 2023-08-25T09:03:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.742+0000 7f5c40ff9700 1 -- 172.21.15.99:0/3572901426 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44007e70 con 0x55bf6b0c0860 2023-08-25T09:03:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 -- 172.21.15.99:0/3572901426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 msgr2=0x55bf6b0bea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bea50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009f60 tx=0x7f5c440091e0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 -- 172.21.15.99:0/3572901426 shutdown_connections 2023-08-25T09:03:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bea50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/3572901426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0bff40 0x55bf6b0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 -- 172.21.15.99:0/3572901426 >> 172.21.15.99:0/3572901426 conn(0x55bf6b02f080 msgr2=0x55bf6b0bdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 -- 172.21.15.99:0/3572901426 shutdown_connections 2023-08-25T09:03:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 -- 172.21.15.99:0/3572901426 wait complete. 2023-08-25T09:03:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 -- start start 2023-08-25T09:03:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.743+0000 7f5c6787bb80 1 Processor -- start 2023-08-25T09:03:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0bd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0bff40 0x55bf6b0bba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c6787bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b0bc750 con 0x55bf6b0bff40 2023-08-25T09:03:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b0bc8c0 con 0x55bf6b0c0860 2023-08-25T09:03:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c6787bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6b127cc0 con 0x55bf6b0ba990 2023-08-25T09:03:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c6787bb80 10 client.0 _ll_register_callbacks cb 0x7f5c480094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5159a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0bff40 0x55bf6b0bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5159a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0bff40 0x55bf6b0bba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40252/0 (socket says 172.21.15.99:40252) 2023-08-25T09:03:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5159a700 1 -- 172.21.15.99:0/1878160028 learned_addr learned my addr 172.21.15.99:0/1878160028 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0bd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5259c700 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5159a700 1 -- 172.21.15.99:0/1878160028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 msgr2=0x55bf6b0bd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5159a700 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0bd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5159a700 1 -- 172.21.15.99:0/1878160028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0c0860 msgr2=0x55bf6b0bc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5159a700 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.744+0000 7f5c5159a700 1 -- 172.21.15.99:0/1878160028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c009c70 con 0x55bf6b0bff40 2023-08-25T09:03:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c5159a700 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0bff40 0x55bf6b0bba80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55bf6b0d6bc0 tx=0x7f5c3c0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe54d17cebd5c5f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c357fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:03:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c357fa700 1 -- 172.21.15.99:0/1878160028 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c016070 con 0x55bf6b0bff40 2023-08-25T09:03:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c6787bb80 1 -- 172.21.15.99:0/1878160028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf6b0ad600 con 0x55bf6b0bff40 2023-08-25T09:03:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c357fa700 1 -- 172.21.15.99:0/1878160028 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5c3c003c40 con 0x55bf6b0bff40 2023-08-25T09:03:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c357fa700 1 -- 172.21.15.99:0/1878160028 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c3c01d5a0 con 0x55bf6b0bff40 2023-08-25T09:03:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c6787bb80 10 client.15537 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c6787bb80 1 -- 172.21.15.99:0/1878160028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bf6ae8c5d0 con 0x55bf6b0bff40 2023-08-25T09:03:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.745+0000 7f5c357fa700 1 -- 172.21.15.99:0/1878160028 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf6b0ad600 con 0x55bf6b0bff40 2023-08-25T09:03:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c357fa700 1 -- 172.21.15.99:0/1878160028 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5c3c023030 con 0x55bf6b0bff40 2023-08-25T09:03:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c6787bb80 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c6787bb80 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c6787bb80 20 client.15537 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bf6b1294e0 0x55bf6b12b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c6787bb80 4 client.15537 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:03:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c6787bb80 1 -- 172.21.15.99:0/1878160028 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bf6b005910 con 0x55bf6b1294e0 2023-08-25T09:03:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bf6b1294e0 0x55bf6b12b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c51d9b700 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bf6b1294e0 0x55bf6b12b900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7fbb2b45b62991 server_cookie=f8db8470443dab6c in_seq=0 out_seq=0 2023-08-25T09:03:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.746+0000 7f5c357fa700 10 client.15537 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.795+0000 7f5c357fa700 1 -- 172.21.15.99:0/1878160028 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bf6b005910 con 0x55bf6b1294e0 2023-08-25T09:03:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.795+0000 7f5c357fa700 10 client.15537 handle_command_reply: tid=0 2023-08-25T09:03:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.795+0000 7f5c6787bb80 1 client.15537 shutdown 2023-08-25T09:03:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 -- 172.21.15.99:0/1878160028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0bff40 msgr2=0x55bf6b0bba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0bff40 0x55bf6b0bba80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55bf6b0d6bc0 tx=0x7f5c3c0043d0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 -- 172.21.15.99:0/1878160028 shutdown_connections 2023-08-25T09:03:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bf6b1294e0 0x55bf6b12b900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf6b0c0860 0x55bf6b0bc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf6b0ba990 0x55bf6b0bd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 --2- 172.21.15.99:0/1878160028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf6b0bff40 0x55bf6b0bba80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c357fa700 0 client.15537 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 -- 172.21.15.99:0/1878160028 >> 172.21.15.99:0/1878160028 conn(0x55bf6b02f080 msgr2=0x55bf6b0bea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 -- 172.21.15.99:0/1878160028 shutdown_connections 2023-08-25T09:03:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 1 -- 172.21.15.99:0/1878160028 wait complete. 2023-08-25T09:03:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.796+0000 7f5c6787bb80 20 client.15537 trim_cache size 0 max 16384 2023-08-25T09:03:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:03:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:03:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:03:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:03:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 -- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c380a00c0 msgr2=0x7f5c380a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c380a00c0 0x7f5c380a2580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c4401b000 tx=0x7f5c4401d000 comp rx=0 tx=0).stop 2023-08-25T09:03:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 -- 172.21.15.99:0/3956162823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c102ae0 msgr2=0x7f5c4c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c102ae0 0x7f5c4c19ebf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005910 tx=0x7f5c48005740 comp rx=0 tx=0).stop 2023-08-25T09:03:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 -- 172.21.15.99:0/3956162823 shutdown_connections 2023-08-25T09:03:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c380a00c0 0x7f5c380a2580 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c108b80 0x7f5c4c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c102ae0 0x7f5c4c19ebf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 --2- 172.21.15.99:0/3956162823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c1034a0 0x7f5c4c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 -- 172.21.15.99:0/3956162823 >> 172.21.15.99:0/3956162823 conn(0x7f5c4c0fe5b0 msgr2=0x7f5c4c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 -- 172.21.15.99:0/3956162823 shutdown_connections 2023-08-25T09:03:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:04.797+0000 7f5c53fff700 1 -- 172.21.15.99:0/3956162823 wait complete. 2023-08-25T09:03:04.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 93060 1.92 MB/sec execute 70 sec latency 1998.609 ms 2023-08-25T09:03:05.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:05 smithi157 ceph-mon[130879]: pgmap v120: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 761 KiB/s wr, 111 op/s 2023-08-25T09:03:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:05 smithi181 ceph-mon[131296]: pgmap v120: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 761 KiB/s wr, 111 op/s 2023-08-25T09:03:05.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:05 smithi099 ceph-mon[97641]: pgmap v120: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 761 KiB/s wr, 111 op/s 2023-08-25T09:03:05.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 93060 1.89 MB/sec execute 71 sec latency 2998.674 ms 2023-08-25T09:03:06.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 93060 1.86 MB/sec execute 72 sec latency 3998.753 ms 2023-08-25T09:03:07.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:07 smithi099 ceph-mon[97641]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2023-08-25T09:03:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:07 smithi157 ceph-mon[130879]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2023-08-25T09:03:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:07 smithi181 ceph-mon[131296]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2023-08-25T09:03:07.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 93193 1.84 MB/sec execute 73 sec latency 4911.000 ms 2023-08-25T09:03:08.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 95286 1.98 MB/sec execute 74 sec latency 20.845 ms 2023-08-25T09:03:09.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:09 smithi099 ceph-mon[97641]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 705 KiB/s wr, 101 op/s 2023-08-25T09:03:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:09 smithi157 ceph-mon[130879]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 705 KiB/s wr, 101 op/s 2023-08-25T09:03:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:09 smithi181 ceph-mon[131296]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 705 KiB/s wr, 101 op/s 2023-08-25T09:03:09.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96483 2.04 MB/sec execute 75 sec latency 137.149 ms 2023-08-25T09:03:10.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96483 2.01 MB/sec execute 76 sec latency 1137.278 ms 2023-08-25T09:03:11.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:11 smithi099 ceph-mon[97641]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 705 KiB/s wr, 101 op/s 2023-08-25T09:03:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:11 smithi157 ceph-mon[130879]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 705 KiB/s wr, 101 op/s 2023-08-25T09:03:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:11 smithi181 ceph-mon[131296]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 705 KiB/s wr, 101 op/s 2023-08-25T09:03:11.904 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96483 1.99 MB/sec execute 77 sec latency 2137.418 ms 2023-08-25T09:03:12.905 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96539 1.96 MB/sec execute 78 sec latency 3051.624 ms 2023-08-25T09:03:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:13 smithi099 ceph-mon[97641]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 946 KiB/s wr, 138 op/s 2023-08-25T09:03:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:13 smithi157 ceph-mon[130879]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 946 KiB/s wr, 138 op/s 2023-08-25T09:03:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:13 smithi181 ceph-mon[131296]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 946 KiB/s wr, 138 op/s 2023-08-25T09:03:13.905 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96539 1.94 MB/sec execute 79 sec latency 1041.123 ms 2023-08-25T09:03:14.905 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96539 1.91 MB/sec execute 80 sec latency 2041.209 ms 2023-08-25T09:03:15.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:15 smithi157 ceph-mon[130879]: pgmap v125: 97 pgs: 97 active+clean; 48 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 664 KiB/s wr, 82 op/s 2023-08-25T09:03:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:15 smithi181 ceph-mon[131296]: pgmap v125: 97 pgs: 97 active+clean; 48 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 664 KiB/s wr, 82 op/s 2023-08-25T09:03:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:15 smithi099 ceph-mon[97641]: pgmap v125: 97 pgs: 97 active+clean; 48 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 664 KiB/s wr, 82 op/s 2023-08-25T09:03:15.905 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96539 1.89 MB/sec execute 81 sec latency 3041.300 ms 2023-08-25T09:03:16.905 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96539 1.87 MB/sec execute 82 sec latency 4041.431 ms 2023-08-25T09:03:17.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:17 smithi099 ceph-mon[97641]: pgmap v126: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 927 KiB/s wr, 95 op/s 2023-08-25T09:03:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:17 smithi157 ceph-mon[130879]: pgmap v126: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 927 KiB/s wr, 95 op/s 2023-08-25T09:03:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:17 smithi181 ceph-mon[131296]: pgmap v126: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 927 KiB/s wr, 95 op/s 2023-08-25T09:03:17.905 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96684 1.85 MB/sec execute 83 sec latency 4952.598 ms 2023-08-25T09:03:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 -- 172.21.15.99:0/1326338909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 msgr2=0x7f6f54100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 --2- 172.21.15.99:0/1326338909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 0x7f6f54100d20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009990 tx=0x7f6f48009ca0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 -- 172.21.15.99:0/1326338909 shutdown_connections 2023-08-25T09:03:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 --2- 172.21.15.99:0/1326338909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f54101260 0x7f6f5410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 --2- 172.21.15.99:0/1326338909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f541068b0 0x7f6f54106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 --2- 172.21.15.99:0/1326338909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 0x7f6f54100d20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 -- 172.21.15.99:0/1326338909 >> 172.21.15.99:0/1326338909 conn(0x7f6f540fc370 msgr2=0x7f6f540fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 -- 172.21.15.99:0/1326338909 shutdown_connections 2023-08-25T09:03:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.963+0000 7f6f589ff700 1 -- 172.21.15.99:0/1326338909 wait complete. 2023-08-25T09:03:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.964+0000 7f6f589ff700 1 Processor -- start 2023-08-25T09:03:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f589ff700 1 -- start start 2023-08-25T09:03:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f589ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 0x7f6f54072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f589ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f54101260 0x7f6f54073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f589ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f541068b0 0x7f6f541a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f589ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541a50d0 con 0x7f6f541008a0 2023-08-25T09:03:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f589ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541a5210 con 0x7f6f541068b0 2023-08-25T09:03:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f589ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541a5350 con 0x7f6f54101260 2023-08-25T09:03:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 0x7f6f54072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f51d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f54101260 0x7f6f54073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f51d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f54101260 0x7f6f54073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:51310/0 (socket says 172.21.15.99:51310) 2023-08-25T09:03:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f51d9b700 1 -- 172.21.15.99:0/2927224905 learned_addr learned my addr 172.21.15.99:0/2927224905 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.965+0000 7f6f52d9d700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f541068b0 0x7f6f541a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f52d9d700 1 -- 172.21.15.99:0/2927224905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f54101260 msgr2=0x7f6f54073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f52d9d700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f54101260 0x7f6f54073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f52d9d700 1 -- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 msgr2=0x7f6f54072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f52d9d700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 0x7f6f54072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f52d9d700 1 -- 172.21.15.99:0/2927224905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480095f0 con 0x7f6f541068b0 2023-08-25T09:03:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f54101260 0x7f6f54073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f5259c700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 0x7f6f54072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f52d9d700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f541068b0 0x7f6f541a4b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00efc0 tx=0x7f6f4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f437fe700 1 -- 172.21.15.99:0/2927224905 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c018070 con 0x7f6f541068b0 2023-08-25T09:03:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f589ff700 1 -- 172.21.15.99:0/2927224905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f541a55a0 con 0x7f6f541068b0 2023-08-25T09:03:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f437fe700 1 -- 172.21.15.99:0/2927224905 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4c00f040 con 0x7f6f541068b0 2023-08-25T09:03:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f437fe700 1 -- 172.21.15.99:0/2927224905 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c00cd40 con 0x7f6f541068b0 2023-08-25T09:03:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.966+0000 7f6f589ff700 1 -- 172.21.15.99:0/2927224905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f541a5ab0 con 0x7f6f541068b0 2023-08-25T09:03:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.971+0000 7f6f437fe700 1 -- 172.21.15.99:0/2927224905 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f6f4c003a00 con 0x7f6f541068b0 2023-08-25T09:03:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f6c316b80 1 Processor -- start 2023-08-25T09:03:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f6c316b80 1 -- start start 2023-08-25T09:03:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 0x55d1011dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d101147750 con 0x55d1011e5860 2023-08-25T09:03:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d101138720 con 0x55d1011df990 2023-08-25T09:03:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d101136040 con 0x55d1011e4f40 2023-08-25T09:03:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.972+0000 7f6f437fe700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f6f3c0a00c0 0x7f6f3c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:51316/0 (socket says 172.21.15.99:51316) 2023-08-25T09:03:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 -- 172.21.15.99:0/3101338141 learned_addr learned my addr 172.21.15.99:0/3101338141 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f437fe700 1 -- 172.21.15.99:0/2927224905 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f4c0cb2c0 con 0x7f6f541068b0 2023-08-25T09:03:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f5259c700 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 0x55d1011dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f52d9d700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f6f3c0a00c0 0x7f6f3c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 -- 172.21.15.99:0/3101338141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 msgr2=0x55d1011dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 0x55d1011dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 -- 172.21.15.99:0/3101338141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 msgr2=0x55d1011df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011df450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 -- 172.21.15.99:0/3101338141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d100fb15d0 con 0x55d1011e4f40 2023-08-25T09:03:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e5320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f48005cd0 tx=0x7f6f4801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=784ea86886ebeb5b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.973+0000 7f6f52d9d700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f6f3c0a00c0 0x7f6f3c0a2580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0c89d0 tx=0x7f6f4c009040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.974+0000 7f6f40ff9700 1 -- 172.21.15.99:0/3101338141 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800f460 con 0x55d1011e4f40 2023-08-25T09:03:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.974+0000 7f6f40ff9700 1 -- 172.21.15.99:0/3101338141 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4800fa80 con 0x55d1011e4f40 2023-08-25T09:03:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.974+0000 7f6f40ff9700 1 -- 172.21.15.99:0/3101338141 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f480179e0 con 0x55d1011e4f40 2023-08-25T09:03:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.974+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3101338141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 msgr2=0x55d1011e5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.974+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e5320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f48005cd0 tx=0x7f6f4801a040 comp rx=0 tx=0).stop 2023-08-25T09:03:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3101338141 shutdown_connections 2023-08-25T09:03:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 0x55d1011dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e5320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3101338141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3101338141 >> 172.21.15.99:0/3101338141 conn(0x55d101154080 msgr2=0x55d1011d9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3101338141 shutdown_connections 2023-08-25T09:03:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3101338141 wait complete. 2023-08-25T09:03:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 -- start start 2023-08-25T09:03:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.975+0000 7f6f6c316b80 1 Processor -- start 2023-08-25T09:03:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011df990 0x55d1011eb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d101131930 con 0x55d1011e5860 2023-08-25T09:03:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1010cfb50 con 0x55d1011e4f40 2023-08-25T09:03:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1010cf850 con 0x55d1011df990 2023-08-25T09:03:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f6c316b80 10 client.0 _ll_register_callbacks cb 0x55d101019a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f5259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f5259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011e88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40156/0 (socket says 172.21.15.99:40156) 2023-08-25T09:03:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f51d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011df990 0x55d1011eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f5259c700 1 -- 172.21.15.99:0/4245173883 learned_addr learned my addr 172.21.15.99:0/4245173883 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.976+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f5259c700 1 -- 172.21.15.99:0/4245173883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011df990 msgr2=0x55d1011eb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f5259c700 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011df990 0x55d1011eb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f5259c700 1 -- 172.21.15.99:0/4245173883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 msgr2=0x55d1011e8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f5259c700 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f5259c700 1 -- 172.21.15.99:0/4245173883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f48020040 con 0x55d1011e4f40 2023-08-25T09:03:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:03:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f5259c700 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011e88c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f44012d20 tx=0x7f6f4400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e76aa07478b3e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:03:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f39ffb700 1 -- 172.21.15.99:0/4245173883 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44017070 con 0x55d1011e4f40 2023-08-25T09:03:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f6c316b80 1 -- 172.21.15.99:0/4245173883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1011af160 con 0x55d1011e4f40 2023-08-25T09:03:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f39ffb700 1 -- 172.21.15.99:0/4245173883 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f4400fc30 con 0x55d1011e4f40 2023-08-25T09:03:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f39ffb700 1 -- 172.21.15.99:0/4245173883 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f4400fdf0 con 0x55d1011e4f40 2023-08-25T09:03:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f6c316b80 10 client.25604 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f6c316b80 1 -- 172.21.15.99:0/4245173883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d100f5dd80 con 0x55d1011e4f40 2023-08-25T09:03:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.977+0000 7f6f39ffb700 1 -- 172.21.15.99:0/4245173883 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1011af160 con 0x55d1011e4f40 2023-08-25T09:03:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f39ffb700 1 -- 172.21.15.99:0/4245173883 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6f4401c420 con 0x55d1011e4f40 2023-08-25T09:03:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f6c316b80 10 client.25604 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f6c316b80 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f6c316b80 20 client.25604 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1011f92b0 0x55d1011fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f6c316b80 4 client.25604 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:03:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f6c316b80 1 -- 172.21.15.99:0/4245173883 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1010d7b40 con 0x55d1011f92b0 2023-08-25T09:03:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1011f92b0 0x55d1011fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f39ffb700 10 client.25604 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.978+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1011f92b0 0x55d1011fb6d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c499d9218eb4b95 server_cookie=62b4500d4d17783f in_seq=0 out_seq=0 2023-08-25T09:03:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.979+0000 7f6f39ffb700 1 -- 172.21.15.99:0/4245173883 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d1010d7b40 con 0x55d1011f92b0 2023-08-25T09:03:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.979+0000 7f6f39ffb700 10 client.25604 handle_command_reply: tid=0 2023-08-25T09:03:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.979+0000 7f6f6c316b80 1 client.25604 shutdown 2023-08-25T09:03:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.979+0000 7f6f6c316b80 1 -- 172.21.15.99:0/4245173883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 msgr2=0x55d1011e88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.979+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011e88c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f44012d20 tx=0x7f6f4400f6a0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 1 -- 172.21.15.99:0/4245173883 shutdown_connections 2023-08-25T09:03:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011df990 0x55d1011eb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1011f92b0 0x55d1011fb6d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/4245173883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011e88c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f39ffb700 0 client.25604 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 1 -- 172.21.15.99:0/4245173883 >> 172.21.15.99:0/4245173883 conn(0x55d101154080 msgr2=0x55d1011d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 1 -- 172.21.15.99:0/4245173883 shutdown_connections 2023-08-25T09:03:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 1 -- 172.21.15.99:0/4245173883 wait complete. 2023-08-25T09:03:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.980+0000 7f6f6c316b80 20 client.25604 trim_cache size 0 max 16384 2023-08-25T09:03:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.996+0000 7f6f6c316b80 1 Processor -- start 2023-08-25T09:03:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.996+0000 7f6f6c316b80 1 -- start start 2023-08-25T09:03:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.996+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011df990 0x55d1011df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e5860 0x55d1011e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d100fb38c0 con 0x55d1011df990 2023-08-25T09:03:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1011d6700 con 0x55d1011e4f40 2023-08-25T09:03:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1011d2460 con 0x55d1011e5860 2023-08-25T09:03:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f51d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011df990 0x55d1011df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40168/0 (socket says 172.21.15.99:40168) 2023-08-25T09:03:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 -- 172.21.15.99:0/2896779432 learned_addr learned my addr 172.21.15.99:0/2896779432 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e5860 0x55d1011e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 -- 172.21.15.99:0/2896779432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e5860 msgr2=0x55d1011e80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e5860 0x55d1011e80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 -- 172.21.15.99:0/2896779432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011df990 msgr2=0x55d1011df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011df990 0x55d1011df450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 -- 172.21.15.99:0/2896779432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d100f46c00 con 0x55d1011e4f40 2023-08-25T09:03:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e5860 0x55d1011e80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:03:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.997+0000 7f6f5259c700 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011f0270 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6f4401c800 tx=0x7f6f440123a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d847b837380314e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.998+0000 7f6f417fa700 1 -- 172.21.15.99:0/2896779432 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44017070 con 0x55d1011e4f40 2023-08-25T09:03:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.998+0000 7f6f417fa700 1 -- 172.21.15.99:0/2896779432 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4401c4e0 con 0x55d1011e4f40 2023-08-25T09:03:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.998+0000 7f6f417fa700 1 -- 172.21.15.99:0/2896779432 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44004df0 con 0x55d1011e4f40 2023-08-25T09:03:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.998+0000 7f6f6c316b80 1 -- 172.21.15.99:0/2896779432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 msgr2=0x55d1011f0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.998+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011f0270 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6f4401c800 tx=0x7f6f440123a0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.998+0000 7f6f6c316b80 1 -- 172.21.15.99:0/2896779432 shutdown_connections 2023-08-25T09:03:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.999+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e5860 0x55d1011e80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.999+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011df990 0x55d1011df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.999+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/2896779432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011e4f40 0x55d1011f0270 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.999+0000 7f6f6c316b80 1 -- 172.21.15.99:0/2896779432 >> 172.21.15.99:0/2896779432 conn(0x55d101154080 msgr2=0x55d1011f54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.999+0000 7f6f6c316b80 1 -- 172.21.15.99:0/2896779432 shutdown_connections 2023-08-25T09:03:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.999+0000 7f6f6c316b80 1 -- 172.21.15.99:0/2896779432 wait complete. 2023-08-25T09:03:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.999+0000 7f6f6c316b80 1 -- start start 2023-08-25T09:03:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:17.999+0000 7f6f6c316b80 1 Processor -- start 2023-08-25T09:03:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 0x55d1011e1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f6c316b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1011e0ef0 con 0x55d1011e5860 2023-08-25T09:03:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1011e1060 con 0x55d1011df990 2023-08-25T09:03:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f6c316b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1011e1220 con 0x55d1011e4f40 2023-08-25T09:03:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f6c316b80 10 client.0 _ll_register_callbacks cb 0x7f6f440127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f5259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f5259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:51350/0 (socket says 172.21.15.99:51350) 2023-08-25T09:03:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f5259c700 1 -- 172.21.15.99:0/3672681335 learned_addr learned my addr 172.21.15.99:0/3672681335 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 0x55d1011e1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.000+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f5259c700 1 -- 172.21.15.99:0/3672681335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 msgr2=0x55d1011e1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f5259c700 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 0x55d1011e1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f5259c700 1 -- 172.21.15.99:0/3672681335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 msgr2=0x55d1011e08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f5259c700 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f5259c700 1 -- 172.21.15.99:0/3672681335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d100f5dd80 con 0x55d1011e4f40 2023-08-25T09:03:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:03:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f5259c700 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e2140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f44012070 tx=0x7f6f4400fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b1b994ff599f53 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:03:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f39ffb700 1 -- 172.21.15.99:0/3672681335 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44017040 con 0x55d1011e4f40 2023-08-25T09:03:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3672681335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1011d2600 con 0x55d1011e4f40 2023-08-25T09:03:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.001+0000 7f6f39ffb700 1 -- 172.21.15.99:0/3672681335 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f44005520 con 0x55d1011e4f40 2023-08-25T09:03:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f39ffb700 1 -- 172.21.15.99:0/3672681335 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f440056e0 con 0x55d1011e4f40 2023-08-25T09:03:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f6c316b80 10 client.25594 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3672681335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d100f841f0 con 0x55d1011e4f40 2023-08-25T09:03:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f39ffb700 1 -- 172.21.15.99:0/3672681335 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1011d2600 con 0x55d1011e4f40 2023-08-25T09:03:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f39ffb700 1 -- 172.21.15.99:0/3672681335 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6f44010730 con 0x55d1011e4f40 2023-08-25T09:03:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f6c316b80 10 client.25594 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f6c316b80 10 client.25594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f6c316b80 20 client.25594 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d10124ddc0 0x55d1012501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f6c316b80 4 client.25594 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:03:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3672681335 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d10112a910 con 0x55d10124ddc0 2023-08-25T09:03:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.002+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d10124ddc0 0x55d1012501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.003+0000 7f6f39ffb700 10 client.25594 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.003+0000 7f6f51d9b700 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d10124ddc0 0x55d1012501e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d721e03289892 server_cookie=ea595d68f8a57ddf in_seq=0 out_seq=0 2023-08-25T09:03:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f39ffb700 1 -- 172.21.15.99:0/3672681335 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d10112a910 con 0x55d10124ddc0 2023-08-25T09:03:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f39ffb700 10 client.25594 handle_command_reply: tid=0 2023-08-25T09:03:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 client.25594 shutdown 2023-08-25T09:03:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3672681335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 msgr2=0x55d1011e2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e2140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f44012070 tx=0x7f6f4400fb20 comp rx=0 tx=0).stop 2023-08-25T09:03:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3672681335 shutdown_connections 2023-08-25T09:03:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1011e5860 0x55d1011e08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d10124ddc0 0x55d1012501e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1011df990 0x55d1011e1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f39ffb700 0 client.25594 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 --2- 172.21.15.99:0/3672681335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1011e4f40 0x55d1011e2140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3672681335 >> 172.21.15.99:0/3672681335 conn(0x55d101154080 msgr2=0x55d1011d9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3672681335 shutdown_connections 2023-08-25T09:03:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.026+0000 7f6f6c316b80 1 -- 172.21.15.99:0/3672681335 wait complete. 2023-08-25T09:03:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f6c316b80 20 client.25594 trim_cache size 0 max 16384 2023-08-25T09:03:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:03:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:03:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:03:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:03:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 -- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f6f3c0a00c0 msgr2=0x7f6f3c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f6f3c0a00c0 0x7f6f3c0a2580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0c89d0 tx=0x7f6f4c009040 comp rx=0 tx=0).stop 2023-08-25T09:03:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 -- 172.21.15.99:0/2927224905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f541068b0 msgr2=0x7f6f541a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f541068b0 0x7f6f541a4b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00efc0 tx=0x7f6f4c00c3a0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 -- 172.21.15.99:0/2927224905 shutdown_connections 2023-08-25T09:03:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f6f3c0a00c0 0x7f6f3c0a2580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f541068b0 0x7f6f541a4b90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f541008a0 0x7f6f54072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 --2- 172.21.15.99:0/2927224905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f54101260 0x7f6f54073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 -- 172.21.15.99:0/2927224905 >> 172.21.15.99:0/2927224905 conn(0x7f6f540fc370 msgr2=0x7f6f541043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.027+0000 7f6f589ff700 1 -- 172.21.15.99:0/2927224905 shutdown_connections 2023-08-25T09:03:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:18.028+0000 7f6f589ff700 1 -- 172.21.15.99:0/2927224905 wait complete. 2023-08-25T09:03:18.905 INFO:tasks.workunit.client.0.smithi099.stdout: 1 98790 1.97 MB/sec execute 84 sec latency 21.370 ms 2023-08-25T09:03:19.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:19 smithi099 ceph-mon[97641]: pgmap v127: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 505 KiB/s wr, 48 op/s 2023-08-25T09:03:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:19 smithi157 ceph-mon[130879]: pgmap v127: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 505 KiB/s wr, 48 op/s 2023-08-25T09:03:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:19 smithi181 ceph-mon[131296]: pgmap v127: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 505 KiB/s wr, 48 op/s 2023-08-25T09:03:19.905 INFO:tasks.workunit.client.0.smithi099.stdout: 1 100028 2.02 MB/sec execute 85 sec latency 87.779 ms 2023-08-25T09:03:20.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 100028 2.00 MB/sec execute 86 sec latency 1087.936 ms 2023-08-25T09:03:21.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:21 smithi099 ceph-mon[97641]: pgmap v128: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 505 KiB/s wr, 48 op/s 2023-08-25T09:03:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:21 smithi157 ceph-mon[130879]: pgmap v128: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 505 KiB/s wr, 48 op/s 2023-08-25T09:03:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:21 smithi181 ceph-mon[131296]: pgmap v128: 97 pgs: 97 active+clean; 51 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 505 KiB/s wr, 48 op/s 2023-08-25T09:03:21.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 100028 1.98 MB/sec execute 87 sec latency 2088.080 ms 2023-08-25T09:03:22.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:22 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:03:22.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:22 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:03:22.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:22 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:03:22.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 100152 1.96 MB/sec execute 88 sec latency 2999.049 ms 2023-08-25T09:03:23.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:23 smithi099 ceph-mon[97641]: pgmap v129: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 787 KiB/s wr, 98 op/s 2023-08-25T09:03:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:23 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:03:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:23 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:03:23.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:23 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:03:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:23 smithi157 ceph-mon[130879]: pgmap v129: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 787 KiB/s wr, 98 op/s 2023-08-25T09:03:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:23 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:03:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:23 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:03:23.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:23 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:03:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:23 smithi181 ceph-mon[131296]: pgmap v129: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 787 KiB/s wr, 98 op/s 2023-08-25T09:03:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:23 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:03:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:23 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:03:23.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:23 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:03:23.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 102289 2.07 MB/sec execute 89 sec latency 18.647 ms 2023-08-25T09:03:24.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 103576 2.12 MB/sec execute 90 sec latency 92.534 ms 2023-08-25T09:03:25.379 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:25 smithi157 ceph-mon[130879]: pgmap v130: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 546 KiB/s wr, 62 op/s 2023-08-25T09:03:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:25 smithi181 ceph-mon[131296]: pgmap v130: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 546 KiB/s wr, 62 op/s 2023-08-25T09:03:25.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:25 smithi099 ceph-mon[97641]: pgmap v130: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 546 KiB/s wr, 62 op/s 2023-08-25T09:03:25.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 103632 2.10 MB/sec execute 91 sec latency 846.097 ms 2023-08-25T09:03:26.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 103632 2.08 MB/sec execute 92 sec latency 1846.246 ms 2023-08-25T09:03:27.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:27 smithi099 ceph-mon[97641]: pgmap v131: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-08-25T09:03:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:27 smithi157 ceph-mon[130879]: pgmap v131: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-08-25T09:03:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:27 smithi181 ceph-mon[131296]: pgmap v131: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-08-25T09:03:27.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 103736 2.06 MB/sec execute 93 sec latency 2755.710 ms 2023-08-25T09:03:28.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 105889 2.17 MB/sec execute 94 sec latency 15.515 ms 2023-08-25T09:03:29.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:29 smithi099 ceph-mon[97641]: pgmap v132: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 907 KiB/s wr, 98 op/s 2023-08-25T09:03:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:29 smithi157 ceph-mon[130879]: pgmap v132: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 907 KiB/s wr, 98 op/s 2023-08-25T09:03:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:29 smithi181 ceph-mon[131296]: pgmap v132: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 907 KiB/s wr, 98 op/s 2023-08-25T09:03:29.906 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107117 2.21 MB/sec execute 95 sec latency 113.941 ms 2023-08-25T09:03:30.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107117 2.19 MB/sec execute 96 sec latency 1114.049 ms 2023-08-25T09:03:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 -- 172.21.15.99:0/3393088073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 msgr2=0x7efdb4106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 --2- 172.21.15.99:0/3393088073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 0x7efdb4106c90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efda40098f0 tx=0x7efda4009c00 comp rx=0 tx=0).stop 2023-08-25T09:03:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 -- 172.21.15.99:0/3393088073 shutdown_connections 2023-08-25T09:03:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 --2- 172.21.15.99:0/3393088073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101260 0x7efdb410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 --2- 172.21.15.99:0/3393088073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 0x7efdb4106c90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 --2- 172.21.15.99:0/3393088073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb41008a0 0x7efdb4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 -- 172.21.15.99:0/3393088073 >> 172.21.15.99:0/3393088073 conn(0x7efdb40fc370 msgr2=0x7efdb40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 -- 172.21.15.99:0/3393088073 shutdown_connections 2023-08-25T09:03:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.183+0000 7efdb95ff700 1 -- 172.21.15.99:0/3393088073 wait complete. 2023-08-25T09:03:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.184+0000 7efdb95ff700 1 Processor -- start 2023-08-25T09:03:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.184+0000 7efdb95ff700 1 -- start start 2023-08-25T09:03:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb95ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb41008a0 0x7efdb419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb95ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101260 0x7efdb419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb95ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 0x7efdb41967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb95ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4196e10 con 0x7efdb41008a0 2023-08-25T09:03:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb95ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4196f80 con 0x7efdb41068b0 2023-08-25T09:03:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb95ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4197140 con 0x7efdb4101260 2023-08-25T09:03:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb41008a0 0x7efdb419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb41008a0 0x7efdb419c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48178/0 (socket says 172.21.15.99:48178) 2023-08-25T09:03:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb2ffd700 1 -- 172.21.15.99:0/2317642099 learned_addr learned my addr 172.21.15.99:0/2317642099 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb27fc700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101260 0x7efdb419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb37fe700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 0x7efdb41967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb2ffd700 1 -- 172.21.15.99:0/2317642099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101260 msgr2=0x7efdb419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101260 0x7efdb419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb2ffd700 1 -- 172.21.15.99:0/2317642099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 msgr2=0x7efdb41967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 0x7efdb41967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb2ffd700 1 -- 172.21.15.99:0/2317642099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0095f0 con 0x7efdb41008a0 2023-08-25T09:03:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.185+0000 7efdb37fe700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 0x7efdb41967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.186+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb41008a0 0x7efdb419c940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efda4009230 tx=0x7efda400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.186+0000 7efdabfff700 1 -- 172.21.15.99:0/2317642099 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda401c070 con 0x7efdb41008a0 2023-08-25T09:03:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.186+0000 7efdb95ff700 1 -- 172.21.15.99:0/2317642099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda4009520 con 0x7efdb41008a0 2023-08-25T09:03:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.186+0000 7efdabfff700 1 -- 172.21.15.99:0/2317642099 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efda40058e0 con 0x7efdb41008a0 2023-08-25T09:03:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.186+0000 7efdabfff700 1 -- 172.21.15.99:0/2317642099 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40038e0 con 0x7efdb41008a0 2023-08-25T09:03:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.186+0000 7efdb95ff700 1 -- 172.21.15.99:0/2317642099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb41a3750 con 0x7efdb41008a0 2023-08-25T09:03:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.189+0000 7efdabfff700 1 -- 172.21.15.99:0/2317642099 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7efda4020030 con 0x7efdb41008a0 2023-08-25T09:03:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.191+0000 7efdcce31b80 1 Processor -- start 2023-08-25T09:03:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.191+0000 7efdcce31b80 1 -- start start 2023-08-25T09:03:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a201320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a1fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdcce31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a163750 con 0x55b50a200f40 2023-08-25T09:03:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdcce31b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a154720 con 0x55b50a1fb990 2023-08-25T09:03:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdcce31b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a152040 con 0x55b50a201860 2023-08-25T09:03:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdabfff700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7efda00a00c0 0x7efda00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdb37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a201320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdb37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a201320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48186/0 (socket says 172.21.15.99:48186) 2023-08-25T09:03:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdb37fe700 1 -- 172.21.15.99:0/1317853101 learned_addr learned my addr 172.21.15.99:0/1317853101 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdabfff700 1 -- 172.21.15.99:0/2317642099 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efda40cc430 con 0x7efdb41008a0 2023-08-25T09:03:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdb27fc700 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a1fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.192+0000 7efdb37fe700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7efda00a00c0 0x7efda00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efdb2ffd700 1 -- 172.21.15.99:0/1317853101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 msgr2=0x55b50a1fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a1fbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efdb2ffd700 1 -- 172.21.15.99:0/1317853101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 msgr2=0x55b50a201320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a201320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efdb2ffd700 1 -- 172.21.15.99:0/1317853101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b509fcd5d0 con 0x55b50a201860 2023-08-25T09:03:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fb450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efda40c9b40 tx=0x7efda4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11673463d7a4a469 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efdb37fe700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7efda00a00c0 0x7efda00a2580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efdac00efd0 tx=0x7efdac00e530 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efda97fa700 1 -- 172.21.15.99:0/1317853101 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40b1ea0 con 0x55b50a201860 2023-08-25T09:03:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efda97fa700 1 -- 172.21.15.99:0/1317853101 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efda409a290 con 0x55b50a201860 2023-08-25T09:03:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.193+0000 7efda97fa700 1 -- 172.21.15.99:0/1317853101 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40a1910 con 0x55b50a201860 2023-08-25T09:03:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 -- 172.21.15.99:0/1317853101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 msgr2=0x55b50a1fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fb450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efda40c9b40 tx=0x7efda4009c00 comp rx=0 tx=0).stop 2023-08-25T09:03:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 -- 172.21.15.99:0/1317853101 shutdown_connections 2023-08-25T09:03:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a1fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a201320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1317853101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fb450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 -- 172.21.15.99:0/1317853101 >> 172.21.15.99:0/1317853101 conn(0x55b50a170080 msgr2=0x55b50a1f5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 -- 172.21.15.99:0/1317853101 shutdown_connections 2023-08-25T09:03:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 -- 172.21.15.99:0/1317853101 wait complete. 2023-08-25T09:03:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.194+0000 7efdcce31b80 1 -- start start 2023-08-25T09:03:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.195+0000 7efdcce31b80 1 Processor -- start 2023-08-25T09:03:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.195+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a1fb990 0x55b50a20ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.195+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.195+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a201860 0x55b50a208f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.195+0000 7efdcce31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a14d930 con 0x55b50a201860 2023-08-25T09:03:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.195+0000 7efdcce31b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a0ebb50 con 0x55b50a200f40 2023-08-25T09:03:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.195+0000 7efdcce31b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a0eb850 con 0x55b50a1fb990 2023-08-25T09:03:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.195+0000 7efdcce31b80 10 client.0 _ll_register_callbacks cb 0x55b50a035a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a1fb990 0x55b50a20ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a1fb990 0x55b50a20ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:44898/0 (socket says 172.21.15.99:44898) 2023-08-25T09:03:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb2ffd700 1 -- 172.21.15.99:0/3168865576 learned_addr learned my addr 172.21.15.99:0/3168865576 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb27fc700 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb37fe700 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a201860 0x55b50a208f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb27fc700 1 -- 172.21.15.99:0/3168865576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a1fb990 msgr2=0x55b50a20ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb27fc700 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a1fb990 0x55b50a20ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb27fc700 1 -- 172.21.15.99:0/3168865576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a201860 msgr2=0x55b50a208f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb27fc700 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a201860 0x55b50a208f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb27fc700 1 -- 172.21.15.99:0/3168865576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb41a3750 con 0x55b50a200f40 2023-08-25T09:03:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.196+0000 7efdb37fe700 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a201860 0x55b50a208f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.197+0000 7efdb27fc700 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20f230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005fd0 tx=0x7efd9c012ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0ba1e83536de593 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.197+0000 7efd9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:03:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.197+0000 7efd9a7fc700 1 -- 172.21.15.99:0/3168865576 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c017070 con 0x55b50a200f40 2023-08-25T09:03:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.197+0000 7efdcce31b80 1 -- 172.21.15.99:0/3168865576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b50a1cb160 con 0x55b50a200f40 2023-08-25T09:03:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.197+0000 7efd9a7fc700 1 -- 172.21.15.99:0/3168865576 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efd9c00f7a0 con 0x55b50a200f40 2023-08-25T09:03:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.197+0000 7efd9a7fc700 1 -- 172.21.15.99:0/3168865576 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd9c00f980 con 0x55b50a200f40 2023-08-25T09:03:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.197+0000 7efdcce31b80 10 client.25622 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.197+0000 7efdcce31b80 1 -- 172.21.15.99:0/3168865576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b509f79d80 con 0x55b50a200f40 2023-08-25T09:03:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efd9a7fc700 1 -- 172.21.15.99:0/3168865576 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b50a1cb160 con 0x55b50a200f40 2023-08-25T09:03:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efd9a7fc700 1 -- 172.21.15.99:0/3168865576 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efd9c010b60 con 0x55b50a200f40 2023-08-25T09:03:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efdcce31b80 10 client.25622 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efdcce31b80 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efdcce31b80 20 client.25622 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efdcce31b80 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50a219310 0x55b50a21b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efdcce31b80 4 client.25622 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:03:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efdcce31b80 1 -- 172.21.15.99:0/3168865576 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b50a0f3b40 con 0x55b50a219310 2023-08-25T09:03:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.198+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50a219310 0x55b50a21b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.199+0000 7efd9a7fc700 10 client.25622 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.199+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50a219310 0x55b50a21b730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88accf847027218a server_cookie=dc2c6f6961dea354 in_seq=0 out_seq=0 2023-08-25T09:03:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efd9a7fc700 1 -- 172.21.15.99:0/3168865576 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b50a0f3b40 con 0x55b50a219310 2023-08-25T09:03:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efd9a7fc700 10 client.25622 handle_command_reply: tid=0 2023-08-25T09:03:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 client.25622 shutdown 2023-08-25T09:03:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 -- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 msgr2=0x55b50a20f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20f230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005fd0 tx=0x7efd9c012ce0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 -- 172.21.15.99:0/3168865576 shutdown_connections 2023-08-25T09:03:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a201860 0x55b50a208f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a1fb990 0x55b50a20ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50a219310 0x55b50a21b730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 --2- 172.21.15.99:0/3168865576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20f230 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 -- 172.21.15.99:0/3168865576 >> 172.21.15.99:0/3168865576 conn(0x55b50a170080 msgr2=0x55b50a1f0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efd9a7fc700 0 client.25622 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 -- 172.21.15.99:0/3168865576 shutdown_connections 2023-08-25T09:03:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 1 -- 172.21.15.99:0/3168865576 wait complete. 2023-08-25T09:03:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.200+0000 7efdcce31b80 20 client.25622 trim_cache size 0 max 16384 2023-08-25T09:03:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.215+0000 7efdcce31b80 1 Processor -- start 2023-08-25T09:03:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.215+0000 7efdcce31b80 1 -- start start 2023-08-25T09:03:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a208f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a207060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdcce31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b509fcf8c0 con 0x55b50a200f40 2023-08-25T09:03:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdcce31b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a2094a0 con 0x55b50a1fb990 2023-08-25T09:03:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdcce31b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a216940 con 0x55b50a201860 2023-08-25T09:03:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a207060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a207060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48206/0 (socket says 172.21.15.99:48206) 2023-08-25T09:03:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb27fc700 1 -- 172.21.15.99:0/551886768 learned_addr learned my addr 172.21.15.99:0/551886768 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a208f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb37fe700 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb27fc700 1 -- 172.21.15.99:0/551886768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 msgr2=0x55b50a1fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb27fc700 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb27fc700 1 -- 172.21.15.99:0/551886768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 msgr2=0x55b50a208f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb27fc700 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a208f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb27fc700 1 -- 172.21.15.99:0/551886768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b509f62c00 con 0x55b50a200f40 2023-08-25T09:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.216+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a208f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.217+0000 7efdb27fc700 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a207060 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efd9c001eb0 tx=0x7efd9c0099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37541fe38aa35e50 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.217+0000 7efda9ffb700 1 -- 172.21.15.99:0/551886768 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c017070 con 0x55b50a200f40 2023-08-25T09:03:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.217+0000 7efda9ffb700 1 -- 172.21.15.99:0/551886768 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd9c018040 con 0x55b50a200f40 2023-08-25T09:03:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.217+0000 7efda9ffb700 1 -- 172.21.15.99:0/551886768 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c003d20 con 0x55b50a200f40 2023-08-25T09:03:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.217+0000 7efdcce31b80 1 -- 172.21.15.99:0/551886768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 msgr2=0x55b50a207060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.217+0000 7efdcce31b80 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a207060 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efd9c001eb0 tx=0x7efd9c0099f0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 -- 172.21.15.99:0/551886768 shutdown_connections 2023-08-25T09:03:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a1fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a1fb990 0x55b50a208f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 --2- 172.21.15.99:0/551886768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a200f40 0x55b50a207060 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 -- 172.21.15.99:0/551886768 >> 172.21.15.99:0/551886768 conn(0x55b50a170080 msgr2=0x55b50a1f5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 -- 172.21.15.99:0/551886768 shutdown_connections 2023-08-25T09:03:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 -- 172.21.15.99:0/551886768 wait complete. 2023-08-25T09:03:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 -- start start 2023-08-25T09:03:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.218+0000 7efdcce31b80 1 Processor -- start 2023-08-25T09:03:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a1fb990 0x55b50a20c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdcce31b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a20d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdcce31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a26c270 con 0x55b50a1fb990 2023-08-25T09:03:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdcce31b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a26c3e0 con 0x55b50a200f40 2023-08-25T09:03:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdcce31b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50a26c5a0 con 0x55b50a201860 2023-08-25T09:03:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdcce31b80 10 client.0 _ll_register_callbacks cb 0x7efd9c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdb2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a1fb990 0x55b50a20c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdb2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a1fb990 0x55b50a20c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48208/0 (socket says 172.21.15.99:48208) 2023-08-25T09:03:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.219+0000 7efdb2ffd700 1 -- 172.21.15.99:0/1226945557 learned_addr learned my addr 172.21.15.99:0/1226945557 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb27fc700 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb37fe700 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a20d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb2ffd700 1 -- 172.21.15.99:0/1226945557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 msgr2=0x55b50a20d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a20d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb2ffd700 1 -- 172.21.15.99:0/1226945557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 msgr2=0x55b50a20cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb2ffd700 1 -- 172.21.15.99:0/1226945557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b509fcd5d0 con 0x55b50a1fb990 2023-08-25T09:03:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb37fe700 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a20d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:03:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdb2ffd700 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a1fb990 0x55b50a20c830 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efda4097ee0 tx=0x7efda409f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a440e8ccb20737b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efd9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:03:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efd9a7fc700 1 -- 172.21.15.99:0/1226945557 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40b1ea0 con 0x55b50a1fb990 2023-08-25T09:03:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efd9a7fc700 1 -- 172.21.15.99:0/1226945557 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efda40a1910 con 0x55b50a1fb990 2023-08-25T09:03:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdcce31b80 1 -- 172.21.15.99:0/1226945557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b50a1ee600 con 0x55b50a1fb990 2023-08-25T09:03:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efd9a7fc700 1 -- 172.21.15.99:0/1226945557 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efda4021400 con 0x55b50a1fb990 2023-08-25T09:03:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdcce31b80 10 client.15573 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.220+0000 7efdcce31b80 1 -- 172.21.15.99:0/1226945557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b509f79d80 con 0x55b50a1fb990 2023-08-25T09:03:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efd9a7fc700 1 -- 172.21.15.99:0/1226945557 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efda407c020 con 0x55b50a1fb990 2023-08-25T09:03:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efd9a7fc700 1 -- 172.21.15.99:0/1226945557 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efda40c9e50 con 0x55b50a1fb990 2023-08-25T09:03:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efdcce31b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efdcce31b80 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efdcce31b80 20 client.15573 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50a26d940 0x55b50a26fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efdcce31b80 4 client.15573 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:03:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efdcce31b80 1 -- 172.21.15.99:0/1226945557 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b50a146910 con 0x55b50a26d940 2023-08-25T09:03:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.221+0000 7efdb27fc700 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50a26d940 0x55b50a26fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.222+0000 7efdb27fc700 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50a26d940 0x55b50a26fd60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a659dcfc12118cab server_cookie=3ae377156bb12c6b in_seq=0 out_seq=0 2023-08-25T09:03:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.222+0000 7efd9a7fc700 10 client.15573 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efd9a7fc700 1 -- 172.21.15.99:0/1226945557 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b50a146910 con 0x55b50a26d940 2023-08-25T09:03:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efd9a7fc700 10 client.15573 handle_command_reply: tid=0 2023-08-25T09:03:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 client.15573 shutdown 2023-08-25T09:03:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 -- 172.21.15.99:0/1226945557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a1fb990 msgr2=0x55b50a20c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a1fb990 0x55b50a20c830 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efda4097ee0 tx=0x7efda409f040 comp rx=0 tx=0).stop 2023-08-25T09:03:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 -- 172.21.15.99:0/1226945557 shutdown_connections 2023-08-25T09:03:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50a201860 0x55b50a20d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50a1fb990 0x55b50a20c830 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50a26d940 0x55b50a26fd60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 --2- 172.21.15.99:0/1226945557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50a200f40 0x55b50a20cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 -- 172.21.15.99:0/1226945557 >> 172.21.15.99:0/1226945557 conn(0x55b50a170080 msgr2=0x55b50a1eff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efd9a7fc700 0 client.15573 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 -- 172.21.15.99:0/1226945557 shutdown_connections 2023-08-25T09:03:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.272+0000 7efdcce31b80 1 -- 172.21.15.99:0/1226945557 wait complete. 2023-08-25T09:03:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdcce31b80 20 client.15573 trim_cache size 0 max 16384 2023-08-25T09:03:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:03:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:03:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:03:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:03:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 -- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7efda00a00c0 msgr2=0x7efda00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7efda00a00c0 0x7efda00a2580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efdac00efd0 tx=0x7efdac00e530 comp rx=0 tx=0).stop 2023-08-25T09:03:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 -- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb41008a0 msgr2=0x7efdb419c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb41008a0 0x7efdb419c940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efda4009230 tx=0x7efda400fa00 comp rx=0 tx=0).stop 2023-08-25T09:03:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 -- 172.21.15.99:0/2317642099 shutdown_connections 2023-08-25T09:03:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7efda00a00c0 0x7efda00a2580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdb41068b0 0x7efdb41967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb41008a0 0x7efdb419c940 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 --2- 172.21.15.99:0/2317642099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdb4101260 0x7efdb419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 -- 172.21.15.99:0/2317642099 >> 172.21.15.99:0/2317642099 conn(0x7efdb40fc370 msgr2=0x7efdb40fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 -- 172.21.15.99:0/2317642099 shutdown_connections 2023-08-25T09:03:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:31.273+0000 7efdb95ff700 1 -- 172.21.15.99:0/2317642099 wait complete. 2023-08-25T09:03:31.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:31 smithi099 ceph-mon[97641]: pgmap v133: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 907 KiB/s wr, 98 op/s 2023-08-25T09:03:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:31 smithi157 ceph-mon[130879]: pgmap v133: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 907 KiB/s wr, 98 op/s 2023-08-25T09:03:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:31 smithi181 ceph-mon[131296]: pgmap v133: 97 pgs: 97 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 907 KiB/s wr, 98 op/s 2023-08-25T09:03:31.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107117 2.17 MB/sec execute 97 sec latency 2114.103 ms 2023-08-25T09:03:32.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107173 2.15 MB/sec execute 98 sec latency 3028.204 ms 2023-08-25T09:03:33.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:33 smithi099 ceph-mon[97641]: pgmap v134: 97 pgs: 97 active+clean; 52 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-08-25T09:03:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:33 smithi157 ceph-mon[130879]: pgmap v134: 97 pgs: 97 active+clean; 52 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-08-25T09:03:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:33 smithi181 ceph-mon[131296]: pgmap v134: 97 pgs: 97 active+clean; 52 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-08-25T09:03:33.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107173 2.13 MB/sec execute 99 sec latency 1042.345 ms 2023-08-25T09:03:34.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107173 2.10 MB/sec execute 100 sec latency 2042.491 ms 2023-08-25T09:03:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:35 smithi181 ceph-mon[131296]: pgmap v135: 97 pgs: 97 active+clean; 52 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 869 KiB/s wr, 83 op/s 2023-08-25T09:03:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:35 smithi099 ceph-mon[97641]: pgmap v135: 97 pgs: 97 active+clean; 52 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 869 KiB/s wr, 83 op/s 2023-08-25T09:03:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:35 smithi157 ceph-mon[130879]: pgmap v135: 97 pgs: 97 active+clean; 52 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 869 KiB/s wr, 83 op/s 2023-08-25T09:03:35.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107173 2.08 MB/sec execute 101 sec latency 3042.597 ms 2023-08-25T09:03:36.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107173 2.06 MB/sec execute 102 sec latency 4042.645 ms 2023-08-25T09:03:37.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:37 smithi099 ceph-mon[97641]: pgmap v136: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:03:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:37 smithi157 ceph-mon[130879]: pgmap v136: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:03:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:37 smithi181 ceph-mon[131296]: pgmap v136: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:03:37.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107324 2.05 MB/sec execute 103 sec latency 4951.194 ms 2023-08-25T09:03:38.907 INFO:tasks.workunit.client.0.smithi099.stdout: 1 109440 2.14 MB/sec execute 104 sec latency 19.829 ms 2023-08-25T09:03:39.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:39 smithi099 ceph-mon[97641]: pgmap v137: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 533 KiB/s wr, 65 op/s 2023-08-25T09:03:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:39 smithi157 ceph-mon[130879]: pgmap v137: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 533 KiB/s wr, 65 op/s 2023-08-25T09:03:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:39 smithi181 ceph-mon[131296]: pgmap v137: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 533 KiB/s wr, 65 op/s 2023-08-25T09:03:39.908 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110667 2.18 MB/sec execute 105 sec latency 121.457 ms 2023-08-25T09:03:40.908 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110667 2.16 MB/sec execute 106 sec latency 1121.645 ms 2023-08-25T09:03:41.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:41 smithi099 ceph-mon[97641]: pgmap v138: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 533 KiB/s wr, 65 op/s 2023-08-25T09:03:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:41 smithi157 ceph-mon[130879]: pgmap v138: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 533 KiB/s wr, 65 op/s 2023-08-25T09:03:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:41 smithi181 ceph-mon[131296]: pgmap v138: 97 pgs: 97 active+clean; 51 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 533 KiB/s wr, 65 op/s 2023-08-25T09:03:41.908 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110667 2.14 MB/sec execute 107 sec latency 2121.786 ms 2023-08-25T09:03:42.908 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110723 2.13 MB/sec execute 108 sec latency 3034.603 ms 2023-08-25T09:03:43.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:43 smithi099 ceph-mon[97641]: pgmap v139: 97 pgs: 97 active+clean; 53 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1017 KiB/s wr, 108 op/s 2023-08-25T09:03:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:43 smithi157 ceph-mon[130879]: pgmap v139: 97 pgs: 97 active+clean; 53 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1017 KiB/s wr, 108 op/s 2023-08-25T09:03:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:43 smithi181 ceph-mon[131296]: pgmap v139: 97 pgs: 97 active+clean; 53 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1017 KiB/s wr, 108 op/s 2023-08-25T09:03:43.908 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110723 2.11 MB/sec execute 109 sec latency 1046.431 ms 2023-08-25T09:03:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/4149700491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 msgr2=0x7f4b78106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/4149700491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 0x7f4b78106c90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009990 tx=0x7f4b68009ca0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/4149700491 shutdown_connections 2023-08-25T09:03:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/4149700491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78101260 0x7f4b7810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/4149700491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 0x7f4b78106c90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/4149700491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b781008a0 0x7f4b78100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/4149700491 >> 172.21.15.99:0/4149700491 conn(0x7f4b780fc370 msgr2=0x7f4b780fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/4149700491 shutdown_connections 2023-08-25T09:03:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.428+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/4149700491 wait complete. 2023-08-25T09:03:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.429+0000 7f4b7e5ff700 1 Processor -- start 2023-08-25T09:03:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.429+0000 7f4b7e5ff700 1 -- start start 2023-08-25T09:03:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7e5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b781008a0 0x7f4b7819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7e5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78101260 0x7f4b7819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7e5ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 0x7f4b781989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7e5ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78199180 con 0x7f4b781068b0 2023-08-25T09:03:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7e5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b781992f0 con 0x7f4b78101260 2023-08-25T09:03:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7e5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b781994b0 con 0x7f4b781008a0 2023-08-25T09:03:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78101260 0x7f4b7819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7cb9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 0x7f4b781989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78101260 0x7f4b7819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48416/0 (socket says 172.21.15.99:48416) 2023-08-25T09:03:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7cb9c700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 0x7f4b781989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39916/0 (socket says 172.21.15.99:39916) 2023-08-25T09:03:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b777fe700 1 -- 172.21.15.99:0/961382725 learned_addr learned my addr 172.21.15.99:0/961382725 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b77fff700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b781008a0 0x7f4b7819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7cb9c700 1 -- 172.21.15.99:0/961382725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b781008a0 msgr2=0x7f4b7819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7cb9c700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b781008a0 0x7f4b7819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7cb9c700 1 -- 172.21.15.99:0/961382725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78101260 msgr2=0x7f4b7819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7cb9c700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78101260 0x7f4b7819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b7cb9c700 1 -- 172.21.15.99:0/961382725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b680095f0 con 0x7f4b781068b0 2023-08-25T09:03:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.430+0000 7f4b777fe700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78101260 0x7f4b7819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.431+0000 7f4b7cb9c700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 0x7f4b781989b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000efc0 tx=0x7f4b7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.431+0000 7f4b757fa700 1 -- 172.21.15.99:0/961382725 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70018070 con 0x7f4b781068b0 2023-08-25T09:03:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.431+0000 7f4b757fa700 1 -- 172.21.15.99:0/961382725 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b7000f040 con 0x7f4b781068b0 2023-08-25T09:03:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.431+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/961382725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b78199730 con 0x7f4b781068b0 2023-08-25T09:03:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.431+0000 7f4b757fa700 1 -- 172.21.15.99:0/961382725 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000cd40 con 0x7f4b781068b0 2023-08-25T09:03:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.431+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/961382725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b781a5b60 con 0x7f4b781068b0 2023-08-25T09:03:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.435+0000 7f4b757fa700 1 -- 172.21.15.99:0/961382725 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f4b70003a00 con 0x7f4b781068b0 2023-08-25T09:03:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.436+0000 7f4b91ed0b80 1 Processor -- start 2023-08-25T09:03:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b91ed0b80 1 -- start start 2023-08-25T09:03:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a968f40 0x55a01a969320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a963450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 0x55a01a963dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a8cb750 con 0x55a01a968f40 2023-08-25T09:03:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a8bc720 con 0x55a01a963990 2023-08-25T09:03:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a8ba040 con 0x55a01a969860 2023-08-25T09:03:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b757fa700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4b600a00c0 0x7f4b600a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b757fa700 1 -- 172.21.15.99:0/961382725 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4b700cb220 con 0x7f4b781068b0 2023-08-25T09:03:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a968f40 0x55a01a969320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a968f40 0x55a01a969320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39920/0 (socket says 172.21.15.99:39920) 2023-08-25T09:03:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b777fe700 1 -- 172.21.15.99:0/2844930882 learned_addr learned my addr 172.21.15.99:0/2844930882 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b77fff700 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a963450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.437+0000 7f4b7cb9c700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4b600a00c0 0x7f4b600a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b777fe700 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 0x55a01a963dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b777fe700 1 -- 172.21.15.99:0/2844930882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 msgr2=0x55a01a963450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b777fe700 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a963450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b777fe700 1 -- 172.21.15.99:0/2844930882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 msgr2=0x55a01a963dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b777fe700 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 0x55a01a963dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b777fe700 1 -- 172.21.15.99:0/2844930882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01a7355d0 con 0x55a01a968f40 2023-08-25T09:03:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b77fff700 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a963450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b7cb9c700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4b600a00c0 0x7f4b600a2580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b700c89e0 tx=0x7f4b70009040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b777fe700 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a968f40 0x55a01a969320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c013b30 tx=0x7f4b6c009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784c7c11e99ad6d8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b5e7fc700 1 -- 172.21.15.99:0/2844930882 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c004030 con 0x55a01a968f40 2023-08-25T09:03:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b5e7fc700 1 -- 172.21.15.99:0/2844930882 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b6c00f7d0 con 0x55a01a968f40 2023-08-25T09:03:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.438+0000 7f4b5e7fc700 1 -- 172.21.15.99:0/2844930882 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c003880 con 0x55a01a968f40 2023-08-25T09:03:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2844930882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a968f40 msgr2=0x55a01a969320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a968f40 0x55a01a969320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c013b30 tx=0x7f4b6c009d50 comp rx=0 tx=0).stop 2023-08-25T09:03:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2844930882 shutdown_connections 2023-08-25T09:03:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 0x55a01a963dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a968f40 0x55a01a969320 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2844930882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a963450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2844930882 >> 172.21.15.99:0/2844930882 conn(0x55a01a8d8080 msgr2=0x55a01a95dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2844930882 shutdown_connections 2023-08-25T09:03:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.439+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2844930882 wait complete. 2023-08-25T09:03:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.440+0000 7f4b91ed0b80 1 -- start start 2023-08-25T09:03:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.440+0000 7f4b91ed0b80 1 Processor -- start 2023-08-25T09:03:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.440+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 0x55a01a96f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a968f40 0x55a01a96c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a969860 0x55a01a96d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a8b5930 con 0x55a01a969860 2023-08-25T09:03:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a853b50 con 0x55a01a963990 2023-08-25T09:03:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a853850 con 0x55a01a968f40 2023-08-25T09:03:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b91ed0b80 10 client.0 _ll_register_callbacks cb 0x55a01a79da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a969860 0x55a01a96d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a969860 0x55a01a96d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39936/0 (socket says 172.21.15.99:39936) 2023-08-25T09:03:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b777fe700 1 -- 172.21.15.99:0/2491417835 learned_addr learned my addr 172.21.15.99:0/2491417835 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b77fff700 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a968f40 0x55a01a96c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b777fe700 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 0x55a01a96f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b77fff700 1 -- 172.21.15.99:0/2491417835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 msgr2=0x55a01a96f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b77fff700 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 0x55a01a96f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b77fff700 1 -- 172.21.15.99:0/2491417835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a969860 msgr2=0x55a01a96d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b77fff700 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a969860 0x55a01a96d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b77fff700 1 -- 172.21.15.99:0/2491417835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7810e070 con 0x55a01a968f40 2023-08-25T09:03:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.441+0000 7f4b777fe700 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a969860 0x55a01a96d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b77fff700 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a968f40 0x55a01a96c990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4b68000c00 tx=0x7f4b68005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0b85640c986fed server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b577fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:03:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b577fe700 1 -- 172.21.15.99:0/2491417835 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6801d070 con 0x55a01a968f40 2023-08-25T09:03:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2491417835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a01a933160 con 0x55a01a968f40 2023-08-25T09:03:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b577fe700 1 -- 172.21.15.99:0/2491417835 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4b68004500 con 0x55a01a968f40 2023-08-25T09:03:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b577fe700 1 -- 172.21.15.99:0/2491417835 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b680046c0 con 0x55a01a968f40 2023-08-25T09:03:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b91ed0b80 10 client.25624 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2491417835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f4b6c0136e0 con 0x55a01a968f40 2023-08-25T09:03:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.442+0000 7f4b577fe700 1 -- 172.21.15.99:0/2491417835 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a01a933160 con 0x55a01a968f40 2023-08-25T09:03:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.443+0000 7f4b577fe700 1 -- 172.21.15.99:0/2491417835 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b6800f790 con 0x55a01a968f40 2023-08-25T09:03:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.443+0000 7f4b91ed0b80 10 client.25624 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.443+0000 7f4b91ed0b80 10 client.25624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.443+0000 7f4b91ed0b80 20 client.25624 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.443+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55a01a97d240 0x55a01a97f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.443+0000 7f4b91ed0b80 4 client.25624 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:03:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.443+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2491417835 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a01a85bb40 con 0x55a01a97d240 2023-08-25T09:03:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.443+0000 7f4b777fe700 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55a01a97d240 0x55a01a97f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.444+0000 7f4b577fe700 10 client.25624 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.444+0000 7f4b777fe700 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55a01a97d240 0x55a01a97f660 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7b5c3891daf644 server_cookie=3930ae29f132941a in_seq=0 out_seq=0 2023-08-25T09:03:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.444+0000 7f4b577fe700 1 -- 172.21.15.99:0/2491417835 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a01a85bb40 con 0x55a01a97d240 2023-08-25T09:03:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.444+0000 7f4b577fe700 10 client.25624 handle_command_reply: tid=0 2023-08-25T09:03:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 client.25624 shutdown 2023-08-25T09:03:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2491417835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a968f40 msgr2=0x55a01a96c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a968f40 0x55a01a96c990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4b68000c00 tx=0x7f4b68005e80 comp rx=0 tx=0).stop 2023-08-25T09:03:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2491417835 shutdown_connections 2023-08-25T09:03:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a969860 0x55a01a96d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55a01a97d240 0x55a01a97f660 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a963990 0x55a01a96f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b577fe700 0 client.25624 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/2491417835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a968f40 0x55a01a96c990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2491417835 >> 172.21.15.99:0/2491417835 conn(0x55a01a8d8080 msgr2=0x55a01a958820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2491417835 shutdown_connections 2023-08-25T09:03:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/2491417835 wait complete. 2023-08-25T09:03:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.445+0000 7f4b91ed0b80 20 client.25624 trim_cache size 0 max 16384 2023-08-25T09:03:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.460+0000 7f4b91ed0b80 1 Processor -- start 2023-08-25T09:03:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b91ed0b80 1 -- start start 2023-08-25T09:03:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a963450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a974270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a966af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a7378c0 con 0x55a01a963990 2023-08-25T09:03:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a970030 con 0x55a01a968f40 2023-08-25T09:03:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a95a700 con 0x55a01a969860 2023-08-25T09:03:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a974270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a963450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a974270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48444/0 (socket says 172.21.15.99:48444) 2023-08-25T09:03:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a963450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39950/0 (socket says 172.21.15.99:39950) 2023-08-25T09:03:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b77fff700 1 -- 172.21.15.99:0/3223682178 learned_addr learned my addr 172.21.15.99:0/3223682178 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b777fe700 1 --2- 172.21.15.99:0/3223682178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a966af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b77fff700 1 -- 172.21.15.99:0/3223682178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 msgr2=0x55a01a966af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b77fff700 1 --2- 172.21.15.99:0/3223682178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a966af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b77fff700 1 -- 172.21.15.99:0/3223682178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 msgr2=0x55a01a963450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.461+0000 7f4b77fff700 1 --2- 172.21.15.99:0/3223682178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a963450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.462+0000 7f4b77fff700 1 -- 172.21.15.99:0/3223682178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01a964040 con 0x55a01a968f40 2023-08-25T09:03:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.462+0000 7f4b77fff700 1 --2- 172.21.15.99:0/3223682178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a974270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009990 tx=0x7f4b6800b340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5638b40b70874cdd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.462+0000 7f4b5effd700 1 -- 172.21.15.99:0/3223682178 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6801d030 con 0x55a01a968f40 2023-08-25T09:03:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.462+0000 7f4b5effd700 1 -- 172.21.15.99:0/3223682178 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b6801e040 con 0x55a01a968f40 2023-08-25T09:03:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.462+0000 7f4b5effd700 1 -- 172.21.15.99:0/3223682178 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68004a30 con 0x55a01a968f40 2023-08-25T09:03:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3223682178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 msgr2=0x55a01a974270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3223682178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a974270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009990 tx=0x7f4b6800b340 comp rx=0 tx=0).stop 2023-08-25T09:03:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3223682178 shutdown_connections 2023-08-25T09:03:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3223682178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a966af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3223682178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a963450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3223682178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a974270 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3223682178 >> 172.21.15.99:0/3223682178 conn(0x55a01a8d8080 msgr2=0x55a01a9789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3223682178 shutdown_connections 2023-08-25T09:03:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3223682178 wait complete. 2023-08-25T09:03:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 -- start start 2023-08-25T09:03:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.463+0000 7f4b91ed0b80 1 Processor -- start 2023-08-25T09:03:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a974470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a964950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b91ed0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a964f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a965620 con 0x55a01a963990 2023-08-25T09:03:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a965790 con 0x55a01a968f40 2023-08-25T09:03:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b91ed0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01a965950 con 0x55a01a969860 2023-08-25T09:03:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b91ed0b80 10 client.0 _ll_register_callbacks cb 0x7f4b6800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a974470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a964950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a964950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:48448/0 (socket says 172.21.15.99:48448) 2023-08-25T09:03:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a974470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39954/0 (socket says 172.21.15.99:39954) 2023-08-25T09:03:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b77fff700 1 -- 172.21.15.99:0/3687731684 learned_addr learned my addr 172.21.15.99:0/3687731684 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b777fe700 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a964f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b77fff700 1 -- 172.21.15.99:0/3687731684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 msgr2=0x55a01a964f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b77fff700 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a964f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b77fff700 1 -- 172.21.15.99:0/3687731684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 msgr2=0x55a01a974470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b77fff700 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a974470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b77fff700 1 -- 172.21.15.99:0/3687731684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c0136e0 con 0x55a01a968f40 2023-08-25T09:03:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.464+0000 7f4b777fe700 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a974470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b77fff700 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a964950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c010 tx=0x7f4b6800f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad22ef56ec9e2b0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b577fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:03:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b577fe700 1 -- 172.21.15.99:0/3687731684 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6801d070 con 0x55a01a968f40 2023-08-25T09:03:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3687731684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a01a956600 con 0x55a01a968f40 2023-08-25T09:03:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b577fe700 1 -- 172.21.15.99:0/3687731684 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4b68000e40 con 0x55a01a968f40 2023-08-25T09:03:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b577fe700 1 -- 172.21.15.99:0/3687731684 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b68004610 con 0x55a01a968f40 2023-08-25T09:03:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b91ed0b80 10 client.25646 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3687731684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a01a6e1d80 con 0x55a01a968f40 2023-08-25T09:03:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b577fe700 1 -- 172.21.15.99:0/3687731684 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a01a956600 con 0x55a01a968f40 2023-08-25T09:03:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.465+0000 7f4b577fe700 1 -- 172.21.15.99:0/3687731684 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b6803e030 con 0x55a01a968f40 2023-08-25T09:03:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b91ed0b80 10 client.25646 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b91ed0b80 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b91ed0b80 20 client.25646 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55a01a9d9da0 0x55a01a9dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b91ed0b80 4 client.25646 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:03:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3687731684 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a01a8ae910 con 0x55a01a9d9da0 2023-08-25T09:03:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b777fe700 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55a01a9d9da0 0x55a01a9dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b577fe700 10 client.25646 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.466+0000 7f4b777fe700 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55a01a9d9da0 0x55a01a9dc1c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6dbb978ab8861e server_cookie=fb3c19df60a0acf2 in_seq=0 out_seq=0 2023-08-25T09:03:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b577fe700 1 -- 172.21.15.99:0/3687731684 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a01a8ae910 con 0x55a01a9d9da0 2023-08-25T09:03:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b577fe700 10 client.25646 handle_command_reply: tid=0 2023-08-25T09:03:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 client.25646 shutdown 2023-08-25T09:03:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3687731684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 msgr2=0x55a01a964950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a964950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c010 tx=0x7f4b6800f870 comp rx=0 tx=0).stop 2023-08-25T09:03:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3687731684 shutdown_connections 2023-08-25T09:03:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a01a969860 0x55a01a964f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a01a963990 0x55a01a974470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55a01a9d9da0 0x55a01a9dc1c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 --2- 172.21.15.99:0/3687731684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a01a968f40 0x55a01a964950 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b577fe700 0 client.25646 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3687731684 >> 172.21.15.99:0/3687731684 conn(0x55a01a8d8080 msgr2=0x55a01a966af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3687731684 shutdown_connections 2023-08-25T09:03:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.495+0000 7f4b91ed0b80 1 -- 172.21.15.99:0/3687731684 wait complete. 2023-08-25T09:03:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b91ed0b80 20 client.25646 trim_cache size 0 max 16384 2023-08-25T09:03:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:03:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:03:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:03:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:03:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4b600a00c0 msgr2=0x7f4b600a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4b600a00c0 0x7f4b600a2580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b700c89e0 tx=0x7f4b70009040 comp rx=0 tx=0).stop 2023-08-25T09:03:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 msgr2=0x7f4b781989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 0x7f4b781989b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000efc0 tx=0x7f4b7000c3a0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/961382725 shutdown_connections 2023-08-25T09:03:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4b600a00c0 0x7f4b600a2580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b781068b0 0x7f4b781989b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b781008a0 0x7f4b7819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 --2- 172.21.15.99:0/961382725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78101260 0x7f4b7819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/961382725 >> 172.21.15.99:0/961382725 conn(0x7f4b780fc370 msgr2=0x7f4b7810bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/961382725 shutdown_connections 2023-08-25T09:03:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:44.496+0000 7f4b7e5ff700 1 -- 172.21.15.99:0/961382725 wait complete. 2023-08-25T09:03:44.908 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110723 2.09 MB/sec execute 110 sec latency 2046.599 ms 2023-08-25T09:03:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:45 smithi181 ceph-mon[131296]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 773 KiB/s wr, 72 op/s 2023-08-25T09:03:45.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:45 smithi099 ceph-mon[97641]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 773 KiB/s wr, 72 op/s 2023-08-25T09:03:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:45 smithi157 ceph-mon[130879]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 773 KiB/s wr, 72 op/s 2023-08-25T09:03:45.908 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110723 2.07 MB/sec execute 111 sec latency 3046.746 ms 2023-08-25T09:03:46.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110723 2.05 MB/sec execute 112 sec latency 4046.921 ms 2023-08-25T09:03:47.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:47 smithi099 ceph-mon[97641]: pgmap v141: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 837 KiB/s wr, 94 op/s 2023-08-25T09:03:47.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:47 smithi157 ceph-mon[130879]: pgmap v141: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 837 KiB/s wr, 94 op/s 2023-08-25T09:03:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:47 smithi181 ceph-mon[131296]: pgmap v141: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 837 KiB/s wr, 94 op/s 2023-08-25T09:03:47.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110868 2.03 MB/sec execute 113 sec latency 4956.552 ms 2023-08-25T09:03:48.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 112978 2.12 MB/sec execute 114 sec latency 14.711 ms 2023-08-25T09:03:49.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:49 smithi099 ceph-mon[97641]: pgmap v142: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 548 KiB/s wr, 65 op/s 2023-08-25T09:03:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:49 smithi157 ceph-mon[130879]: pgmap v142: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 548 KiB/s wr, 65 op/s 2023-08-25T09:03:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:49 smithi181 ceph-mon[131296]: pgmap v142: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 548 KiB/s wr, 65 op/s 2023-08-25T09:03:49.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114202 2.16 MB/sec execute 115 sec latency 80.082 ms 2023-08-25T09:03:50.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114266 2.14 MB/sec execute 116 sec latency 950.629 ms 2023-08-25T09:03:51.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:51 smithi099 ceph-mon[97641]: pgmap v143: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 548 KiB/s wr, 65 op/s 2023-08-25T09:03:51.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:51 smithi157 ceph-mon[130879]: pgmap v143: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 548 KiB/s wr, 65 op/s 2023-08-25T09:03:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:51 smithi181 ceph-mon[131296]: pgmap v143: 97 pgs: 97 active+clean; 50 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 548 KiB/s wr, 65 op/s 2023-08-25T09:03:51.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114266 2.13 MB/sec execute 117 sec latency 1950.772 ms 2023-08-25T09:03:52.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114334 2.11 MB/sec execute 118 sec latency 2859.397 ms 2023-08-25T09:03:53.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:53 smithi099 ceph-mon[97641]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-08-25T09:03:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:53 smithi157 ceph-mon[130879]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-08-25T09:03:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:53 smithi181 ceph-mon[131296]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-08-25T09:03:53.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114334 2.09 MB/sec execute 119 sec latency 1045.647 ms 2023-08-25T09:03:54.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114334 2.07 MB/sec execute 120 sec latency 2045.716 ms 2023-08-25T09:03:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:55 smithi181 ceph-mon[131296]: pgmap v145: 97 pgs: 97 active+clean; 55 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 547 KiB/s wr, 61 op/s 2023-08-25T09:03:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:55 smithi099 ceph-mon[97641]: pgmap v145: 97 pgs: 97 active+clean; 55 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 547 KiB/s wr, 61 op/s 2023-08-25T09:03:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:55 smithi157 ceph-mon[130879]: pgmap v145: 97 pgs: 97 active+clean; 55 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 547 KiB/s wr, 61 op/s 2023-08-25T09:03:55.909 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114334 2.06 MB/sec execute 121 sec latency 3045.828 ms 2023-08-25T09:03:56.910 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114334 2.04 MB/sec execute 122 sec latency 4046.016 ms 2023-08-25T09:03:57.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:57 smithi099 ceph-mon[97641]: pgmap v146: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 830 KiB/s wr, 78 op/s 2023-08-25T09:03:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:57 smithi157 ceph-mon[130879]: pgmap v146: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 830 KiB/s wr, 78 op/s 2023-08-25T09:03:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 -- 172.21.15.99:0/2032221220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73641008b0 msgr2=0x7f7364100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 --2- 172.21.15.99:0/2032221220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73641008b0 0x7f7364100d30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7358009990 tx=0x7f7358009ca0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 -- 172.21.15.99:0/2032221220 shutdown_connections 2023-08-25T09:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 --2- 172.21.15.99:0/2032221220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7364101270 0x7f736410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 --2- 172.21.15.99:0/2032221220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73641068c0 0x7f7364106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 --2- 172.21.15.99:0/2032221220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73641008b0 0x7f7364100d30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 -- 172.21.15.99:0/2032221220 >> 172.21.15.99:0/2032221220 conn(0x7f73640fc380 msgr2=0x7f73640fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 -- 172.21.15.99:0/2032221220 shutdown_connections 2023-08-25T09:03:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.651+0000 7f7368dff700 1 -- 172.21.15.99:0/2032221220 wait complete. 2023-08-25T09:03:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.653+0000 7f7368dff700 1 Processor -- start 2023-08-25T09:03:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.653+0000 7f7368dff700 1 -- start start 2023-08-25T09:03:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7368dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73641008b0 0x7f736419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7368dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7364101270 0x7f736419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7368dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73641068c0 0x7f7364196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7368dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7364196e00 con 0x7f7364101270 2023-08-25T09:03:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7368dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7364196f70 con 0x7f73641008b0 2023-08-25T09:03:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7368dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7364197130 con 0x7f73641068c0 2023-08-25T09:03:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7361d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7364101270 0x7f736419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f736259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73641008b0 0x7f736419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7361d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7364101270 0x7f736419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58258/0 (socket says 172.21.15.99:58258) 2023-08-25T09:03:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7361d9b700 1 -- 172.21.15.99:0/806602746 learned_addr learned my addr 172.21.15.99:0/806602746 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7362d9d700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73641068c0 0x7f7364196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7361d9b700 1 -- 172.21.15.99:0/806602746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73641068c0 msgr2=0x7f7364196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7361d9b700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73641068c0 0x7f7364196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7361d9b700 1 -- 172.21.15.99:0/806602746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73641008b0 msgr2=0x7f736419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7361d9b700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73641008b0 0x7f736419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f7361d9b700 1 -- 172.21.15.99:0/806602746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73580095f0 con 0x7f7364101270 2023-08-25T09:03:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.654+0000 7f736259c700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73641008b0 0x7f736419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.655+0000 7f7361d9b700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7364101270 0x7f736419ce70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7358000c00 tx=0x7f7358004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.655+0000 7f73537fe700 1 -- 172.21.15.99:0/806602746 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735801d070 con 0x7f7364101270 2023-08-25T09:03:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.655+0000 7f73537fe700 1 -- 172.21.15.99:0/806602746 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7358021410 con 0x7f7364101270 2023-08-25T09:03:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.655+0000 7f7368dff700 1 -- 172.21.15.99:0/806602746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7364197410 con 0x7f7364101270 2023-08-25T09:03:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.655+0000 7f73537fe700 1 -- 172.21.15.99:0/806602746 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358017650 con 0x7f7364101270 2023-08-25T09:03:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.655+0000 7f7368dff700 1 -- 172.21.15.99:0/806602746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73641a3920 con 0x7f7364101270 2023-08-25T09:03:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.658+0000 7f73537fe700 1 -- 172.21.15.99:0/806602746 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f7358021580 con 0x7f7364101270 2023-08-25T09:03:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.659+0000 7f737c811b80 1 Processor -- start 2023-08-25T09:03:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f737c811b80 1 -- start start 2023-08-25T09:03:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f73537fe700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f734c0a00c0 0x7f734c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f73537fe700 1 -- 172.21.15.99:0/806602746 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f73580cc9c0 con 0x7f7364101270 2023-08-25T09:03:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f737c811b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 0x564d0dc97e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f737c811b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dc9beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f737c811b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc9c3f0 0x564d0dc9c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f737c811b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dbfb750 con 0x564d0dc98370 2023-08-25T09:03:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f737c811b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dbec720 con 0x564d0dc9c3f0 2023-08-25T09:03:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f737c811b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dbea040 con 0x564d0dc97a50 2023-08-25T09:03:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f7362d9d700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f734c0a00c0 0x7f734c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f7361d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dc9beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f736259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 0x564d0dc97e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f7361d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dc9beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58268/0 (socket says 172.21.15.99:58268) 2023-08-25T09:03:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.660+0000 7f7361d9b700 1 -- 172.21.15.99:0/4029406515 learned_addr learned my addr 172.21.15.99:0/4029406515 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7361d9b700 1 -- 172.21.15.99:0/4029406515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 msgr2=0x564d0dc97e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7361d9b700 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 0x564d0dc97e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7361d9b700 1 -- 172.21.15.99:0/4029406515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc9c3f0 msgr2=0x564d0dc9c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7362d9d700 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc9c3f0 0x564d0dc9c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7361d9b700 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc9c3f0 0x564d0dc9c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7361d9b700 1 -- 172.21.15.99:0/4029406515 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d0da655d0 con 0x564d0dc98370 2023-08-25T09:03:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7362d9d700 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc9c3f0 0x564d0dc9c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:03:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7362d9d700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f734c0a00c0 0x7f734c0a2580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f736419d630 tx=0x7f735c009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7361d9b700 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dc9beb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7358096b30 tx=0x7f7358099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8992e41b6f325b1c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7350ff9700 1 -- 172.21.15.99:0/4029406515 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735809d070 con 0x564d0dc98370 2023-08-25T09:03:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7350ff9700 1 -- 172.21.15.99:0/4029406515 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f735809a7c0 con 0x564d0dc98370 2023-08-25T09:03:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.661+0000 7f7350ff9700 1 -- 172.21.15.99:0/4029406515 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735809a320 con 0x564d0dc98370 2023-08-25T09:03:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 -- 172.21.15.99:0/4029406515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 msgr2=0x564d0dc9beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dc9beb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7358096b30 tx=0x7f7358099f10 comp rx=0 tx=0).stop 2023-08-25T09:03:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 -- 172.21.15.99:0/4029406515 shutdown_connections 2023-08-25T09:03:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc9c3f0 0x564d0dc9c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 0x564d0dc97e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 --2- 172.21.15.99:0/4029406515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dc9beb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 -- 172.21.15.99:0/4029406515 >> 172.21.15.99:0/4029406515 conn(0x564d0dc08080 msgr2=0x564d0dc8dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 -- 172.21.15.99:0/4029406515 shutdown_connections 2023-08-25T09:03:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.662+0000 7f737c811b80 1 -- 172.21.15.99:0/4029406515 wait complete. 2023-08-25T09:03:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.663+0000 7f737c811b80 1 -- start start 2023-08-25T09:03:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.663+0000 7f737c811b80 1 Processor -- start 2023-08-25T09:03:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f737c811b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 0x564d0dca3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f737c811b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc98370 0x564d0dca0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f737c811b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f737c811b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dbe5930 con 0x564d0dc9c3f0 2023-08-25T09:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f737c811b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0db83b50 con 0x564d0dc98370 2023-08-25T09:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f737c811b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0db83850 con 0x564d0dc97a50 2023-08-25T09:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f737c811b80 10 client.0 _ll_register_callbacks cb 0x564d0dacda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f7362d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f7361d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc98370 0x564d0dca0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f7362d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58282/0 (socket says 172.21.15.99:58282) 2023-08-25T09:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f7362d9d700 1 -- 172.21.15.99:0/3719664453 learned_addr learned my addr 172.21.15.99:0/3719664453 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.664+0000 7f736259c700 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 0x564d0dca3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7361d9b700 1 -- 172.21.15.99:0/3719664453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 msgr2=0x564d0dca3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7361d9b700 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 0x564d0dca3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7361d9b700 1 -- 172.21.15.99:0/3719664453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 msgr2=0x564d0dca13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7361d9b700 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7361d9b700 1 -- 172.21.15.99:0/3719664453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73641a3920 con 0x564d0dc98370 2023-08-25T09:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7361d9b700 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc98370 0x564d0dca0db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7358096b30 tx=0x7f7358021c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34a9090dbc1d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7349ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7349ffb700 1 -- 172.21.15.99:0/3719664453 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735809d070 con 0x564d0dc98370 2023-08-25T09:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f737c811b80 1 -- 172.21.15.99:0/3719664453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d0dc63160 con 0x564d0dc98370 2023-08-25T09:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7349ffb700 1 -- 172.21.15.99:0/3719664453 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f735809a0c0 con 0x564d0dc98370 2023-08-25T09:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f7349ffb700 1 -- 172.21.15.99:0/3719664453 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73580b2210 con 0x564d0dc98370 2023-08-25T09:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f737c811b80 10 client.25655 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.665+0000 7f737c811b80 1 -- 172.21.15.99:0/3719664453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564d0da11d80 con 0x564d0dc98370 2023-08-25T09:03:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f7349ffb700 1 -- 172.21.15.99:0/3719664453 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d0dc63160 con 0x564d0dc98370 2023-08-25T09:03:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f7349ffb700 1 -- 172.21.15.99:0/3719664453 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7358095eb0 con 0x564d0dc98370 2023-08-25T09:03:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f737c811b80 10 client.25655 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f737c811b80 10 client.25655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f737c811b80 20 client.25655 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f737c811b80 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564d0dcb0d90 0x564d0dcb3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f737c811b80 4 client.25655 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:03:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f737c811b80 1 -- 172.21.15.99:0/3719664453 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d0db8bb40 con 0x564d0dcb0d90 2023-08-25T09:03:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.666+0000 7f736259c700 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564d0dcb0d90 0x564d0dcb3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.667+0000 7f736259c700 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564d0dcb0d90 0x564d0dcb3180 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc2d0492ae8f3a6 server_cookie=d9b65c36bfea3cb5 in_seq=0 out_seq=0 2023-08-25T09:03:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.667+0000 7f7349ffb700 10 client.25655 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.667+0000 7f7349ffb700 1 -- 172.21.15.99:0/3719664453 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564d0db8bb40 con 0x564d0dcb0d90 2023-08-25T09:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.667+0000 7f7349ffb700 10 client.25655 handle_command_reply: tid=0 2023-08-25T09:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.667+0000 7f737c811b80 1 client.25655 shutdown 2023-08-25T09:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 -- 172.21.15.99:0/3719664453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc98370 msgr2=0x564d0dca0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc98370 0x564d0dca0db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7358096b30 tx=0x7f7358021c00 comp rx=0 tx=0).stop 2023-08-25T09:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 -- 172.21.15.99:0/3719664453 shutdown_connections 2023-08-25T09:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc97a50 0x564d0dca3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564d0dcb0d90 0x564d0dcb3180 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 --2- 172.21.15.99:0/3719664453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc98370 0x564d0dca0db0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f7349ffb700 0 client.25655 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 -- 172.21.15.99:0/3719664453 >> 172.21.15.99:0/3719664453 conn(0x564d0dc08080 msgr2=0x564d0dc9d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 -- 172.21.15.99:0/3719664453 shutdown_connections 2023-08-25T09:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 1 -- 172.21.15.99:0/3719664453 wait complete. 2023-08-25T09:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.668+0000 7f737c811b80 20 client.25655 trim_cache size 0 max 16384 2023-08-25T09:03:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.683+0000 7f737c811b80 1 Processor -- start 2023-08-25T09:03:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.683+0000 7f737c811b80 1 -- start start 2023-08-25T09:03:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.683+0000 7f737c811b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dc99ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.683+0000 7f737c811b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dca8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.683+0000 7f737c811b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc9c3f0 0x564d0dca0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.683+0000 7f737c811b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0da678c0 con 0x564d0dc98370 2023-08-25T09:03:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.683+0000 7f737c811b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dc8ec60 con 0x564d0dc97a50 2023-08-25T09:03:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.683+0000 7f737c811b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dc86390 con 0x564d0dc9c3f0 2023-08-25T09:03:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dc99ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f7361d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dca8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dc99ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:46834/0 (socket says 172.21.15.99:46834) 2023-08-25T09:03:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 -- 172.21.15.99:0/2956638035 learned_addr learned my addr 172.21.15.99:0/2956638035 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f7362d9d700 1 --2- 172.21.15.99:0/2956638035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc9c3f0 0x564d0dca0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 -- 172.21.15.99:0/2956638035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc9c3f0 msgr2=0x564d0dca0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 --2- 172.21.15.99:0/2956638035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc9c3f0 0x564d0dca0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 -- 172.21.15.99:0/2956638035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 msgr2=0x564d0dca8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 --2- 172.21.15.99:0/2956638035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dca8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 -- 172.21.15.99:0/2956638035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d0dc88be0 con 0x564d0dc97a50 2023-08-25T09:03:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.684+0000 7f736259c700 1 --2- 172.21.15.99:0/2956638035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dc99ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7354005d90 tx=0x7f7354009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae33b32f4fa5da1c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f73517fa700 1 -- 172.21.15.99:0/2956638035 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354017040 con 0x564d0dc97a50 2023-08-25T09:03:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f73517fa700 1 -- 172.21.15.99:0/2956638035 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7354018040 con 0x564d0dc97a50 2023-08-25T09:03:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f73517fa700 1 -- 172.21.15.99:0/2956638035 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354005020 con 0x564d0dc97a50 2023-08-25T09:03:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 -- 172.21.15.99:0/2956638035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 msgr2=0x564d0dc99ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 --2- 172.21.15.99:0/2956638035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dc99ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7354005d90 tx=0x7f7354009260 comp rx=0 tx=0).stop 2023-08-25T09:03:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 -- 172.21.15.99:0/2956638035 shutdown_connections 2023-08-25T09:03:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 --2- 172.21.15.99:0/2956638035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc9c3f0 0x564d0dca0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 --2- 172.21.15.99:0/2956638035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dc99ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 --2- 172.21.15.99:0/2956638035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc98370 0x564d0dca8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 -- 172.21.15.99:0/2956638035 >> 172.21.15.99:0/2956638035 conn(0x564d0dc08080 msgr2=0x564d0dca3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 -- 172.21.15.99:0/2956638035 shutdown_connections 2023-08-25T09:03:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.685+0000 7f737c811b80 1 -- 172.21.15.99:0/2956638035 wait complete. 2023-08-25T09:03:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.686+0000 7f737c811b80 1 -- start start 2023-08-25T09:03:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.686+0000 7f737c811b80 1 Processor -- start 2023-08-25T09:03:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.686+0000 7f737c811b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dca7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.686+0000 7f737c811b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc98370 0x564d0dca80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.686+0000 7f737c811b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.686+0000 7f737c811b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dca6890 con 0x564d0dc9c3f0 2023-08-25T09:03:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.686+0000 7f737c811b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dca6a00 con 0x564d0dc97a50 2023-08-25T09:03:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.686+0000 7f737c811b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0dca6bc0 con 0x564d0dc98370 2023-08-25T09:03:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f737c811b80 10 client.0 _ll_register_callbacks cb 0x7f7354009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:03:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f7362d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f7361d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc98370 0x564d0dca80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f736259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dca7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f7361d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc98370 0x564d0dca80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:48132/0 (socket says 172.21.15.99:48132) 2023-08-25T09:03:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f7361d9b700 1 -- 172.21.15.99:0/3305093916 learned_addr learned my addr 172.21.15.99:0/3305093916 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:03:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f736259c700 1 -- 172.21.15.99:0/3305093916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc98370 msgr2=0x564d0dca80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f736259c700 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc98370 0x564d0dca80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f736259c700 1 -- 172.21.15.99:0/3305093916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 msgr2=0x564d0dca6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:03:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f736259c700 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f736259c700 1 -- 172.21.15.99:0/3305093916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7354012040 con 0x564d0dc97a50 2023-08-25T09:03:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.687+0000 7f7361d9b700 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc98370 0x564d0dca80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:03:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f736259c700 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dca7b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7354012590 tx=0x7f7354004900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a577e9b6314881f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:03:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f7349ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:03:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f7349ffb700 1 -- 172.21.15.99:0/3305093916 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354017040 con 0x564d0dc97a50 2023-08-25T09:03:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f737c811b80 1 -- 172.21.15.99:0/3305093916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d0dc86600 con 0x564d0dc97a50 2023-08-25T09:03:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f7349ffb700 1 -- 172.21.15.99:0/3305093916 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f735401e530 con 0x564d0dc97a50 2023-08-25T09:03:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f7349ffb700 1 -- 172.21.15.99:0/3305093916 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7354004d90 con 0x564d0dc97a50 2023-08-25T09:03:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f737c811b80 10 client.25661 fetch_fsmap learned FSMap version 28 2023-08-25T09:03:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f737c811b80 1 -- 172.21.15.99:0/3305093916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564d0d9fac00 con 0x564d0dc97a50 2023-08-25T09:03:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.688+0000 7f7349ffb700 1 -- 172.21.15.99:0/3305093916 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d0dc86600 con 0x564d0dc97a50 2023-08-25T09:03:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f7349ffb700 1 -- 172.21.15.99:0/3305093916 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7354012640 con 0x564d0dc97a50 2023-08-25T09:03:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f737c811b80 10 client.25661 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:03:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f737c811b80 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:03:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f737c811b80 20 client.25661 populate_metadata read hostname 'smithi099' 2023-08-25T09:03:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f737c811b80 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564d0dd05c80 0x564d0dd08070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:03:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f737c811b80 4 client.25661 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:03:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f737c811b80 1 -- 172.21.15.99:0/3305093916 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564d0dbde910 con 0x564d0dd05c80 2023-08-25T09:03:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f7361d9b700 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564d0dd05c80 0x564d0dd08070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:03:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f7349ffb700 10 client.25661 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.689+0000 7f7361d9b700 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564d0dd05c80 0x564d0dd08070 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58f4d84e81781409 server_cookie=18711b0771870164 in_seq=0 out_seq=0 2023-08-25T09:03:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f7349ffb700 1 -- 172.21.15.99:0/3305093916 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564d0dbde910 con 0x564d0dd05c80 2023-08-25T09:03:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f7349ffb700 10 client.25661 handle_command_reply: tid=0 2023-08-25T09:03:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 client.25661 shutdown 2023-08-25T09:03:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 -- 172.21.15.99:0/3305093916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 msgr2=0x564d0dca7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dca7b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7354012590 tx=0x7f7354004900 comp rx=0 tx=0).stop 2023-08-25T09:03:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 -- 172.21.15.99:0/3305093916 shutdown_connections 2023-08-25T09:03:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d0dc9c3f0 0x564d0dca6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564d0dd05c80 0x564d0dd08070 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d0dc97a50 0x564d0dca7b80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f7349ffb700 0 client.25661 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:03:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 --2- 172.21.15.99:0/3305093916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d0dc98370 0x564d0dca80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.726+0000 7f737c811b80 1 -- 172.21.15.99:0/3305093916 >> 172.21.15.99:0/3305093916 conn(0x564d0dc08080 msgr2=0x564d0dc87f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f737c811b80 1 -- 172.21.15.99:0/3305093916 shutdown_connections 2023-08-25T09:03:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f737c811b80 1 -- 172.21.15.99:0/3305093916 wait complete. 2023-08-25T09:03:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f737c811b80 20 client.25661 trim_cache size 0 max 16384 2023-08-25T09:03:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:03:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:03:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:03:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:03:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f7368dff700 1 -- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f734c0a00c0 msgr2=0x7f734c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f7368dff700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f734c0a00c0 0x7f734c0a2580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f736419d630 tx=0x7f735c009920 comp rx=0 tx=0).stop 2023-08-25T09:03:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f7368dff700 1 -- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7364101270 msgr2=0x7f736419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:03:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f7368dff700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7364101270 0x7f736419ce70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7358000c00 tx=0x7f7358004790 comp rx=0 tx=0).stop 2023-08-25T09:03:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f7368dff700 1 -- 172.21.15.99:0/806602746 shutdown_connections 2023-08-25T09:03:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.727+0000 7f7368dff700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f734c0a00c0 0x7f734c0a2580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.728+0000 7f7368dff700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73641068c0 0x7f7364196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.728+0000 7f7368dff700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73641008b0 0x7f736419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.728+0000 7f7368dff700 1 --2- 172.21.15.99:0/806602746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7364101270 0x7f736419ce70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:03:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.728+0000 7f7368dff700 1 -- 172.21.15.99:0/806602746 >> 172.21.15.99:0/806602746 conn(0x7f73640fc380 msgr2=0x7f73640fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:03:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.728+0000 7f7368dff700 1 -- 172.21.15.99:0/806602746 shutdown_connections 2023-08-25T09:03:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:03:57.728+0000 7f7368dff700 1 -- 172.21.15.99:0/806602746 wait complete. 2023-08-25T09:03:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:57 smithi181 ceph-mon[131296]: pgmap v146: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 830 KiB/s wr, 78 op/s 2023-08-25T09:03:57.910 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114487 2.03 MB/sec execute 123 sec latency 4952.562 ms 2023-08-25T09:03:58.910 INFO:tasks.workunit.client.0.smithi099.stdout: 1 116715 2.11 MB/sec execute 124 sec latency 18.035 ms 2023-08-25T09:03:59.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:03:59 smithi099 ceph-mon[97641]: pgmap v147: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 766 KiB/s wr, 55 op/s 2023-08-25T09:03:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:03:59 smithi157 ceph-mon[130879]: pgmap v147: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 766 KiB/s wr, 55 op/s 2023-08-25T09:03:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:03:59 smithi181 ceph-mon[131296]: pgmap v147: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 766 KiB/s wr, 55 op/s 2023-08-25T09:03:59.910 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117756 2.14 MB/sec execute 125 sec latency 137.443 ms 2023-08-25T09:04:00.910 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117812 2.12 MB/sec execute 126 sec latency 757.192 ms 2023-08-25T09:04:01.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:01 smithi099 ceph-mon[97641]: pgmap v148: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 766 KiB/s wr, 55 op/s 2023-08-25T09:04:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:01 smithi157 ceph-mon[130879]: pgmap v148: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 766 KiB/s wr, 55 op/s 2023-08-25T09:04:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:01 smithi181 ceph-mon[131296]: pgmap v148: 97 pgs: 97 active+clean; 55 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 766 KiB/s wr, 55 op/s 2023-08-25T09:04:01.910 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117812 2.11 MB/sec execute 127 sec latency 1757.371 ms 2023-08-25T09:04:02.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117880 2.09 MB/sec execute 128 sec latency 2667.022 ms 2023-08-25T09:04:03.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:03 smithi099 ceph-mon[97641]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-08-25T09:04:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:03 smithi157 ceph-mon[130879]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-08-25T09:04:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:03 smithi181 ceph-mon[131296]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-08-25T09:04:03.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117880 2.08 MB/sec execute 129 sec latency 1039.188 ms 2023-08-25T09:04:04.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117880 2.06 MB/sec execute 130 sec latency 2039.355 ms 2023-08-25T09:04:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:05 smithi181 ceph-mon[131296]: pgmap v150: 97 pgs: 97 active+clean; 57 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 890 KiB/s wr, 83 op/s 2023-08-25T09:04:05.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:05 smithi099 ceph-mon[97641]: pgmap v150: 97 pgs: 97 active+clean; 57 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 890 KiB/s wr, 83 op/s 2023-08-25T09:04:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:05 smithi157 ceph-mon[130879]: pgmap v150: 97 pgs: 97 active+clean; 57 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 890 KiB/s wr, 83 op/s 2023-08-25T09:04:05.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117880 2.05 MB/sec execute 131 sec latency 3039.516 ms 2023-08-25T09:04:06.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117880 2.03 MB/sec execute 132 sec latency 4039.678 ms 2023-08-25T09:04:07.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:07 smithi099 ceph-mon[97641]: pgmap v151: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 938 KiB/s wr, 97 op/s 2023-08-25T09:04:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:07 smithi157 ceph-mon[130879]: pgmap v151: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 938 KiB/s wr, 97 op/s 2023-08-25T09:04:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:07 smithi181 ceph-mon[131296]: pgmap v151: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 938 KiB/s wr, 97 op/s 2023-08-25T09:04:07.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 118023 2.02 MB/sec execute 133 sec latency 4947.064 ms 2023-08-25T09:04:08.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 119508 2.05 MB/sec execute 134 sec latency 15.774 ms 2023-08-25T09:04:09.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:09 smithi099 ceph-mon[97641]: pgmap v152: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 655 KiB/s wr, 80 op/s 2023-08-25T09:04:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:09 smithi157 ceph-mon[130879]: pgmap v152: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 655 KiB/s wr, 80 op/s 2023-08-25T09:04:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:09 smithi181 ceph-mon[131296]: pgmap v152: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 655 KiB/s wr, 80 op/s 2023-08-25T09:04:09.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 121099 2.12 MB/sec execute 135 sec latency 11.210 ms 2023-08-25T09:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 -- 172.21.15.99:0/2795075255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa594073670 msgr2=0x7fa59410cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/2795075255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa594073670 0x7fa59410cbb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa584009990 tx=0x7fa584009ca0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 -- 172.21.15.99:0/2795075255 shutdown_connections 2023-08-25T09:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/2795075255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa59410d0f0 0x7fa59410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/2795075255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa594072d50 0x7fa594073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/2795075255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa594073670 0x7fa59410cbb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 -- 172.21.15.99:0/2795075255 >> 172.21.15.99:0/2795075255 conn(0x7fa5940fc380 msgr2=0x7fa5940fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 -- 172.21.15.99:0/2795075255 shutdown_connections 2023-08-25T09:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.879+0000 7fa59c7ff700 1 -- 172.21.15.99:0/2795075255 wait complete. 2023-08-25T09:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.880+0000 7fa59c7ff700 1 Processor -- start 2023-08-25T09:04:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.880+0000 7fa59c7ff700 1 -- start start 2023-08-25T09:04:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59c7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa594072d50 0x7fa59419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59c7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa594073670 0x7fa59419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59c7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59410d0f0 0x7fa594198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59c7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941990c0 con 0x7fa594072d50 2023-08-25T09:04:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59c7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594199230 con 0x7fa594073670 2023-08-25T09:04:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59c7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941993f0 con 0x7fa59410d0f0 2023-08-25T09:04:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa594072d50 0x7fa59419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa594072d50 0x7fa59419ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48092/0 (socket says 172.21.15.99:48092) 2023-08-25T09:04:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59a59b700 1 -- 172.21.15.99:0/648499509 learned_addr learned my addr 172.21.15.99:0/648499509 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59410d0f0 0x7fa594198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa599d9a700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa594073670 0x7fa59419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59a59b700 1 -- 172.21.15.99:0/648499509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59410d0f0 msgr2=0x7fa594198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59a59b700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59410d0f0 0x7fa594198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59a59b700 1 -- 172.21.15.99:0/648499509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa594073670 msgr2=0x7fa59419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59a59b700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa594073670 0x7fa59419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.881+0000 7fa59a59b700 1 -- 172.21.15.99:0/648499509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840095f0 con 0x7fa594072d50 2023-08-25T09:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.882+0000 7fa59a59b700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa594072d50 0x7fa59419ebf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa59000e9a0 tx=0x7fa59000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.882+0000 7fa58b7fe700 1 -- 172.21.15.99:0/648499509 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590009790 con 0x7fa594072d50 2023-08-25T09:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.882+0000 7fa58b7fe700 1 -- 172.21.15.99:0/648499509 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa590010430 con 0x7fa594072d50 2023-08-25T09:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.882+0000 7fa59c7ff700 1 -- 172.21.15.99:0/648499509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5941996d0 con 0x7fa594072d50 2023-08-25T09:04:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.882+0000 7fa58b7fe700 1 -- 172.21.15.99:0/648499509 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590018680 con 0x7fa594072d50 2023-08-25T09:04:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.882+0000 7fa59c7ff700 1 -- 172.21.15.99:0/648499509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941a5be0 con 0x7fa594072d50 2023-08-25T09:04:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.885+0000 7fa58b7fe700 1 -- 172.21.15.99:0/648499509 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa5900105a0 con 0x7fa594072d50 2023-08-25T09:04:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa5b0203b80 1 Processor -- start 2023-08-25T09:04:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa5b0203b80 1 -- start start 2023-08-25T09:04:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa58b7fe700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa5800a00c0 0x7fa5800a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa5282d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282d860 0x55fa52827450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa58b7fe700 1 -- 172.21.15.99:0/648499509 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa590014070 con 0x7fa594072d50 2023-08-25T09:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa52827990 0x55fa52827dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5278f750 con 0x55fa5282cf40 2023-08-25T09:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa52780720 con 0x55fa5282d860 2023-08-25T09:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.887+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5277e040 con 0x55fa52827990 2023-08-25T09:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa5800a00c0 0x7fa5800a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa5282d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa52827990 0x55fa52827dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa52827990 0x55fa52827dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57038/0 (socket says 172.21.15.99:57038) 2023-08-25T09:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 -- 172.21.15.99:0/1783048467 learned_addr learned my addr 172.21.15.99:0/1783048467 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa599d9a700 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282d860 0x55fa52827450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 -- 172.21.15.99:0/1783048467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282d860 msgr2=0x55fa52827450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282d860 0x55fa52827450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 -- 172.21.15.99:0/1783048467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 msgr2=0x55fa5282d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa5282d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59ad9c700 1 -- 172.21.15.99:0/1783048467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa525f95d0 con 0x55fa52827990 2023-08-25T09:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa599d9a700 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282d860 0x55fa52827450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.888+0000 7fa59a59b700 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa5282d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.889+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa52827990 0x55fa52827dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa58c01a490 tx=0x7fa58c01a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77473e86ef98ea9e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.889+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa5800a00c0 0x7fa5800a2580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa58c01aba0 tx=0x7fa58c012000 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.889+0000 7fa588ff9700 1 -- 172.21.15.99:0/1783048467 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00e040 con 0x55fa52827990 2023-08-25T09:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.889+0000 7fa588ff9700 1 -- 172.21.15.99:0/1783048467 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58c01f070 con 0x55fa52827990 2023-08-25T09:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.889+0000 7fa588ff9700 1 -- 172.21.15.99:0/1783048467 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c003e60 con 0x55fa52827990 2023-08-25T09:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.889+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1783048467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa52827990 msgr2=0x55fa52827dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.889+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa52827990 0x55fa52827dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa58c01a490 tx=0x7fa58c01a7a0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1783048467 shutdown_connections 2023-08-25T09:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa52827990 0x55fa52827dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa5282d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1783048467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282d860 0x55fa52827450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1783048467 >> 172.21.15.99:0/1783048467 conn(0x55fa5279c080 msgr2=0x55fa52821d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1783048467 shutdown_connections 2023-08-25T09:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1783048467 wait complete. 2023-08-25T09:04:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 -- start start 2023-08-25T09:04:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.890+0000 7fa5b0203b80 1 Processor -- start 2023-08-25T09:04:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa52827990 0x55fa52833740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa52830a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52831170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa52779930 con 0x55fa5282cf40 2023-08-25T09:04:10.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa52717b50 con 0x55fa52827990 2023-08-25T09:04:10.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa52717850 con 0x55fa5282d860 2023-08-25T09:04:10.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa5b0203b80 10 client.0 _ll_register_callbacks cb 0x55fa52661a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:04:10.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59a59b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa52827990 0x55fa52833740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59a59b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa52827990 0x55fa52833740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:59566/0 (socket says 172.21.15.99:59566) 2023-08-25T09:04:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa599d9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa52830a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59a59b700 1 -- 172.21.15.99:0/1757544780 learned_addr learned my addr 172.21.15.99:0/1757544780 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52831170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59a59b700 1 -- 172.21.15.99:0/1757544780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 msgr2=0x55fa52831170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59a59b700 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52831170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59a59b700 1 -- 172.21.15.99:0/1757544780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 msgr2=0x55fa52830a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59a59b700 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa52830a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.891+0000 7fa59a59b700 1 -- 172.21.15.99:0/1757544780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941a5be0 con 0x55fa52827990 2023-08-25T09:04:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa59a59b700 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa52827990 0x55fa52833740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa590098c20 tx=0x7fa5900996a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9af1838a604a513 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa57dffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:04:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa57dffb700 1 -- 172.21.15.99:0/1757544780 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590098610 con 0x55fa52827990 2023-08-25T09:04:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa57dffb700 1 -- 172.21.15.99:0/1757544780 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa59009cd50 con 0x55fa52827990 2023-08-25T09:04:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1757544780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa527f7160 con 0x55fa52827990 2023-08-25T09:04:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa57dffb700 1 -- 172.21.15.99:0/1757544780 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5900a0d30 con 0x55fa52827990 2023-08-25T09:04:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa5b0203b80 10 client.25673 fetch_fsmap learned FSMap version 28 2023-08-25T09:04:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1757544780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fa58c013c80 con 0x55fa52827990 2023-08-25T09:04:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.892+0000 7fa57dffb700 1 -- 172.21.15.99:0/1757544780 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fa527f7160 con 0x55fa52827990 2023-08-25T09:04:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa57dffb700 1 -- 172.21.15.99:0/1757544780 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa590093cc0 con 0x55fa52827990 2023-08-25T09:04:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa5b0203b80 10 client.25673 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:04:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa5b0203b80 10 client.25673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:04:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa5b0203b80 20 client.25673 populate_metadata read hostname 'smithi099' 2023-08-25T09:04:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fa52841290 0x55fa528436b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa5b0203b80 4 client.25673 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:04:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1757544780 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa5271fb40 con 0x55fa52841290 2023-08-25T09:04:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa599d9a700 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fa52841290 0x55fa528436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa57dffb700 10 client.25673 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.893+0000 7fa599d9a700 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fa52841290 0x55fa528436b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9130d819f85bfe server_cookie=1d5006c26c97db71 in_seq=0 out_seq=0 2023-08-25T09:04:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa57dffb700 1 -- 172.21.15.99:0/1757544780 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fa5271fb40 con 0x55fa52841290 2023-08-25T09:04:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa57dffb700 10 client.25673 handle_command_reply: tid=0 2023-08-25T09:04:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 client.25673 shutdown 2023-08-25T09:04:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1757544780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa52827990 msgr2=0x55fa52833740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa52827990 0x55fa52833740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa590098c20 tx=0x7fa5900996a0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1757544780 shutdown_connections 2023-08-25T09:04:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52831170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fa52841290 0x55fa528436b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa52827990 0x55fa52833740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa57dffb700 0 client.25673 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/1757544780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa5282cf40 0x55fa52830a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1757544780 >> 172.21.15.99:0/1757544780 conn(0x55fa5279c080 msgr2=0x55fa5281c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.894+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1757544780 shutdown_connections 2023-08-25T09:04:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.895+0000 7fa5b0203b80 1 -- 172.21.15.99:0/1757544780 wait complete. 2023-08-25T09:04:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.895+0000 7fa5b0203b80 20 client.25673 trim_cache size 0 max 16384 2023-08-25T09:04:10.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 122810 2.15 MB/sec execute 136 sec latency 19.320 ms 2023-08-25T09:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.911+0000 7fa5b0203b80 1 Processor -- start 2023-08-25T09:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa5b0203b80 1 -- start start 2023-08-25T09:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52827450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa528381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa528332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa525fb8c0 con 0x55fa52827990 2023-08-25T09:04:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa52822c90 con 0x55fa5282cf40 2023-08-25T09:04:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa52823830 con 0x55fa5282d860 2023-08-25T09:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa599d9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa528381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa59a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52827450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa599d9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa528381b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:59578/0 (socket says 172.21.15.99:59578) 2023-08-25T09:04:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.912+0000 7fa599d9a700 1 -- 172.21.15.99:0/971608045 learned_addr learned my addr 172.21.15.99:0/971608045 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa528332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa599d9a700 1 -- 172.21.15.99:0/971608045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 msgr2=0x55fa528332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa599d9a700 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa528332e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa599d9a700 1 -- 172.21.15.99:0/971608045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 msgr2=0x55fa52827450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa599d9a700 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52827450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa599d9a700 1 -- 172.21.15.99:0/971608045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa525a5d80 con 0x55fa5282cf40 2023-08-25T09:04:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa528332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:04:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa59a59b700 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52827450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa599d9a700 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa528381b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa584009990 tx=0x7fa584005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cb35779d1e720 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa5897fa700 1 -- 172.21.15.99:0/971608045 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58401d040 con 0x55fa5282cf40 2023-08-25T09:04:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa5897fa700 1 -- 172.21.15.99:0/971608045 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58401e040 con 0x55fa5282cf40 2023-08-25T09:04:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.913+0000 7fa5897fa700 1 -- 172.21.15.99:0/971608045 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400be30 con 0x55fa5282cf40 2023-08-25T09:04:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 -- 172.21.15.99:0/971608045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 msgr2=0x55fa528381b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa528381b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa584009990 tx=0x7fa584005fb0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 -- 172.21.15.99:0/971608045 shutdown_connections 2023-08-25T09:04:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa528332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52827450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/971608045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa528381b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 -- 172.21.15.99:0/971608045 >> 172.21.15.99:0/971608045 conn(0x55fa5279c080 msgr2=0x55fa5282ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 -- 172.21.15.99:0/971608045 shutdown_connections 2023-08-25T09:04:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.914+0000 7fa5b0203b80 1 -- 172.21.15.99:0/971608045 wait complete. 2023-08-25T09:04:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 1 -- start start 2023-08-25T09:04:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 1 Processor -- start 2023-08-25T09:04:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52829f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa52828880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52828dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5282a660 con 0x55fa52827990 2023-08-25T09:04:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa52829300 con 0x55fa5282cf40 2023-08-25T09:04:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa528294c0 con 0x55fa5282d860 2023-08-25T09:04:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.915+0000 7fa5b0203b80 10 client.0 _ll_register_callbacks cb 0x7fa590098880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:04:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa52828880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa59ad9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52828dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa59a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52829f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa52828880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:59584/0 (socket says 172.21.15.99:59584) 2023-08-25T09:04:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 -- 172.21.15.99:0/3340263738 learned_addr learned my addr 172.21.15.99:0/3340263738 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa59ad9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52828dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57070/0 (socket says 172.21.15.99:57070) 2023-08-25T09:04:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 -- 172.21.15.99:0/3340263738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 msgr2=0x55fa52828dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52828dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 -- 172.21.15.99:0/3340263738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 msgr2=0x55fa52829f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52829f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 -- 172.21.15.99:0/3340263738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c013c80 con 0x55fa5282cf40 2023-08-25T09:04:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa59a59b700 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52829f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa59ad9c700 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52828dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa599d9a700 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa52828880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b2e0 tx=0x7fa58400bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d5377b0d07561e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.916+0000 7fa57dffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:04:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa57dffb700 1 -- 172.21.15.99:0/3340263738 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58401d040 con 0x55fa5282cf40 2023-08-25T09:04:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa5b0203b80 1 -- 172.21.15.99:0/3340263738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa5281a600 con 0x55fa5282cf40 2023-08-25T09:04:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa57dffb700 1 -- 172.21.15.99:0/3340263738 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa584028a80 con 0x55fa5282cf40 2023-08-25T09:04:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa57dffb700 1 -- 172.21.15.99:0/3340263738 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa584028c60 con 0x55fa5282cf40 2023-08-25T09:04:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa5b0203b80 10 client.25679 fetch_fsmap learned FSMap version 28 2023-08-25T09:04:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa5b0203b80 1 -- 172.21.15.99:0/3340263738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fa594110220 con 0x55fa5282cf40 2023-08-25T09:04:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa57dffb700 1 -- 172.21.15.99:0/3340263738 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fa5281a600 con 0x55fa5282cf40 2023-08-25T09:04:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa57dffb700 1 -- 172.21.15.99:0/3340263738 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa584004500 con 0x55fa5282cf40 2023-08-25T09:04:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa5b0203b80 10 client.25679 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:04:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa5b0203b80 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:04:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.917+0000 7fa5b0203b80 20 client.25679 populate_metadata read hostname 'smithi099' 2023-08-25T09:04:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.918+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fa528960c0 0x55fa528984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.918+0000 7fa5b0203b80 4 client.25679 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:04:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.918+0000 7fa5b0203b80 1 -- 172.21.15.99:0/3340263738 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fa52772910 con 0x55fa528960c0 2023-08-25T09:04:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.918+0000 7fa59a59b700 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fa528960c0 0x55fa528984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.918+0000 7fa57dffb700 10 client.25679 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.918+0000 7fa59a59b700 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fa528960c0 0x55fa528984e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21204b941a033a0f server_cookie=68ebd45901d6a098 in_seq=0 out_seq=0 2023-08-25T09:04:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa57dffb700 1 -- 172.21.15.99:0/3340263738 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fa52772910 con 0x55fa528960c0 2023-08-25T09:04:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa57dffb700 10 client.25679 handle_command_reply: tid=0 2023-08-25T09:04:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 client.25679 shutdown 2023-08-25T09:04:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 -- 172.21.15.99:0/3340263738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 msgr2=0x55fa52828880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa52828880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b2e0 tx=0x7fa58400bbc0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 -- 172.21.15.99:0/3340263738 shutdown_connections 2023-08-25T09:04:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa5282d860 0x55fa52828dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa52827990 0x55fa52829f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fa528960c0 0x55fa528984e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 --2- 172.21.15.99:0/3340263738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5282cf40 0x55fa52828880 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa57dffb700 0 client.25679 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 -- 172.21.15.99:0/3340263738 >> 172.21.15.99:0/3340263738 conn(0x55fa5279c080 msgr2=0x55fa5282baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.956+0000 7fa5b0203b80 1 -- 172.21.15.99:0/3340263738 shutdown_connections 2023-08-25T09:04:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.957+0000 7fa5b0203b80 1 -- 172.21.15.99:0/3340263738 wait complete. 2023-08-25T09:04:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.957+0000 7fa5b0203b80 20 client.25679 trim_cache size 0 max 16384 2023-08-25T09:04:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:04:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:04:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:04:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:04:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 -- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa5800a00c0 msgr2=0x7fa5800a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa5800a00c0 0x7fa5800a2580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa58c01aba0 tx=0x7fa58c012000 comp rx=0 tx=0).stop 2023-08-25T09:04:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 -- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa594072d50 msgr2=0x7fa59419ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa594072d50 0x7fa59419ebf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa59000e9a0 tx=0x7fa59000ecb0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 -- 172.21.15.99:0/648499509 shutdown_connections 2023-08-25T09:04:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa5800a00c0 0x7fa5800a2580 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59410d0f0 0x7fa594198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa594072d50 0x7fa59419ebf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 --2- 172.21.15.99:0/648499509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa594073670 0x7fa59419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 -- 172.21.15.99:0/648499509 >> 172.21.15.99:0/648499509 conn(0x7fa5940fc380 msgr2=0x7fa59410f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 -- 172.21.15.99:0/648499509 shutdown_connections 2023-08-25T09:04:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:10.958+0000 7fa59c7ff700 1 -- 172.21.15.99:0/648499509 wait complete. 2023-08-25T09:04:11.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:11 smithi099 ceph-mon[97641]: pgmap v153: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 655 KiB/s wr, 80 op/s 2023-08-25T09:04:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:11 smithi157 ceph-mon[130879]: pgmap v153: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 655 KiB/s wr, 80 op/s 2023-08-25T09:04:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:11 smithi181 ceph-mon[131296]: pgmap v153: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 655 KiB/s wr, 80 op/s 2023-08-25T09:04:11.911 INFO:tasks.workunit.client.0.smithi099.stdout: 1 124569 2.23 MB/sec execute 137 sec latency 16.647 ms 2023-08-25T09:04:12.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 124973 2.22 MB/sec execute 138 sec latency 614.096 ms 2023-08-25T09:04:13.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:13 smithi099 ceph-mon[97641]: pgmap v154: 97 pgs: 97 active+clean; 58 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2023-08-25T09:04:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:13 smithi157 ceph-mon[130879]: pgmap v154: 97 pgs: 97 active+clean; 58 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2023-08-25T09:04:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:13 smithi181 ceph-mon[131296]: pgmap v154: 97 pgs: 97 active+clean; 58 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2023-08-25T09:04:13.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 124973 2.20 MB/sec execute 139 sec latency 1001.589 ms 2023-08-25T09:04:14.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 124973 2.19 MB/sec execute 140 sec latency 2001.738 ms 2023-08-25T09:04:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:15 smithi181 ceph-mon[131296]: pgmap v155: 97 pgs: 97 active+clean; 58 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 729 KiB/s wr, 89 op/s 2023-08-25T09:04:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:15 smithi099 ceph-mon[97641]: pgmap v155: 97 pgs: 97 active+clean; 58 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 729 KiB/s wr, 89 op/s 2023-08-25T09:04:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:15 smithi157 ceph-mon[130879]: pgmap v155: 97 pgs: 97 active+clean; 58 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 729 KiB/s wr, 89 op/s 2023-08-25T09:04:15.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 124973 2.17 MB/sec execute 141 sec latency 3001.882 ms 2023-08-25T09:04:16.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 124973 2.16 MB/sec execute 142 sec latency 4001.977 ms 2023-08-25T09:04:17.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:17 smithi099 ceph-mon[97641]: pgmap v156: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 118 op/s 2023-08-25T09:04:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:17 smithi157 ceph-mon[130879]: pgmap v156: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 118 op/s 2023-08-25T09:04:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:17 smithi181 ceph-mon[131296]: pgmap v156: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 118 op/s 2023-08-25T09:04:17.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 125106 2.14 MB/sec execute 143 sec latency 4911.145 ms 2023-08-25T09:04:18.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 127196 2.21 MB/sec execute 144 sec latency 15.217 ms 2023-08-25T09:04:19.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:19 smithi099 ceph-mon[97641]: pgmap v157: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 105 op/s 2023-08-25T09:04:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:19 smithi157 ceph-mon[130879]: pgmap v157: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 105 op/s 2023-08-25T09:04:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:19 smithi181 ceph-mon[131296]: pgmap v157: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 105 op/s 2023-08-25T09:04:19.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128393 2.24 MB/sec execute 145 sec latency 120.830 ms 2023-08-25T09:04:20.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128393 2.23 MB/sec execute 146 sec latency 1120.959 ms 2023-08-25T09:04:21.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:21 smithi099 ceph-mon[97641]: pgmap v158: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 105 op/s 2023-08-25T09:04:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:21 smithi157 ceph-mon[130879]: pgmap v158: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 105 op/s 2023-08-25T09:04:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:21 smithi181 ceph-mon[131296]: pgmap v158: 97 pgs: 97 active+clean; 56 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 105 op/s 2023-08-25T09:04:21.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128393 2.21 MB/sec execute 147 sec latency 2121.097 ms 2023-08-25T09:04:22.912 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128517 2.20 MB/sec execute 148 sec latency 3030.659 ms 2023-08-25T09:04:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:23 smithi157 ceph-mon[130879]: pgmap v159: 97 pgs: 97 active+clean; 58 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 140 op/s 2023-08-25T09:04:23.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:23 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:04:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:23 smithi181 ceph-mon[131296]: pgmap v159: 97 pgs: 97 active+clean; 58 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 140 op/s 2023-08-25T09:04:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:23 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:04:23.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:23 smithi099 ceph-mon[97641]: pgmap v159: 97 pgs: 97 active+clean; 58 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 140 op/s 2023-08-25T09:04:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:23 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:04:23.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128518 2.18 MB/sec execute 149 sec latency 988.422 ms 2023-08-25T09:04:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 -- 172.21.15.99:0/2656964509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 msgr2=0x7f27d4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 --2- 172.21.15.99:0/2656964509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 0x7f27d4100d30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009990 tx=0x7f27c8009ca0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 -- 172.21.15.99:0/2656964509 shutdown_connections 2023-08-25T09:04:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 --2- 172.21.15.99:0/2656964509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d4101270 0x7f27d410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 --2- 172.21.15.99:0/2656964509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d41068c0 0x7f27d4106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 --2- 172.21.15.99:0/2656964509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 0x7f27d4100d30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 -- 172.21.15.99:0/2656964509 >> 172.21.15.99:0/2656964509 conn(0x7f27d40fc380 msgr2=0x7f27d40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 -- 172.21.15.99:0/2656964509 shutdown_connections 2023-08-25T09:04:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.109+0000 7f27d91ff700 1 -- 172.21.15.99:0/2656964509 wait complete. 2023-08-25T09:04:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.110+0000 7f27d91ff700 1 Processor -- start 2023-08-25T09:04:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.110+0000 7f27d91ff700 1 -- start start 2023-08-25T09:04:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d91ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 0x7f27d419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d91ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101270 0x7f27d419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d91ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d41068c0 0x7f27d41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d91ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d41991a0 con 0x7f27d41008b0 2023-08-25T09:04:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d91ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d4199310 con 0x7f27d41068c0 2023-08-25T09:04:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d91ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d41994d0 con 0x7f27d4101270 2023-08-25T09:04:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 0x7f27d419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 0x7f27d419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34282/0 (socket says 172.21.15.99:34282) 2023-08-25T09:04:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d2d9d700 1 -- 172.21.15.99:0/751793927 learned_addr learned my addr 172.21.15.99:0/751793927 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d259c700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101270 0x7f27d419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d359e700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d41068c0 0x7f27d41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d2d9d700 1 -- 172.21.15.99:0/751793927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101270 msgr2=0x7f27d419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101270 0x7f27d419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d2d9d700 1 -- 172.21.15.99:0/751793927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d41068c0 msgr2=0x7f27d41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d41068c0 0x7f27d41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.111+0000 7f27d2d9d700 1 -- 172.21.15.99:0/751793927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80095f0 con 0x7f27d41008b0 2023-08-25T09:04:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.112+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 0x7f27d419ec00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e9a0 tx=0x7f27c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.112+0000 7f27bbfff700 1 -- 172.21.15.99:0/751793927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4009790 con 0x7f27d41008b0 2023-08-25T09:04:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.112+0000 7f27d91ff700 1 -- 172.21.15.99:0/751793927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d41997b0 con 0x7f27d41008b0 2023-08-25T09:04:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.112+0000 7f27bbfff700 1 -- 172.21.15.99:0/751793927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c4010430 con 0x7f27d41008b0 2023-08-25T09:04:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.112+0000 7f27bbfff700 1 -- 172.21.15.99:0/751793927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4018680 con 0x7f27d41008b0 2023-08-25T09:04:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.112+0000 7f27d91ff700 1 -- 172.21.15.99:0/751793927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d41a5bd0 con 0x7f27d41008b0 2023-08-25T09:04:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.116+0000 7f27bbfff700 1 -- 172.21.15.99:0/751793927 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f27c40105a0 con 0x7f27d41008b0 2023-08-25T09:04:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.117+0000 7f27ecbf2b80 1 Processor -- start 2023-08-25T09:04:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.117+0000 7f27ecbf2b80 1 -- start start 2023-08-25T09:04:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891e0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891da450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f389142750 con 0x55f3891da990 2023-08-25T09:04:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f389133720 con 0x55f3891e0860 2023-08-25T09:04:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27bbfff700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f27bc0a00c0 0x7f27bc0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f389131040 con 0x55f3891dff40 2023-08-25T09:04:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27bbfff700 1 -- 172.21.15.99:0/751793927 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f27c4014070 con 0x7f27d41008b0 2023-08-25T09:04:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891da450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:55848/0 (socket says 172.21.15.99:55848) 2023-08-25T09:04:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d2d9d700 1 -- 172.21.15.99:0/968779047 learned_addr learned my addr 172.21.15.99:0/968779047 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d359e700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f27bc0a00c0 0x7f27bc0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d359e700 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891e0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d259c700 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d2d9d700 1 -- 172.21.15.99:0/968779047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 msgr2=0x55f3891e0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891e0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d2d9d700 1 -- 172.21.15.99:0/968779047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 msgr2=0x55f3891dadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.118+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.119+0000 7f27d2d9d700 1 -- 172.21.15.99:0/968779047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f388fac5d0 con 0x55f3891e0860 2023-08-25T09:04:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.119+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891da450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f27c40c8f00 tx=0x7f27c40989a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ba60ff6784895c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.119+0000 7f27d359e700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f27bc0a00c0 0x7f27bc0a2580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27d419f890 tx=0x7f27cc009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.119+0000 7f27b97fa700 1 -- 172.21.15.99:0/968779047 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4099560 con 0x55f3891e0860 2023-08-25T09:04:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.119+0000 7f27b97fa700 1 -- 172.21.15.99:0/968779047 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c4098e60 con 0x55f3891e0860 2023-08-25T09:04:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.119+0000 7f27b97fa700 1 -- 172.21.15.99:0/968779047 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40b8310 con 0x55f3891e0860 2023-08-25T09:04:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/968779047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 msgr2=0x55f3891da450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891da450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f27c40c8f00 tx=0x7f27c40989a0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/968779047 shutdown_connections 2023-08-25T09:04:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891e0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/968779047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891da450 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/968779047 >> 172.21.15.99:0/968779047 conn(0x55f38914f080 msgr2=0x55f3891d4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/968779047 shutdown_connections 2023-08-25T09:04:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/968779047 wait complete. 2023-08-25T09:04:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.120+0000 7f27ecbf2b80 1 -- start start 2023-08-25T09:04:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27ecbf2b80 1 Processor -- start 2023-08-25T09:04:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891da990 0x55f3891e6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891e0860 0x55f3891e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38912c930 con 0x55f3891e0860 2023-08-25T09:04:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3890cab50 con 0x55f3891da990 2023-08-25T09:04:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3890ca850 con 0x55f3891dff40 2023-08-25T09:04:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27ecbf2b80 10 client.0 _ll_register_callbacks cb 0x55f389014a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:04:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891da990 0x55f3891e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891da990 0x55f3891e6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:55860/0 (socket says 172.21.15.99:55860) 2023-08-25T09:04:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27d2d9d700 1 -- 172.21.15.99:0/308774525 learned_addr learned my addr 172.21.15.99:0/308774525 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27d359e700 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891e0860 0x55f3891e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.121+0000 7f27d259c700 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27d2d9d700 1 -- 172.21.15.99:0/308774525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 msgr2=0x55f3891e39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891e39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27d2d9d700 1 -- 172.21.15.99:0/308774525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891e0860 msgr2=0x55f3891e4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891e0860 0x55f3891e4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27d2d9d700 1 -- 172.21.15.99:0/308774525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d41a5bd0 con 0x55f3891da990 2023-08-25T09:04:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891da990 0x55f3891e6740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27c4000c00 tx=0x7f27c40c42b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880a90ba3b1f5ada server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:04:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27b6ff5700 1 -- 172.21.15.99:0/308774525 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4098e60 con 0x55f3891da990 2023-08-25T09:04:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/308774525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3891aa160 con 0x55f3891da990 2023-08-25T09:04:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27b6ff5700 1 -- 172.21.15.99:0/308774525 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f27c409cc90 con 0x55f3891da990 2023-08-25T09:04:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27b6ff5700 1 -- 172.21.15.99:0/308774525 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c40a0c70 con 0x55f3891da990 2023-08-25T09:04:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27ecbf2b80 10 client.25691 fetch_fsmap learned FSMap version 28 2023-08-25T09:04:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.122+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/308774525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f388f58d80 con 0x55f3891da990 2023-08-25T09:04:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27b6ff5700 1 -- 172.21.15.99:0/308774525 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3891aa160 con 0x55f3891da990 2023-08-25T09:04:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27b6ff5700 1 -- 172.21.15.99:0/308774525 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f27c40b8040 con 0x55f3891da990 2023-08-25T09:04:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27ecbf2b80 10 client.25691 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:04:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27ecbf2b80 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:04:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27ecbf2b80 20 client.25691 populate_metadata read hostname 'smithi099' 2023-08-25T09:04:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f3891f3fd0 0x55f3891f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27ecbf2b80 4 client.25691 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:04:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/308774525 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3890d2b40 con 0x55f3891f3fd0 2023-08-25T09:04:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.123+0000 7f27d259c700 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f3891f3fd0 0x55f3891f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.124+0000 7f27b6ff5700 10 client.25691 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.124+0000 7f27d259c700 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f3891f3fd0 0x55f3891f63c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d120ec13cc55e0c9 server_cookie=47f8c8789c80ecc1 in_seq=0 out_seq=0 2023-08-25T09:04:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.124+0000 7f27b6ff5700 1 -- 172.21.15.99:0/308774525 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f3890d2b40 con 0x55f3891f3fd0 2023-08-25T09:04:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.124+0000 7f27b6ff5700 10 client.25691 handle_command_reply: tid=0 2023-08-25T09:04:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.124+0000 7f27ecbf2b80 1 client.25691 shutdown 2023-08-25T09:04:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.124+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/308774525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891da990 msgr2=0x55f3891e6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.124+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891da990 0x55f3891e6740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27c4000c00 tx=0x7f27c40c42b0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/308774525 shutdown_connections 2023-08-25T09:04:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891e0860 0x55f3891e4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f3891f3fd0 0x55f3891f63c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891da990 0x55f3891e6740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/308774525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/308774525 >> 172.21.15.99:0/308774525 conn(0x55f38914f080 msgr2=0x55f3891cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27b6ff5700 0 client.25691 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/308774525 shutdown_connections 2023-08-25T09:04:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/308774525 wait complete. 2023-08-25T09:04:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.125+0000 7f27ecbf2b80 20 client.25691 trim_cache size 0 max 16384 2023-08-25T09:04:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27ecbf2b80 1 Processor -- start 2023-08-25T09:04:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27ecbf2b80 1 -- start start 2023-08-25T09:04:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891da990 0x55f3891da450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891dff40 0x55f3891eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f388fae8c0 con 0x55f3891dff40 2023-08-25T09:04:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3891f3df0 con 0x55f3891e0860 2023-08-25T09:04:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3891d6770 con 0x55f3891da990 2023-08-25T09:04:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891da990 0x55f3891da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27d259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891dff40 0x55f3891eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891da990 0x55f3891da450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:36860/0 (socket says 172.21.15.99:36860) 2023-08-25T09:04:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27d2d9d700 1 -- 172.21.15.99:0/4063447352 learned_addr learned my addr 172.21.15.99:0/4063447352 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.141+0000 7f27d359e700 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27d259c700 1 -- 172.21.15.99:0/4063447352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891da990 msgr2=0x55f3891da450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27d259c700 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891da990 0x55f3891da450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27d259c700 1 -- 172.21.15.99:0/4063447352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 msgr2=0x55f3891e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27d259c700 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27d259c700 1 -- 172.21.15.99:0/4063447352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3891db040 con 0x55f3891dff40 2023-08-25T09:04:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891da990 0x55f3891da450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27d259c700 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891dff40 0x55f3891eb240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f27c800b2e0 tx=0x7f27c8005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4c75a2f84fcd31 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27b9ffb700 1 -- 172.21.15.99:0/4063447352 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c801d070 con 0x55f3891dff40 2023-08-25T09:04:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27b9ffb700 1 -- 172.21.15.99:0/4063447352 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c801e040 con 0x55f3891dff40 2023-08-25T09:04:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.142+0000 7f27b9ffb700 1 -- 172.21.15.99:0/4063447352 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8005040 con 0x55f3891dff40 2023-08-25T09:04:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/4063447352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891dff40 msgr2=0x55f3891eb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891dff40 0x55f3891eb240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f27c800b2e0 tx=0x7f27c8005fb0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/4063447352 shutdown_connections 2023-08-25T09:04:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891da990 0x55f3891da450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/4063447352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891dff40 0x55f3891eb240 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/4063447352 >> 172.21.15.99:0/4063447352 conn(0x55f38914f080 msgr2=0x55f3891ddb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/4063447352 shutdown_connections 2023-08-25T09:04:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/4063447352 wait complete. 2023-08-25T09:04:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.143+0000 7f27ecbf2b80 1 -- start start 2023-08-25T09:04:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27ecbf2b80 1 Processor -- start 2023-08-25T09:04:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891dd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27ecbf2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3891dbe00 con 0x55f3891da990 2023-08-25T09:04:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3891dbf70 con 0x55f3891e0860 2023-08-25T09:04:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27ecbf2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3891dc130 con 0x55f3891dff40 2023-08-25T09:04:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27ecbf2b80 10 client.0 _ll_register_callbacks cb 0x55f389014a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:04:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27d2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34318/0 (socket says 172.21.15.99:34318) 2023-08-25T09:04:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27d2d9d700 1 -- 172.21.15.99:0/3357055378 learned_addr learned my addr 172.21.15.99:0/3357055378 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.144+0000 7f27d259c700 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891dd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27d359e700 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27d2d9d700 1 -- 172.21.15.99:0/3357055378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 msgr2=0x55f3891dd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891dd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27d2d9d700 1 -- 172.21.15.99:0/3357055378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 msgr2=0x55f3891db880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891db880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27d2d9d700 1 -- 172.21.15.99:0/3357055378 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f388f58d80 con 0x55f3891da990 2023-08-25T09:04:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27d2d9d700 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dccc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f27c40958d0 tx=0x7f27c40996a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62b83e66d15dc2e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:04:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27b6ff5700 1 -- 172.21.15.99:0/3357055378 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4098e60 con 0x55f3891da990 2023-08-25T09:04:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/3357055378 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3891cd600 con 0x55f3891da990 2023-08-25T09:04:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27b6ff5700 1 -- 172.21.15.99:0/3357055378 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f27c409cc90 con 0x55f3891da990 2023-08-25T09:04:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.145+0000 7f27b6ff5700 1 -- 172.21.15.99:0/3357055378 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c40b1020 con 0x55f3891da990 2023-08-25T09:04:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27ecbf2b80 10 client.15645 fetch_fsmap learned FSMap version 28 2023-08-25T09:04:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/3357055378 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f27d410e080 con 0x55f3891da990 2023-08-25T09:04:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27b6ff5700 1 -- 172.21.15.99:0/3357055378 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3891cd600 con 0x55f3891da990 2023-08-25T09:04:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27b6ff5700 1 -- 172.21.15.99:0/3357055378 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f27c400ca10 con 0x55f3891da990 2023-08-25T09:04:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27ecbf2b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:04:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27ecbf2b80 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:04:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27ecbf2b80 20 client.15645 populate_metadata read hostname 'smithi099' 2023-08-25T09:04:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f389248910 0x55f38924ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27ecbf2b80 4 client.15645 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:04:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/3357055378 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f389125910 con 0x55f389248910 2023-08-25T09:04:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.146+0000 7f27d259c700 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f389248910 0x55f38924ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.147+0000 7f27d259c700 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f389248910 0x55f38924ad00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca65e886a7a9fc7 server_cookie=de879b0ab78ce6ee in_seq=0 out_seq=0 2023-08-25T09:04:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.147+0000 7f27b6ff5700 10 client.15645 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.180+0000 7f27b6ff5700 1 -- 172.21.15.99:0/3357055378 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f389125910 con 0x55f389248910 2023-08-25T09:04:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.180+0000 7f27b6ff5700 10 client.15645 handle_command_reply: tid=0 2023-08-25T09:04:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.180+0000 7f27ecbf2b80 1 client.15645 shutdown 2023-08-25T09:04:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.180+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/3357055378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 msgr2=0x55f3891dccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.180+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dccc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f27c40958d0 tx=0x7f27c40996a0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/3357055378 shutdown_connections 2023-08-25T09:04:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f389248910 0x55f38924ad00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3891e0860 0x55f3891db880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3891da990 0x55f3891dccc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27b6ff5700 0 client.15645 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 1 --2- 172.21.15.99:0/3357055378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3891dff40 0x55f3891dd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/3357055378 >> 172.21.15.99:0/3357055378 conn(0x55f38914f080 msgr2=0x55f3891dea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/3357055378 shutdown_connections 2023-08-25T09:04:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 1 -- 172.21.15.99:0/3357055378 wait complete. 2023-08-25T09:04:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.181+0000 7f27ecbf2b80 20 client.15645 trim_cache size 0 max 16384 2023-08-25T09:04:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:04:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:04:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:04:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:04:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 -- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f27bc0a00c0 msgr2=0x7f27bc0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f27bc0a00c0 0x7f27bc0a2580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27d419f890 tx=0x7f27cc009920 comp rx=0 tx=0).stop 2023-08-25T09:04:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 -- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 msgr2=0x7f27d419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 0x7f27d419ec00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e9a0 tx=0x7f27c400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 -- 172.21.15.99:0/751793927 shutdown_connections 2023-08-25T09:04:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f27bc0a00c0 0x7f27bc0a2580 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d41068c0 0x7f27d41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27d41008b0 0x7f27d419ec00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 --2- 172.21.15.99:0/751793927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101270 0x7f27d419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 -- 172.21.15.99:0/751793927 >> 172.21.15.99:0/751793927 conn(0x7f27d40fc380 msgr2=0x7f27d410bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 -- 172.21.15.99:0/751793927 shutdown_connections 2023-08-25T09:04:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:24.182+0000 7f27d91ff700 1 -- 172.21.15.99:0/751793927 wait complete. 2023-08-25T09:04:24.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:24 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:04:24.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:24 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:04:24.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:24 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:04:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:24 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:04:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:24 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:04:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:24 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:04:24.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:24 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:04:24.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:24 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:04:24.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:24 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:04:24.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128518 2.17 MB/sec execute 150 sec latency 1988.536 ms 2023-08-25T09:04:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:25 smithi157 ceph-mon[130879]: pgmap v160: 97 pgs: 97 active+clean; 58 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 424 KiB/s wr, 64 op/s 2023-08-25T09:04:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:25 smithi181 ceph-mon[131296]: pgmap v160: 97 pgs: 97 active+clean; 58 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 424 KiB/s wr, 64 op/s 2023-08-25T09:04:25.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:25 smithi099 ceph-mon[97641]: pgmap v160: 97 pgs: 97 active+clean; 58 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 424 KiB/s wr, 64 op/s 2023-08-25T09:04:25.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128518 2.15 MB/sec execute 151 sec latency 2988.659 ms 2023-08-25T09:04:26.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128518 2.14 MB/sec execute 152 sec latency 3988.787 ms 2023-08-25T09:04:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:27 smithi157 ceph-mon[130879]: pgmap v161: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 747 KiB/s wr, 96 op/s 2023-08-25T09:04:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:27 smithi181 ceph-mon[131296]: pgmap v161: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 747 KiB/s wr, 96 op/s 2023-08-25T09:04:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:27 smithi099 ceph-mon[97641]: pgmap v161: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 747 KiB/s wr, 96 op/s 2023-08-25T09:04:27.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128651 2.13 MB/sec execute 153 sec latency 4895.665 ms 2023-08-25T09:04:28.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 130903 2.19 MB/sec execute 154 sec latency 15.752 ms 2023-08-25T09:04:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:29 smithi157 ceph-mon[130879]: pgmap v162: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 589 KiB/s wr, 66 op/s 2023-08-25T09:04:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:29 smithi181 ceph-mon[131296]: pgmap v162: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 589 KiB/s wr, 66 op/s 2023-08-25T09:04:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:29 smithi099 ceph-mon[97641]: pgmap v162: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 589 KiB/s wr, 66 op/s 2023-08-25T09:04:29.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 131943 2.22 MB/sec execute 155 sec latency 158.526 ms 2023-08-25T09:04:30.913 INFO:tasks.workunit.client.0.smithi099.stdout: 1 131943 2.21 MB/sec execute 156 sec latency 1158.667 ms 2023-08-25T09:04:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:31 smithi157 ceph-mon[130879]: pgmap v163: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 589 KiB/s wr, 66 op/s 2023-08-25T09:04:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:31 smithi181 ceph-mon[131296]: pgmap v163: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 589 KiB/s wr, 66 op/s 2023-08-25T09:04:31.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:31 smithi099 ceph-mon[97641]: pgmap v163: 97 pgs: 97 active+clean; 55 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 589 KiB/s wr, 66 op/s 2023-08-25T09:04:31.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 131943 2.19 MB/sec execute 157 sec latency 2158.755 ms 2023-08-25T09:04:32.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 131999 2.18 MB/sec execute 158 sec latency 3065.000 ms 2023-08-25T09:04:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:33 smithi157 ceph-mon[130879]: pgmap v164: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 832 KiB/s wr, 99 op/s 2023-08-25T09:04:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:33 smithi181 ceph-mon[131296]: pgmap v164: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 832 KiB/s wr, 99 op/s 2023-08-25T09:04:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:33 smithi099 ceph-mon[97641]: pgmap v164: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 832 KiB/s wr, 99 op/s 2023-08-25T09:04:33.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 131999 2.16 MB/sec execute 159 sec latency 1049.355 ms 2023-08-25T09:04:34.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 131999 2.15 MB/sec execute 160 sec latency 2049.484 ms 2023-08-25T09:04:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:35 smithi181 ceph-mon[131296]: pgmap v165: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 566 KiB/s wr, 63 op/s 2023-08-25T09:04:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:35 smithi099 ceph-mon[97641]: pgmap v165: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 566 KiB/s wr, 63 op/s 2023-08-25T09:04:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:35 smithi157 ceph-mon[130879]: pgmap v165: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 566 KiB/s wr, 63 op/s 2023-08-25T09:04:35.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 131999 2.14 MB/sec execute 161 sec latency 3049.603 ms 2023-08-25T09:04:36.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 131999 2.12 MB/sec execute 162 sec latency 4049.758 ms 2023-08-25T09:04:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.337+0000 7f3ef2bff700 1 -- 172.21.15.99:0/1973878513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 msgr2=0x7f3eec100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.337+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/1973878513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 0x7f3eec100d20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80098f0 tx=0x7f3ed8009c00 comp rx=0 tx=0).stop 2023-08-25T09:04:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.337+0000 7f3ef2bff700 1 -- 172.21.15.99:0/1973878513 shutdown_connections 2023-08-25T09:04:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.337+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/1973878513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eec101260 0x7f3eec10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.337+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/1973878513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3eec1068b0 0x7f3eec106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.337+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/1973878513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 0x7f3eec100d20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.337+0000 7f3ef2bff700 1 -- 172.21.15.99:0/1973878513 >> 172.21.15.99:0/1973878513 conn(0x7f3eec0fc370 msgr2=0x7f3eec0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.338+0000 7f3ef2bff700 1 -- 172.21.15.99:0/1973878513 shutdown_connections 2023-08-25T09:04:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.338+0000 7f3ef2bff700 1 -- 172.21.15.99:0/1973878513 wait complete. 2023-08-25T09:04:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef2bff700 1 Processor -- start 2023-08-25T09:04:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef2bff700 1 -- start start 2023-08-25T09:04:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef2bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 0x7f3eec072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef2bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eec101260 0x7f3eec073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef2bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3eec1068b0 0x7f3eec1a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef2bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1a50f0 con 0x7f3eec1068b0 2023-08-25T09:04:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef2bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1a5230 con 0x7f3eec1008a0 2023-08-25T09:04:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef2bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1a53a0 con 0x7f3eec101260 2023-08-25T09:04:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef099b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 0x7f3eec072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef119c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3eec1068b0 0x7f3eec1a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef099b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 0x7f3eec072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:39942/0 (socket says 172.21.15.99:39942) 2023-08-25T09:04:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.339+0000 7f3ef099b700 1 -- 172.21.15.99:0/3173034961 learned_addr learned my addr 172.21.15.99:0/3173034961 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3eebfff700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eec101260 0x7f3eec073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ef099b700 1 -- 172.21.15.99:0/3173034961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eec101260 msgr2=0x7f3eec073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ef099b700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eec101260 0x7f3eec073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ef099b700 1 -- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3eec1068b0 msgr2=0x7f3eec1a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ef099b700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3eec1068b0 0x7f3eec1a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ef099b700 1 -- 172.21.15.99:0/3173034961 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4009c40 con 0x7f3eec1008a0 2023-08-25T09:04:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ef099b700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 0x7f3eec072e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000ec10 tx=0x7f3ee000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ee9ffb700 1 -- 172.21.15.99:0/3173034961 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000cc80 con 0x7f3eec1008a0 2023-08-25T09:04:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ee9ffb700 1 -- 172.21.15.99:0/3173034961 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee00044e0 con 0x7f3eec1008a0 2023-08-25T09:04:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ef2bff700 1 -- 172.21.15.99:0/3173034961 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee00095f0 con 0x7f3eec1008a0 2023-08-25T09:04:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.340+0000 7f3ee9ffb700 1 -- 172.21.15.99:0/3173034961 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0010430 con 0x7f3eec1008a0 2023-08-25T09:04:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.341+0000 7f3ef2bff700 1 -- 172.21.15.99:0/3173034961 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed8009520 con 0x7f3eec1008a0 2023-08-25T09:04:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.344+0000 7f3ee9ffb700 1 -- 172.21.15.99:0/3173034961 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f3ee0003f90 con 0x7f3eec1008a0 2023-08-25T09:04:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.345+0000 7f3f064d8b80 1 Processor -- start 2023-08-25T09:04:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3f064d8b80 1 -- start start 2023-08-25T09:04:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3ee9ffb700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3edc0a0140 0x7f3edc0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887dde3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dddd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dd45750 con 0x55887dddd990 2023-08-25T09:04:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dd36720 con 0x55887dde2f40 2023-08-25T09:04:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dd34040 con 0x55887dde3860 2023-08-25T09:04:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3ee9ffb700 1 -- 172.21.15.99:0/3173034961 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3ee0014070 con 0x7f3eec1008a0 2023-08-25T09:04:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3ef119c700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3edc0a0140 0x7f3edc0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.346+0000 7f3ef099b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887dde3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3ef119c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3ef099b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887dde3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:39946/0 (socket says 172.21.15.99:39946) 2023-08-25T09:04:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3ef099b700 1 -- 172.21.15.99:0/3637851007 learned_addr learned my addr 172.21.15.99:0/3637851007 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3eebfff700 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dddd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3eebfff700 1 -- 172.21.15.99:0/3637851007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 msgr2=0x55887dde3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3eebfff700 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887dde3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3eebfff700 1 -- 172.21.15.99:0/3637851007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 msgr2=0x55887dddddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3eebfff700 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3eebfff700 1 -- 172.21.15.99:0/3637851007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887dbaf5d0 con 0x55887dde3860 2023-08-25T09:04:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3ef119c700 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3ef119c700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3edc0a0140 0x7f3edc0a2600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009fd0 tx=0x7f3ee4009490 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3eebfff700 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dddd450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009810 tx=0x7f3ed8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bdda930fbd1ec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.347+0000 7f3ed6ffd700 1 -- 172.21.15.99:0/3637851007 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed800f720 con 0x55887dde3860 2023-08-25T09:04:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3ed6ffd700 1 -- 172.21.15.99:0/3637851007 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ed801d040 con 0x55887dde3860 2023-08-25T09:04:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3ed6ffd700 1 -- 172.21.15.99:0/3637851007 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8005460 con 0x55887dde3860 2023-08-25T09:04:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 -- 172.21.15.99:0/3637851007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 msgr2=0x55887dddd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dddd450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009810 tx=0x7f3ed8005dc0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 -- 172.21.15.99:0/3637851007 shutdown_connections 2023-08-25T09:04:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887dde3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/3637851007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dddd450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 -- 172.21.15.99:0/3637851007 >> 172.21.15.99:0/3637851007 conn(0x55887dd52080 msgr2=0x55887ddd7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 -- 172.21.15.99:0/3637851007 shutdown_connections 2023-08-25T09:04:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.348+0000 7f3f064d8b80 1 -- 172.21.15.99:0/3637851007 wait complete. 2023-08-25T09:04:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.349+0000 7f3f064d8b80 1 -- start start 2023-08-25T09:04:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.349+0000 7f3f064d8b80 1 Processor -- start 2023-08-25T09:04:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887dde96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887dde68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 0x55887dde7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dd2f930 con 0x55887dde2f40 2023-08-25T09:04:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dccdb50 con 0x55887dde3860 2023-08-25T09:04:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dccd850 con 0x55887dddd990 2023-08-25T09:04:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3f064d8b80 10 client.0 _ll_register_callbacks cb 0x55887dc17a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:04:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887dde68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3ef099b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887dde96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887dde68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42304/0 (socket says 172.21.15.99:42304) 2023-08-25T09:04:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3eebfff700 1 -- 172.21.15.99:0/2474064212 learned_addr learned my addr 172.21.15.99:0/2474064212 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3ef119c700 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 0x55887dde7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3eebfff700 1 -- 172.21.15.99:0/2474064212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 msgr2=0x55887dde96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3eebfff700 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887dde96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3eebfff700 1 -- 172.21.15.99:0/2474064212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 msgr2=0x55887dde7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3eebfff700 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 0x55887dde7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.350+0000 7f3eebfff700 1 -- 172.21.15.99:0/2474064212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee401abe0 con 0x55887dde2f40 2023-08-25T09:04:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3eebfff700 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887dde68e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8000c00 tx=0x7f3ed800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f04bc3d43b2f848 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3ecbfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:04:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2474064212 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80052c0 con 0x55887dde2f40 2023-08-25T09:04:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2474064212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55887ddad160 con 0x55887dde2f40 2023-08-25T09:04:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2474064212 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3ed800bb90 con 0x55887dde2f40 2023-08-25T09:04:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2474064212 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ed8005920 con 0x55887dde2f40 2023-08-25T09:04:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3f064d8b80 10 client.15657 fetch_fsmap learned FSMap version 28 2023-08-25T09:04:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2474064212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f3eec10e070 con 0x55887dde2f40 2023-08-25T09:04:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2474064212 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55887ddad160 con 0x55887dde2f40 2023-08-25T09:04:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.351+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2474064212 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3ed8015020 con 0x55887dde2f40 2023-08-25T09:04:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.352+0000 7f3f064d8b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:04:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.352+0000 7f3f064d8b80 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:04:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.352+0000 7f3f064d8b80 20 client.15657 populate_metadata read hostname 'smithi099' 2023-08-25T09:04:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.352+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55887ddf6cd0 0x55887ddf90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.352+0000 7f3f064d8b80 4 client.15657 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:04:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.352+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2474064212 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55887dcd5b40 con 0x55887ddf6cd0 2023-08-25T09:04:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.352+0000 7f3ef099b700 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55887ddf6cd0 0x55887ddf90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.353+0000 7f3ecbfff700 10 client.15657 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.353+0000 7f3ef099b700 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55887ddf6cd0 0x55887ddf90f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cd201552b9ad62 server_cookie=1553a771f8de147b in_seq=0 out_seq=0 2023-08-25T09:04:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.353+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2474064212 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55887dcd5b40 con 0x55887ddf6cd0 2023-08-25T09:04:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.353+0000 7f3ecbfff700 10 client.15657 handle_command_reply: tid=0 2023-08-25T09:04:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.353+0000 7f3f064d8b80 1 client.15657 shutdown 2023-08-25T09:04:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.353+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2474064212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 msgr2=0x55887dde68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.353+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887dde68e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8000c00 tx=0x7f3ed800b790 comp rx=0 tx=0).stop 2023-08-25T09:04:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2474064212 shutdown_connections 2023-08-25T09:04:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55887ddf6cd0 0x55887ddf90f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 0x55887dde7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887dde96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3ecbfff700 0 client.15657 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2474064212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887dde68e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2474064212 >> 172.21.15.99:0/2474064212 conn(0x55887dd52080 msgr2=0x55887ddd2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2474064212 shutdown_connections 2023-08-25T09:04:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2474064212 wait complete. 2023-08-25T09:04:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.354+0000 7f3f064d8b80 20 client.15657 trim_cache size 0 max 16384 2023-08-25T09:04:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.369+0000 7f3f064d8b80 1 Processor -- start 2023-08-25T09:04:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.369+0000 7f3f064d8b80 1 -- start start 2023-08-25T09:04:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.369+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.369+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887ddee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.369+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dde9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.369+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dbb18c0 con 0x55887dddd990 2023-08-25T09:04:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.369+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887ddd4750 con 0x55887dde2f40 2023-08-25T09:04:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.369+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887ddd9f30 con 0x55887dde3860 2023-08-25T09:04:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3ef119c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dde9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3ef099b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3ef119c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dde9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:48200/0 (socket says 172.21.15.99:48200) 2023-08-25T09:04:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3ef119c700 1 -- 172.21.15.99:0/4235261079 learned_addr learned my addr 172.21.15.99:0/4235261079 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3eebfff700 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887ddee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3eebfff700 1 -- 172.21.15.99:0/4235261079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 msgr2=0x55887dde9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3eebfff700 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dde9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3eebfff700 1 -- 172.21.15.99:0/4235261079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 msgr2=0x55887dddd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3eebfff700 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3eebfff700 1 -- 172.21.15.99:0/4235261079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887ddde040 con 0x55887dde2f40 2023-08-25T09:04:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3ef119c700 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dde9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.370+0000 7f3eebfff700 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887ddee240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8003f50 tx=0x7f3ed800f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c7fba76398ee07 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3ed77fe700 1 -- 172.21.15.99:0/4235261079 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed800b800 con 0x55887dde2f40 2023-08-25T09:04:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3ed77fe700 1 -- 172.21.15.99:0/4235261079 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ed801d030 con 0x55887dde2f40 2023-08-25T09:04:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3ed77fe700 1 -- 172.21.15.99:0/4235261079 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80054d0 con 0x55887dde2f40 2023-08-25T09:04:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 -- 172.21.15.99:0/4235261079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 msgr2=0x55887ddee240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887ddee240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8003f50 tx=0x7f3ed800f950 comp rx=0 tx=0).stop 2023-08-25T09:04:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 -- 172.21.15.99:0/4235261079 shutdown_connections 2023-08-25T09:04:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dde3860 0x55887dde9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dddd990 0x55887dddd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/4235261079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde2f40 0x55887ddee240 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 -- 172.21.15.99:0/4235261079 >> 172.21.15.99:0/4235261079 conn(0x55887dd52080 msgr2=0x55887dde09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 -- 172.21.15.99:0/4235261079 shutdown_connections 2023-08-25T09:04:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.371+0000 7f3f064d8b80 1 -- 172.21.15.99:0/4235261079 wait complete. 2023-08-25T09:04:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 1 -- start start 2023-08-25T09:04:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 1 Processor -- start 2023-08-25T09:04:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887ddee440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887ddde880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 0x55887dddeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dddf590 con 0x55887dde2f40 2023-08-25T09:04:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dddf6d0 con 0x55887dde3860 2023-08-25T09:04:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887dddf890 con 0x55887dddd990 2023-08-25T09:04:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.372+0000 7f3f064d8b80 10 client.0 _ll_register_callbacks cb 0x7f3ee40088f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:04:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887ddee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887ddee440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:48206/0 (socket says 172.21.15.99:48206) 2023-08-25T09:04:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 -- 172.21.15.99:0/2481326706 learned_addr learned my addr 172.21.15.99:0/2481326706 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3eebfff700 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887ddde880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef119c700 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 0x55887dddeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 -- 172.21.15.99:0/2481326706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 msgr2=0x55887dddeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 0x55887dddeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 -- 172.21.15.99:0/2481326706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 msgr2=0x55887ddde880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887ddde880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 -- 172.21.15.99:0/2481326706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee4009040 con 0x55887dddd990 2023-08-25T09:04:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3eebfff700 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887ddde880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ef099b700 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887ddee440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0098e20 tx=0x7f3ee000c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9248825d65853f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ecbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:04:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2481326706 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee009b070 con 0x55887dddd990 2023-08-25T09:04:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2481326706 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3ee009cf50 con 0x55887dddd990 2023-08-25T09:04:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.373+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2481326706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55887ddd0600 con 0x55887dddd990 2023-08-25T09:04:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2481326706 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee00a0f30 con 0x55887dddd990 2023-08-25T09:04:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3f064d8b80 10 client.25702 fetch_fsmap learned FSMap version 28 2023-08-25T09:04:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2481326706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f3eec10e070 con 0x55887dddd990 2023-08-25T09:04:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2481326706 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55887ddd0600 con 0x55887dddd990 2023-08-25T09:04:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2481326706 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3ee00b1140 con 0x55887dddd990 2023-08-25T09:04:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3f064d8b80 10 client.25702 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:04:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3f064d8b80 10 client.25702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:04:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3f064d8b80 20 client.25702 populate_metadata read hostname 'smithi099' 2023-08-25T09:04:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55887de4b3e0 0x55887de4d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3f064d8b80 4 client.25702 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:04:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2481326706 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55887dd28910 con 0x55887de4b3e0 2023-08-25T09:04:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.374+0000 7f3eebfff700 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55887de4b3e0 0x55887de4d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.375+0000 7f3eebfff700 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55887de4b3e0 0x55887de4d800 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4067cd53bde33d server_cookie=fb60b602a71bf9db in_seq=0 out_seq=0 2023-08-25T09:04:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.375+0000 7f3ecbfff700 10 client.25702 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3ecbfff700 1 -- 172.21.15.99:0/2481326706 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55887dd28910 con 0x55887de4b3e0 2023-08-25T09:04:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3ecbfff700 10 client.25702 handle_command_reply: tid=0 2023-08-25T09:04:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 client.25702 shutdown 2023-08-25T09:04:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2481326706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 msgr2=0x55887ddee440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887ddee440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0098e20 tx=0x7f3ee000c790 comp rx=0 tx=0).stop 2023-08-25T09:04:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2481326706 shutdown_connections 2023-08-25T09:04:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55887de4b3e0 0x55887de4d800 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55887dde3860 0x55887dddeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887dddd990 0x55887ddee440 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3ecbfff700 0 client.25702 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 --2- 172.21.15.99:0/2481326706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55887dde2f40 0x55887ddde880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2481326706 >> 172.21.15.99:0/2481326706 conn(0x55887dd52080 msgr2=0x55887dde6180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2481326706 shutdown_connections 2023-08-25T09:04:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.405+0000 7f3f064d8b80 1 -- 172.21.15.99:0/2481326706 wait complete. 2023-08-25T09:04:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3f064d8b80 20 client.25702 trim_cache size 0 max 16384 2023-08-25T09:04:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:04:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:04:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:04:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:04:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 -- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3edc0a0140 msgr2=0x7f3edc0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3edc0a0140 0x7f3edc0a2600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009fd0 tx=0x7f3ee4009490 comp rx=0 tx=0).stop 2023-08-25T09:04:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 -- 172.21.15.99:0/3173034961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 msgr2=0x7f3eec072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 0x7f3eec072e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000ec10 tx=0x7f3ee000ef20 comp rx=0 tx=0).stop 2023-08-25T09:04:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 -- 172.21.15.99:0/3173034961 shutdown_connections 2023-08-25T09:04:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f3edc0a0140 0x7f3edc0a2600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3eec1068b0 0x7f3eec1a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec1008a0 0x7f3eec072e00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 --2- 172.21.15.99:0/3173034961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eec101260 0x7f3eec073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 -- 172.21.15.99:0/3173034961 >> 172.21.15.99:0/3173034961 conn(0x7f3eec0fc370 msgr2=0x7f3eec0fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 -- 172.21.15.99:0/3173034961 shutdown_connections 2023-08-25T09:04:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:37.406+0000 7f3ef2bff700 1 -- 172.21.15.99:0/3173034961 wait complete. 2023-08-25T09:04:37.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:37 smithi099 ceph-mon[97641]: pgmap v166: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 849 KiB/s wr, 81 op/s 2023-08-25T09:04:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:37 smithi157 ceph-mon[130879]: pgmap v166: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 849 KiB/s wr, 81 op/s 2023-08-25T09:04:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:37 smithi181 ceph-mon[131296]: pgmap v166: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 849 KiB/s wr, 81 op/s 2023-08-25T09:04:37.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 132150 2.11 MB/sec execute 163 sec latency 4953.130 ms 2023-08-25T09:04:38.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 134286 2.17 MB/sec execute 164 sec latency 13.933 ms 2023-08-25T09:04:39.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:39 smithi099 ceph-mon[97641]: pgmap v167: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 527 KiB/s wr, 49 op/s 2023-08-25T09:04:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:39 smithi157 ceph-mon[130879]: pgmap v167: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 527 KiB/s wr, 49 op/s 2023-08-25T09:04:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:39 smithi181 ceph-mon[131296]: pgmap v167: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 527 KiB/s wr, 49 op/s 2023-08-25T09:04:39.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 135626 2.20 MB/sec execute 165 sec latency 17.791 ms 2023-08-25T09:04:40.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 137410 2.24 MB/sec execute 166 sec latency 18.714 ms 2023-08-25T09:04:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:41 smithi157 ceph-mon[130879]: pgmap v168: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 527 KiB/s wr, 49 op/s 2023-08-25T09:04:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:41 smithi181 ceph-mon[131296]: pgmap v168: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 527 KiB/s wr, 49 op/s 2023-08-25T09:04:41.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:41 smithi099 ceph-mon[97641]: pgmap v168: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 527 KiB/s wr, 49 op/s 2023-08-25T09:04:41.914 INFO:tasks.workunit.client.0.smithi099.stdout: 1 138874 2.29 MB/sec execute 167 sec latency 36.004 ms 2023-08-25T09:04:42.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 139158 2.28 MB/sec execute 168 sec latency 779.194 ms 2023-08-25T09:04:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:43 smithi157 ceph-mon[130879]: pgmap v169: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 KiB/s wr, 114 op/s 2023-08-25T09:04:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:43 smithi181 ceph-mon[131296]: pgmap v169: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 KiB/s wr, 114 op/s 2023-08-25T09:04:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:43 smithi099 ceph-mon[97641]: pgmap v169: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 KiB/s wr, 114 op/s 2023-08-25T09:04:43.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 139158 2.26 MB/sec execute 169 sec latency 1007.955 ms 2023-08-25T09:04:44.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 139158 2.25 MB/sec execute 170 sec latency 2008.133 ms 2023-08-25T09:04:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:45 smithi181 ceph-mon[131296]: pgmap v170: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 743 KiB/s wr, 81 op/s 2023-08-25T09:04:45.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:45 smithi099 ceph-mon[97641]: pgmap v170: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 743 KiB/s wr, 81 op/s 2023-08-25T09:04:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:45 smithi157 ceph-mon[130879]: pgmap v170: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 743 KiB/s wr, 81 op/s 2023-08-25T09:04:45.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 139158 2.24 MB/sec execute 171 sec latency 3008.272 ms 2023-08-25T09:04:46.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 139158 2.22 MB/sec execute 172 sec latency 4008.379 ms 2023-08-25T09:04:47.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:47 smithi157 ceph-mon[130879]: pgmap v171: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2023-08-25T09:04:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:47 smithi181 ceph-mon[131296]: pgmap v171: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2023-08-25T09:04:47.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:47 smithi099 ceph-mon[97641]: pgmap v171: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2023-08-25T09:04:47.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 139311 2.21 MB/sec execute 173 sec latency 4912.570 ms 2023-08-25T09:04:48.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 141556 2.27 MB/sec execute 174 sec latency 18.089 ms 2023-08-25T09:04:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:49 smithi157 ceph-mon[130879]: pgmap v172: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-08-25T09:04:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:49 smithi181 ceph-mon[131296]: pgmap v172: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-08-25T09:04:49.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:49 smithi099 ceph-mon[97641]: pgmap v172: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-08-25T09:04:49.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142577 2.29 MB/sec execute 175 sec latency 124.428 ms 2023-08-25T09:04:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.553+0000 7f7570bff700 1 -- 172.21.15.99:0/1903374770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c101270 msgr2=0x7f756c10d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.553+0000 7f7570bff700 1 --2- 172.21.15.99:0/1903374770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c101270 0x7f756c10d780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7564009f90 tx=0x7f756400b330 comp rx=0 tx=0).stop 2023-08-25T09:04:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.554+0000 7f7570bff700 1 -- 172.21.15.99:0/1903374770 shutdown_connections 2023-08-25T09:04:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.554+0000 7f7570bff700 1 --2- 172.21.15.99:0/1903374770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c101270 0x7f756c10d780 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.554+0000 7f7570bff700 1 --2- 172.21.15.99:0/1903374770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c1068c0 0x7f756c106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.554+0000 7f7570bff700 1 --2- 172.21.15.99:0/1903374770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c1008b0 0x7f756c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.554+0000 7f7570bff700 1 -- 172.21.15.99:0/1903374770 >> 172.21.15.99:0/1903374770 conn(0x7f756c0fc380 msgr2=0x7f756c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.554+0000 7f7570bff700 1 -- 172.21.15.99:0/1903374770 shutdown_connections 2023-08-25T09:04:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.554+0000 7f7570bff700 1 -- 172.21.15.99:0/1903374770 wait complete. 2023-08-25T09:04:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f7570bff700 1 Processor -- start 2023-08-25T09:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f7570bff700 1 -- start start 2023-08-25T09:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f7570bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c1008b0 0x7f756c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f7570bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c101270 0x7f756c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f7570bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c1068c0 0x7f756c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f7570bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c199020 con 0x7f756c1008b0 2023-08-25T09:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f7570bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c199190 con 0x7f756c1068c0 2023-08-25T09:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f7570bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c199350 con 0x7f756c101270 2023-08-25T09:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f756a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c1008b0 0x7f756c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f756a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c1008b0 0x7f756c19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49336/0 (socket says 172.21.15.99:49336) 2023-08-25T09:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.555+0000 7f756a59c700 1 -- 172.21.15.99:0/3790894386 learned_addr learned my addr 172.21.15.99:0/3790894386 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f7569d9b700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c101270 0x7f756c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f756ad9d700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c1068c0 0x7f756c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f756a59c700 1 -- 172.21.15.99:0/3790894386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c101270 msgr2=0x7f756c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f756a59c700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c101270 0x7f756c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f756a59c700 1 -- 172.21.15.99:0/3790894386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c1068c0 msgr2=0x7f756c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f756a59c700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c1068c0 0x7f756c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f756a59c700 1 -- 172.21.15.99:0/3790894386 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600095f0 con 0x7f756c1008b0 2023-08-25T09:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f756ad9d700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c1068c0 0x7f756c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f756a59c700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c1008b0 0x7f756c19eb50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f755c00e9a0 tx=0x7f755c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f755b7fe700 1 -- 172.21.15.99:0/3790894386 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c009790 con 0x7f756c1008b0 2023-08-25T09:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f7570bff700 1 -- 172.21.15.99:0/3790894386 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7564009c40 con 0x7f756c1008b0 2023-08-25T09:04:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f755b7fe700 1 -- 172.21.15.99:0/3790894386 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f755c00cbd0 con 0x7f756c1008b0 2023-08-25T09:04:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f755b7fe700 1 -- 172.21.15.99:0/3790894386 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c018680 con 0x7f756c1008b0 2023-08-25T09:04:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.556+0000 7f7570bff700 1 -- 172.21.15.99:0/3790894386 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c1a5960 con 0x7f756c1008b0 2023-08-25T09:04:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.560+0000 7f755b7fe700 1 -- 172.21.15.99:0/3790894386 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f755c00cd40 con 0x7f756c1008b0 2023-08-25T09:04:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.561+0000 7f75845f7b80 1 Processor -- start 2023-08-25T09:04:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.561+0000 7f75845f7b80 1 -- start start 2023-08-25T09:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.561+0000 7f755b7fe700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75540a00c0 0x7f75540a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e96675320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e9666f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9666fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f75845f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e965d7750 con 0x563e96674f40 2023-08-25T09:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f75845f7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e965c8720 con 0x563e9666f990 2023-08-25T09:04:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f75845f7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e965c6040 con 0x563e96675860 2023-08-25T09:04:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f755b7fe700 1 -- 172.21.15.99:0/3790894386 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f755c014070 con 0x7f756c1008b0 2023-08-25T09:04:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756ad9d700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75540a00c0 0x7f75540a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e96675320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e96675320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49340/0 (socket says 172.21.15.99:49340) 2023-08-25T09:04:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756a59c700 1 -- 172.21.15.99:0/1173555064 learned_addr learned my addr 172.21.15.99:0/1173555064 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f7569d9b700 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e9666f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756ad9d700 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9666fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756a59c700 1 -- 172.21.15.99:0/1173555064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 msgr2=0x563e9666f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756a59c700 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e9666f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756a59c700 1 -- 172.21.15.99:0/1173555064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 msgr2=0x563e9666fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756a59c700 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9666fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.562+0000 7f756a59c700 1 -- 172.21.15.99:0/1173555064 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e964415d0 con 0x563e96674f40 2023-08-25T09:04:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.563+0000 7f756a59c700 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e96675320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f755c0954b0 tx=0x7f755c098af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31da419b98aa3fe2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.563+0000 7f756ad9d700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75540a00c0 0x7f75540a2580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7564009790 tx=0x7f7564004c20 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.563+0000 7f7558ff9700 1 -- 172.21.15.99:0/1173555064 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c098fb0 con 0x563e96674f40 2023-08-25T09:04:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.563+0000 7f7558ff9700 1 -- 172.21.15.99:0/1173555064 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f755c09cde0 con 0x563e96674f40 2023-08-25T09:04:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.563+0000 7f7558ff9700 1 -- 172.21.15.99:0/1173555064 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c098700 con 0x563e96674f40 2023-08-25T09:04:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.563+0000 7f75845f7b80 1 -- 172.21.15.99:0/1173555064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 msgr2=0x563e96675320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.563+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e96675320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f755c0954b0 tx=0x7f755c098af0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 -- 172.21.15.99:0/1173555064 shutdown_connections 2023-08-25T09:04:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9666fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e96675320 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1173555064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e9666f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 -- 172.21.15.99:0/1173555064 >> 172.21.15.99:0/1173555064 conn(0x563e965e4080 msgr2=0x563e96669d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 -- 172.21.15.99:0/1173555064 shutdown_connections 2023-08-25T09:04:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 -- 172.21.15.99:0/1173555064 wait complete. 2023-08-25T09:04:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 -- start start 2023-08-25T09:04:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.564+0000 7f75845f7b80 1 Processor -- start 2023-08-25T09:04:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9667b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e966788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e96679010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f75845f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e965c1930 con 0x563e96674f40 2023-08-25T09:04:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f75845f7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9655fb50 con 0x563e9666f990 2023-08-25T09:04:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f75845f7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9655f850 con 0x563e96675860 2023-08-25T09:04:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f75845f7b80 10 client.0 _ll_register_callbacks cb 0x563e964a9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:04:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f7569d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e966788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f756a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9667b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f756ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e96679010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f7569d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e966788d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49348/0 (socket says 172.21.15.99:49348) 2023-08-25T09:04:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f7569d9b700 1 -- 172.21.15.99:0/1580810383 learned_addr learned my addr 172.21.15.99:0/1580810383 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.565+0000 7f756a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9667b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:56204/0 (socket says 172.21.15.99:56204) 2023-08-25T09:04:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f756ad9d700 1 -- 172.21.15.99:0/1580810383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 msgr2=0x563e9667b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f756ad9d700 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9667b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f756ad9d700 1 -- 172.21.15.99:0/1580810383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 msgr2=0x563e966788d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f756ad9d700 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e966788d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f756ad9d700 1 -- 172.21.15.99:0/1580810383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c1a5960 con 0x563e96675860 2023-08-25T09:04:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f756a59c700 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9667b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:04:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f756ad9d700 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e96679010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7564009550 tx=0x7f756400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea98df400a73539 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f7551ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:04:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f7551ffb700 1 -- 172.21.15.99:0/1580810383 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564030070 con 0x563e96675860 2023-08-25T09:04:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f75845f7b80 1 -- 172.21.15.99:0/1580810383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e9663f160 con 0x563e96675860 2023-08-25T09:04:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.566+0000 7f7551ffb700 1 -- 172.21.15.99:0/1580810383 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7564007d50 con 0x563e96675860 2023-08-25T09:04:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f7551ffb700 1 -- 172.21.15.99:0/1580810383 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f756400e070 con 0x563e96675860 2023-08-25T09:04:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f75845f7b80 10 client.25714 fetch_fsmap learned FSMap version 28 2023-08-25T09:04:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f75845f7b80 1 -- 172.21.15.99:0/1580810383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563e963edd80 con 0x563e96675860 2023-08-25T09:04:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f7551ffb700 1 -- 172.21.15.99:0/1580810383 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e9663f160 con 0x563e96675860 2023-08-25T09:04:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f7551ffb700 1 -- 172.21.15.99:0/1580810383 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7564033020 con 0x563e96675860 2023-08-25T09:04:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f75845f7b80 10 client.25714 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:04:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f75845f7b80 10 client.25714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:04:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f75845f7b80 20 client.25714 populate_metadata read hostname 'smithi099' 2023-08-25T09:04:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563e96689190 0x563e9668b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f75845f7b80 4 client.25714 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:04:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.567+0000 7f75845f7b80 1 -- 172.21.15.99:0/1580810383 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e96567b40 con 0x563e96689190 2023-08-25T09:04:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.568+0000 7f756a59c700 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563e96689190 0x563e9668b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.568+0000 7f7551ffb700 10 client.25714 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.568+0000 7f756a59c700 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563e96689190 0x563e9668b5b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f7f10ca8065c5a server_cookie=9d4cc36fc0353b45 in_seq=0 out_seq=0 2023-08-25T09:04:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f7551ffb700 1 -- 172.21.15.99:0/1580810383 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e96567b40 con 0x563e96689190 2023-08-25T09:04:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f7551ffb700 10 client.25714 handle_command_reply: tid=0 2023-08-25T09:04:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 client.25714 shutdown 2023-08-25T09:04:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 -- 172.21.15.99:0/1580810383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 msgr2=0x563e96679010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e96679010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7564009550 tx=0x7f756400b330 comp rx=0 tx=0).stop 2023-08-25T09:04:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 -- 172.21.15.99:0/1580810383 shutdown_connections 2023-08-25T09:04:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e96675860 0x563e96679010 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563e96689190 0x563e9668b5b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9666f990 0x563e9667b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1580810383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96674f40 0x563e966788d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f7551ffb700 0 client.25714 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 -- 172.21.15.99:0/1580810383 >> 172.21.15.99:0/1580810383 conn(0x563e965e4080 msgr2=0x563e96664820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 -- 172.21.15.99:0/1580810383 shutdown_connections 2023-08-25T09:04:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 1 -- 172.21.15.99:0/1580810383 wait complete. 2023-08-25T09:04:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.569+0000 7f75845f7b80 20 client.25714 trim_cache size 0 max 16384 2023-08-25T09:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.584+0000 7f75845f7b80 1 Processor -- start 2023-08-25T09:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.584+0000 7f75845f7b80 1 -- start start 2023-08-25T09:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.584+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e9666f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.584+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 0x563e966801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.584+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e9667b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.584+0000 7f75845f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e964438c0 con 0x563e96675860 2023-08-25T09:04:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.584+0000 7f75845f7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e96680720 con 0x563e96674f40 2023-08-25T09:04:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.584+0000 7f75845f7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e96673550 con 0x563e9666f990 2023-08-25T09:04:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e9667b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e9666f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e9667b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49350/0 (socket says 172.21.15.99:49350) 2023-08-25T09:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 -- 172.21.15.99:0/1045248199 learned_addr learned my addr 172.21.15.99:0/1045248199 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f7569d9b700 1 --2- 172.21.15.99:0/1045248199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 0x563e966801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 -- 172.21.15.99:0/1045248199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 msgr2=0x563e9666f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 --2- 172.21.15.99:0/1045248199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e9666f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 -- 172.21.15.99:0/1045248199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 msgr2=0x563e966801e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 --2- 172.21.15.99:0/1045248199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 0x563e966801e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 -- 172.21.15.99:0/1045248199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e963d6c00 con 0x563e96675860 2023-08-25T09:04:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f756ad9d700 1 --2- 172.21.15.99:0/1045248199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e9667b1f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f756400bf00 tx=0x7f7564009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c49fc12a9610593 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f75597fa700 1 -- 172.21.15.99:0/1045248199 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564030070 con 0x563e96675860 2023-08-25T09:04:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f75597fa700 1 -- 172.21.15.99:0/1045248199 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7564007d50 con 0x563e96675860 2023-08-25T09:04:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.585+0000 7f75597fa700 1 -- 172.21.15.99:0/1045248199 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756400e070 con 0x563e96675860 2023-08-25T09:04:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 -- 172.21.15.99:0/1045248199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 msgr2=0x563e9667b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1045248199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e9667b1f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f756400bf00 tx=0x7f7564009920 comp rx=0 tx=0).stop 2023-08-25T09:04:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 -- 172.21.15.99:0/1045248199 shutdown_connections 2023-08-25T09:04:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1045248199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e9667b1f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1045248199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e9666f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 --2- 172.21.15.99:0/1045248199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 0x563e966801e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 -- 172.21.15.99:0/1045248199 >> 172.21.15.99:0/1045248199 conn(0x563e965e4080 msgr2=0x563e96685400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 -- 172.21.15.99:0/1045248199 shutdown_connections 2023-08-25T09:04:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 -- 172.21.15.99:0/1045248199 wait complete. 2023-08-25T09:04:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.586+0000 7f75845f7b80 1 -- start start 2023-08-25T09:04:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f75845f7b80 1 Processor -- start 2023-08-25T09:04:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e96676e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 0x563e96677370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f75845f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e96677ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f75845f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e966781c0 con 0x563e96675860 2023-08-25T09:04:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f75845f7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e96678330 con 0x563e96674f40 2023-08-25T09:04:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f75845f7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e966784f0 con 0x563e9666f990 2023-08-25T09:04:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f75845f7b80 10 client.0 _ll_register_callbacks cb 0x7f756400bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f756a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e96676e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f756a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e96676e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57588/0 (socket says 172.21.15.99:57588) 2023-08-25T09:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f756a59c700 1 -- 172.21.15.99:0/2847006755 learned_addr learned my addr 172.21.15.99:0/2847006755 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f756ad9d700 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e96677ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.587+0000 7f7569d9b700 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 0x563e96677370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f756a59c700 1 -- 172.21.15.99:0/2847006755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 msgr2=0x563e96677370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f756a59c700 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 0x563e96677370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f756a59c700 1 -- 172.21.15.99:0/2847006755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 msgr2=0x563e96677ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f756a59c700 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e96677ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f756a59c700 1 -- 172.21.15.99:0/2847006755 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e963edd80 con 0x563e9666f990 2023-08-25T09:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f756a59c700 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e96676e30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f755c000c00 tx=0x7f755c098e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a20b44ed1b89e6ea server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f7551ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f7551ffb700 1 -- 172.21.15.99:0/2847006755 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c099410 con 0x563e9666f990 2023-08-25T09:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f75845f7b80 1 -- 172.21.15.99:0/2847006755 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e96662600 con 0x563e9666f990 2023-08-25T09:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f7551ffb700 1 -- 172.21.15.99:0/2847006755 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f755c098fb0 con 0x563e9666f990 2023-08-25T09:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f7551ffb700 1 -- 172.21.15.99:0/2847006755 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755c0a0dc0 con 0x563e9666f990 2023-08-25T09:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f75845f7b80 10 client.25720 fetch_fsmap learned FSMap version 28 2023-08-25T09:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.588+0000 7f75845f7b80 1 -- 172.21.15.99:0/2847006755 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563e964141f0 con 0x563e9666f990 2023-08-25T09:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f7551ffb700 1 -- 172.21.15.99:0/2847006755 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e96662600 con 0x563e9666f990 2023-08-25T09:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f7551ffb700 1 -- 172.21.15.99:0/2847006755 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f755c093c50 con 0x563e9666f990 2023-08-25T09:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f75845f7b80 10 client.25720 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f75845f7b80 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f75845f7b80 20 client.25720 populate_metadata read hostname 'smithi099' 2023-08-25T09:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f75845f7b80 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563e966dda50 0x563e966dfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f75845f7b80 4 client.25720 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f75845f7b80 1 -- 172.21.15.99:0/2847006755 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e965ba910 con 0x563e966dda50 2023-08-25T09:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.589+0000 7f7569d9b700 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563e966dda50 0x563e966dfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.590+0000 7f7569d9b700 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563e966dda50 0x563e966dfe70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bcddbc9e0fa4490 server_cookie=e7cf11a0f7e43b74 in_seq=0 out_seq=0 2023-08-25T09:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.590+0000 7f7551ffb700 10 client.25720 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.632+0000 7f7551ffb700 1 -- 172.21.15.99:0/2847006755 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e965ba910 con 0x563e966dda50 2023-08-25T09:04:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.632+0000 7f7551ffb700 10 client.25720 handle_command_reply: tid=0 2023-08-25T09:04:50.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.632+0000 7f75845f7b80 1 client.25720 shutdown 2023-08-25T09:04:50.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 -- 172.21.15.99:0/2847006755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 msgr2=0x563e96676e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e96676e30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f755c000c00 tx=0x7f755c098e30 comp rx=0 tx=0).stop 2023-08-25T09:04:50.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 -- 172.21.15.99:0/2847006755 shutdown_connections 2023-08-25T09:04:50.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e96675860 0x563e96677ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9666f990 0x563e96676e30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x563e966dda50 0x563e966dfe70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 --2- 172.21.15.99:0/2847006755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e96674f40 0x563e96677370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 -- 172.21.15.99:0/2847006755 >> 172.21.15.99:0/2847006755 conn(0x563e965e4080 msgr2=0x563e96675c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f7551ffb700 0 client.25720 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:04:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 -- 172.21.15.99:0/2847006755 shutdown_connections 2023-08-25T09:04:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 1 -- 172.21.15.99:0/2847006755 wait complete. 2023-08-25T09:04:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.633+0000 7f75845f7b80 20 client.25720 trim_cache size 0 max 16384 2023-08-25T09:04:50.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:04:50.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:04:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:04:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:04:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 -- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75540a00c0 msgr2=0x7f75540a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75540a00c0 0x7f75540a2580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7564009790 tx=0x7f7564004c20 comp rx=0 tx=0).stop 2023-08-25T09:04:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 -- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c1008b0 msgr2=0x7f756c19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:04:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c1008b0 0x7f756c19eb50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f755c00e9a0 tx=0x7f755c00ecb0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 -- 172.21.15.99:0/3790894386 shutdown_connections 2023-08-25T09:04:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75540a00c0 0x7f75540a2580 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c1068c0 0x7f756c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f756c1008b0 0x7f756c19eb50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 --2- 172.21.15.99:0/3790894386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c101270 0x7f756c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:04:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 -- 172.21.15.99:0/3790894386 >> 172.21.15.99:0/3790894386 conn(0x7f756c0fc380 msgr2=0x7f756c10bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:04:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 -- 172.21.15.99:0/3790894386 shutdown_connections 2023-08-25T09:04:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:04:50.634+0000 7f7570bff700 1 -- 172.21.15.99:0/3790894386 wait complete. 2023-08-25T09:04:50.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142577 2.28 MB/sec execute 176 sec latency 1124.554 ms 2023-08-25T09:04:51.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:51 smithi157 ceph-mon[130879]: pgmap v173: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-08-25T09:04:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:51 smithi181 ceph-mon[131296]: pgmap v173: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-08-25T09:04:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:51 smithi099 ceph-mon[97641]: pgmap v173: 97 pgs: 97 active+clean; 59 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-08-25T09:04:51.915 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142577 2.27 MB/sec execute 177 sec latency 2124.690 ms 2023-08-25T09:04:52.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142701 2.26 MB/sec execute 178 sec latency 3026.979 ms 2023-08-25T09:04:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:53 smithi157 ceph-mon[130879]: pgmap v174: 97 pgs: 97 active+clean; 58 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 158 op/s 2023-08-25T09:04:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:53 smithi181 ceph-mon[131296]: pgmap v174: 97 pgs: 97 active+clean; 58 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 158 op/s 2023-08-25T09:04:53.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:53 smithi099 ceph-mon[97641]: pgmap v174: 97 pgs: 97 active+clean; 58 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 158 op/s 2023-08-25T09:04:53.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142701 2.24 MB/sec execute 179 sec latency 1003.439 ms 2023-08-25T09:04:54.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142701 2.23 MB/sec execute 180 sec latency 2003.508 ms 2023-08-25T09:04:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:55 smithi181 ceph-mon[131296]: pgmap v175: 97 pgs: 97 active+clean; 58 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 966 KiB/s wr, 93 op/s 2023-08-25T09:04:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:55 smithi099 ceph-mon[97641]: pgmap v175: 97 pgs: 97 active+clean; 58 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 966 KiB/s wr, 93 op/s 2023-08-25T09:04:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:55 smithi157 ceph-mon[130879]: pgmap v175: 97 pgs: 97 active+clean; 58 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 966 KiB/s wr, 93 op/s 2023-08-25T09:04:55.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142701 2.22 MB/sec execute 181 sec latency 3003.629 ms 2023-08-25T09:04:56.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142701 2.21 MB/sec execute 182 sec latency 4003.740 ms 2023-08-25T09:04:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:57 smithi157 ceph-mon[130879]: pgmap v176: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-08-25T09:04:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:57 smithi181 ceph-mon[131296]: pgmap v176: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-08-25T09:04:57.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:57 smithi099 ceph-mon[97641]: pgmap v176: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-08-25T09:04:57.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142845 2.20 MB/sec execute 183 sec latency 4907.357 ms 2023-08-25T09:04:58.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 144838 2.25 MB/sec execute 184 sec latency 86.654 ms 2023-08-25T09:04:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:04:59 smithi157 ceph-mon[130879]: pgmap v177: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 583 KiB/s wr, 76 op/s 2023-08-25T09:04:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:04:59 smithi181 ceph-mon[131296]: pgmap v177: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 583 KiB/s wr, 76 op/s 2023-08-25T09:04:59.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:04:59 smithi099 ceph-mon[97641]: pgmap v177: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 583 KiB/s wr, 76 op/s 2023-08-25T09:04:59.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 146126 2.27 MB/sec execute 185 sec latency 107.716 ms 2023-08-25T09:05:00.916 INFO:tasks.workunit.client.0.smithi099.stdout: 1 146182 2.26 MB/sec execute 186 sec latency 693.642 ms 2023-08-25T09:05:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:01 smithi157 ceph-mon[130879]: pgmap v178: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 583 KiB/s wr, 76 op/s 2023-08-25T09:05:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:01 smithi181 ceph-mon[131296]: pgmap v178: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 583 KiB/s wr, 76 op/s 2023-08-25T09:05:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:01 smithi099 ceph-mon[97641]: pgmap v178: 97 pgs: 97 active+clean; 58 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 583 KiB/s wr, 76 op/s 2023-08-25T09:05:01.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 146182 2.25 MB/sec execute 187 sec latency 1693.770 ms 2023-08-25T09:05:02.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 146267 2.24 MB/sec execute 188 sec latency 2596.534 ms 2023-08-25T09:05:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:03 smithi157 ceph-mon[130879]: pgmap v179: 97 pgs: 97 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 885 KiB/s wr, 123 op/s 2023-08-25T09:05:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:03 smithi181 ceph-mon[131296]: pgmap v179: 97 pgs: 97 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 885 KiB/s wr, 123 op/s 2023-08-25T09:05:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 -- 172.21.15.99:0/2079342885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01068b0 msgr2=0x7f81a0106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2079342885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01068b0 0x7f81a0106c90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8194009990 tx=0x7f8194009ca0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 -- 172.21.15.99:0/2079342885 shutdown_connections 2023-08-25T09:05:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2079342885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81a0101260 0x7f81a010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2079342885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01068b0 0x7f81a0106c90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2079342885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a01008a0 0x7f81a0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 -- 172.21.15.99:0/2079342885 >> 172.21.15.99:0/2079342885 conn(0x7f81a00fc370 msgr2=0x7f81a00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 -- 172.21.15.99:0/2079342885 shutdown_connections 2023-08-25T09:05:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.788+0000 7f81a6bff700 1 -- 172.21.15.99:0/2079342885 wait complete. 2023-08-25T09:05:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.789+0000 7f81a6bff700 1 Processor -- start 2023-08-25T09:05:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a6bff700 1 -- start start 2023-08-25T09:05:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a6bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01008a0 0x7f81a019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a6bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81a0101260 0x7f81a019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a6bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a01068b0 0x7f81a0196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a6bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0196df0 con 0x7f81a0101260 2023-08-25T09:05:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a6bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0196f60 con 0x7f81a01068b0 2023-08-25T09:05:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a6bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0197120 con 0x7f81a01008a0 2023-08-25T09:05:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01008a0 0x7f81a019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01008a0 0x7f81a019c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:40334/0 (socket says 172.21.15.99:40334) 2023-08-25T09:05:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f819ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81a0101260 0x7f81a019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a499b700 1 -- 172.21.15.99:0/2759812140 learned_addr learned my addr 172.21.15.99:0/2759812140 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.790+0000 7f81a519c700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a01068b0 0x7f81a0196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a519c700 1 -- 172.21.15.99:0/2759812140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01008a0 msgr2=0x7f81a019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a519c700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01008a0 0x7f81a019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a519c700 1 -- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81a0101260 msgr2=0x7f81a019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a519c700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81a0101260 0x7f81a019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a519c700 1 -- 172.21.15.99:0/2759812140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81940095f0 con 0x7f81a01068b0 2023-08-25T09:05:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a499b700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01008a0 0x7f81a019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:05:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a519c700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a01068b0 0x7f81a0196780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f819800efc0 tx=0x7f819800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f819dffb700 1 -- 172.21.15.99:0/2759812140 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198018070 con 0x7f81a01068b0 2023-08-25T09:05:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a6bff700 1 -- 172.21.15.99:0/2759812140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a01973a0 con 0x7f81a01068b0 2023-08-25T09:05:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f819dffb700 1 -- 172.21.15.99:0/2759812140 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f819800f040 con 0x7f81a01068b0 2023-08-25T09:05:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f819dffb700 1 -- 172.21.15.99:0/2759812140 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819800cd40 con 0x7f81a01068b0 2023-08-25T09:05:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.791+0000 7f81a6bff700 1 -- 172.21.15.99:0/2759812140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a01a3910 con 0x7f81a01068b0 2023-08-25T09:05:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.796+0000 7f819dffb700 1 -- 172.21.15.99:0/2759812140 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f8198003a00 con 0x7f81a01068b0 2023-08-25T09:05:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81ba4beb80 1 Processor -- start 2023-08-25T09:05:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81ba4beb80 1 -- start start 2023-08-25T09:05:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f69450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f69990 0x55f6a7f69dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7ed1750 con 0x55f6a7f69990 2023-08-25T09:05:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7ec2720 con 0x55f6a7f6f860 2023-08-25T09:05:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7ec0040 con 0x55f6a7f6ef40 2023-08-25T09:05:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81a499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f69450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f819ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81a499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f69450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:34766/0 (socket says 172.21.15.99:34766) 2023-08-25T09:05:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.797+0000 7f81a499b700 1 -- 172.21.15.99:0/2679476702 learned_addr learned my addr 172.21.15.99:0/2679476702 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f819ffff700 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f69990 0x55f6a7f69dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f81a499b700 1 -- 172.21.15.99:0/2679476702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 msgr2=0x55f6a7f6f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f81a499b700 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f819dffb700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f81900a00c0 0x7f81900a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f81a499b700 1 -- 172.21.15.99:0/2679476702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f69990 msgr2=0x55f6a7f69dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f81a499b700 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f69990 0x55f6a7f69dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f81a499b700 1 -- 172.21.15.99:0/2679476702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6a7d3b5d0 con 0x55f6a7f6f860 2023-08-25T09:05:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f819ffff700 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f69990 0x55f6a7f69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f819dffb700 1 -- 172.21.15.99:0/2759812140 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f81980cb220 con 0x7f81a01068b0 2023-08-25T09:05:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f81a519c700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f81900a00c0 0x7f81900a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.798+0000 7f81a499b700 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f69450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f819400b640 tx=0x7f819400b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cb8b92d5429355c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f818affd700 1 -- 172.21.15.99:0/2679476702 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194024070 con 0x55f6a7f6f860 2023-08-25T09:05:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f818affd700 1 -- 172.21.15.99:0/2679476702 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8194005500 con 0x55f6a7f6f860 2023-08-25T09:05:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f818affd700 1 -- 172.21.15.99:0/2679476702 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194004d50 con 0x55f6a7f6f860 2023-08-25T09:05:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2679476702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 msgr2=0x55f6a7f69450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81a519c700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f81900a00c0 0x7f81900a2580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81980c89e0 tx=0x7f8198009040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f69450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f819400b640 tx=0x7f819400b670 comp rx=0 tx=0).stop 2023-08-25T09:05:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2679476702 shutdown_connections 2023-08-25T09:05:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f69990 0x55f6a7f69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2679476702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f69450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2679476702 >> 172.21.15.99:0/2679476702 conn(0x55f6a7ede080 msgr2=0x55f6a7f63d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.799+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2679476702 shutdown_connections 2023-08-25T09:05:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.800+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2679476702 wait complete. 2023-08-25T09:05:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.800+0000 7f81ba4beb80 1 -- start start 2023-08-25T09:05:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.800+0000 7f81ba4beb80 1 Processor -- start 2023-08-25T09:05:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f756d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f73020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7ebb930 con 0x55f6a7f6f860 2023-08-25T09:05:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7e59b50 con 0x55f6a7f6ef40 2023-08-25T09:05:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7e59850 con 0x55f6a7f69990 2023-08-25T09:05:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f81ba4beb80 10 client.0 _ll_register_callbacks cb 0x55f6a7da3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f81a499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.801+0000 7f819ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f756d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f81a499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:34774/0 (socket says 172.21.15.99:34774) 2023-08-25T09:05:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f81a499b700 1 -- 172.21.15.99:0/2566593223 learned_addr learned my addr 172.21.15.99:0/2566593223 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f819ffff700 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f73020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f81a499b700 1 -- 172.21.15.99:0/2566593223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 msgr2=0x55f6a7f756d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f81a499b700 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f756d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f81a499b700 1 -- 172.21.15.99:0/2566593223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 msgr2=0x55f6a7f73020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f81a499b700 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f73020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f81a499b700 1 -- 172.21.15.99:0/2566593223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a010e070 con 0x55f6a7f6ef40 2023-08-25T09:05:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f819ffff700 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f756d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:05:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f81a499b700 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f728e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8194000c00 tx=0x7f8194020b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d40cfd38715885d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.802+0000 7f817ffff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:05:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.803+0000 7f817ffff700 1 -- 172.21.15.99:0/2566593223 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194024070 con 0x55f6a7f6ef40 2023-08-25T09:05:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.803+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2566593223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6a7f39160 con 0x55f6a7f6ef40 2023-08-25T09:05:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.803+0000 7f817ffff700 1 -- 172.21.15.99:0/2566593223 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8194004500 con 0x55f6a7f6ef40 2023-08-25T09:05:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.803+0000 7f817ffff700 1 -- 172.21.15.99:0/2566593223 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81940046f0 con 0x55f6a7f6ef40 2023-08-25T09:05:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.803+0000 7f81ba4beb80 10 client.25745 fetch_fsmap learned FSMap version 28 2023-08-25T09:05:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.803+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2566593223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f6a7ce7d80 con 0x55f6a7f6ef40 2023-08-25T09:05:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.803+0000 7f817ffff700 1 -- 172.21.15.99:0/2566593223 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f819403d020 con 0x55f6a7f6ef40 2023-08-25T09:05:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f817ffff700 1 -- 172.21.15.99:0/2566593223 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8194029420 con 0x55f6a7f6ef40 2023-08-25T09:05:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f81ba4beb80 10 client.25745 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:05:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f81ba4beb80 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f81ba4beb80 20 client.25745 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f6a7f82d00 0x55f6a7f85120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f81ba4beb80 4 client.25745 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:05:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2566593223 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6a7e61b40 con 0x55f6a7f82d00 2023-08-25T09:05:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f819ffff700 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f6a7f82d00 0x55f6a7f85120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f819ffff700 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f6a7f82d00 0x55f6a7f85120 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e5473dc9bba6f5 server_cookie=db8f446c4770be91 in_seq=0 out_seq=0 2023-08-25T09:05:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.804+0000 7f817ffff700 10 client.25745 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.805+0000 7f817ffff700 1 -- 172.21.15.99:0/2566593223 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6a7e61b40 con 0x55f6a7f82d00 2023-08-25T09:05:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.805+0000 7f817ffff700 10 client.25745 handle_command_reply: tid=0 2023-08-25T09:05:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.805+0000 7f81ba4beb80 1 client.25745 shutdown 2023-08-25T09:05:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2566593223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6ef40 msgr2=0x55f6a7f728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f728e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8194000c00 tx=0x7f8194020b80 comp rx=0 tx=0).stop 2023-08-25T09:05:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2566593223 shutdown_connections 2023-08-25T09:05:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f73020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f756d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f6a7f82d00 0x55f6a7f85120 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2566593223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f728e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f817ffff700 0 client.25745 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2566593223 >> 172.21.15.99:0/2566593223 conn(0x55f6a7ede080 msgr2=0x55f6a7f5e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2566593223 shutdown_connections 2023-08-25T09:05:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2566593223 wait complete. 2023-08-25T09:05:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.806+0000 7f81ba4beb80 20 client.25745 trim_cache size 0 max 16384 2023-08-25T09:05:03.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:03 smithi099 ceph-mon[97641]: pgmap v179: 97 pgs: 97 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 885 KiB/s wr, 123 op/s 2023-08-25T09:05:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.822+0000 7f81ba4beb80 1 Processor -- start 2023-08-25T09:05:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81ba4beb80 1 -- start start 2023-08-25T09:05:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f69450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f7a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f75200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7d3d8c0 con 0x55f6a7f6ef40 2023-08-25T09:05:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7f75ce0 con 0x55f6a7f6f860 2023-08-25T09:05:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7f60560 con 0x55f6a7f69990 2023-08-25T09:05:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81a499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f7a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f819ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f69450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81a499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f7a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41010/0 (socket says 172.21.15.99:41010) 2023-08-25T09:05:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81a499b700 1 -- 172.21.15.99:0/2450794027 learned_addr learned my addr 172.21.15.99:0/2450794027 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f819ffff700 1 --2- 172.21.15.99:0/2450794027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f75200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81a499b700 1 -- 172.21.15.99:0/2450794027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 msgr2=0x55f6a7f69450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81a499b700 1 --2- 172.21.15.99:0/2450794027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f69450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81a499b700 1 -- 172.21.15.99:0/2450794027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 msgr2=0x55f6a7f75200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.823+0000 7f81a499b700 1 --2- 172.21.15.99:0/2450794027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f75200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.824+0000 7f81a499b700 1 -- 172.21.15.99:0/2450794027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6a7cd0c00 con 0x55f6a7f6ef40 2023-08-25T09:05:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.824+0000 7f81a499b700 1 --2- 172.21.15.99:0/2450794027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f7a210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8194009ce0 tx=0x7f8194009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac216758d98fb90 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.824+0000 7f818b7fe700 1 -- 172.21.15.99:0/2450794027 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194024040 con 0x55f6a7f6ef40 2023-08-25T09:05:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.824+0000 7f818b7fe700 1 -- 172.21.15.99:0/2450794027 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f819403d070 con 0x55f6a7f6ef40 2023-08-25T09:05:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.824+0000 7f818b7fe700 1 -- 172.21.15.99:0/2450794027 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194005530 con 0x55f6a7f6ef40 2023-08-25T09:05:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2450794027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6ef40 msgr2=0x55f6a7f7a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2450794027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f7a210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8194009ce0 tx=0x7f8194009380 comp rx=0 tx=0).stop 2023-08-25T09:05:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2450794027 shutdown_connections 2023-08-25T09:05:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2450794027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f6f860 0x55f6a7f75200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2450794027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f69990 0x55f6a7f69450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/2450794027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f7a210 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2450794027 >> 172.21.15.99:0/2450794027 conn(0x55f6a7ede080 msgr2=0x55f6a7f63c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2450794027 shutdown_connections 2023-08-25T09:05:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 -- 172.21.15.99:0/2450794027 wait complete. 2023-08-25T09:05:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 -- start start 2023-08-25T09:05:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.825+0000 7f81ba4beb80 1 Processor -- start 2023-08-25T09:05:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f69990 0x55f6a7f6a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81ba4beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f6b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7f6b840 con 0x55f6a7f6f860 2023-08-25T09:05:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7f6b9b0 con 0x55f6a7f69990 2023-08-25T09:05:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81ba4beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6a7f6bb70 con 0x55f6a7f6ef40 2023-08-25T09:05:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81ba4beb80 10 client.0 _ll_register_callbacks cb 0x7f818c009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81a499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81a499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:40378/0 (socket says 172.21.15.99:40378) 2023-08-25T09:05:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f81a499b700 1 -- 172.21.15.99:0/3224800178 learned_addr learned my addr 172.21.15.99:0/3224800178 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f819ffff700 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f69990 0x55f6a7f6a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.826+0000 7f819ffff700 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f6b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81a499b700 1 -- 172.21.15.99:0/3224800178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f69990 msgr2=0x55f6a7f6a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81a499b700 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f69990 0x55f6a7f6a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81a499b700 1 -- 172.21.15.99:0/3224800178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 msgr2=0x55f6a7f6b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81a499b700 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f6b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81a499b700 1 -- 172.21.15.99:0/3224800178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6a7ce7d80 con 0x55f6a7f6ef40 2023-08-25T09:05:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f819ffff700 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f6b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81a499b700 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6adc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8194005930 tx=0x7f81940299a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2361ebee5609a463 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f817ffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:05:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f817ffff700 1 -- 172.21.15.99:0/3224800178 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194024040 con 0x55f6a7f6ef40 2023-08-25T09:05:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81ba4beb80 1 -- 172.21.15.99:0/3224800178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6a7f5c600 con 0x55f6a7f6ef40 2023-08-25T09:05:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f817ffff700 1 -- 172.21.15.99:0/3224800178 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8194000e40 con 0x55f6a7f6ef40 2023-08-25T09:05:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f817ffff700 1 -- 172.21.15.99:0/3224800178 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8194029ba0 con 0x55f6a7f6ef40 2023-08-25T09:05:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81ba4beb80 10 client.25738 fetch_fsmap learned FSMap version 28 2023-08-25T09:05:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.827+0000 7f81ba4beb80 1 -- 172.21.15.99:0/3224800178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f6a7d0e1f0 con 0x55f6a7f6ef40 2023-08-25T09:05:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f817ffff700 1 -- 172.21.15.99:0/3224800178 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6a7f5c600 con 0x55f6a7f6ef40 2023-08-25T09:05:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f817ffff700 1 -- 172.21.15.99:0/3224800178 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f81940445a0 con 0x55f6a7f6ef40 2023-08-25T09:05:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f81ba4beb80 10 client.25738 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:05:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f81ba4beb80 10 client.25738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f81ba4beb80 20 client.25738 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f6a7fd7040 0x55f6a7fd9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f81ba4beb80 4 client.25738 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:05:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f81ba4beb80 1 -- 172.21.15.99:0/3224800178 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f6a7eb4910 con 0x55f6a7fd7040 2023-08-25T09:05:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f819ffff700 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f6a7fd7040 0x55f6a7fd9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f819ffff700 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f6a7fd7040 0x55f6a7fd9460 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7a889f63577a4c server_cookie=417648da7379e6c2 in_seq=0 out_seq=0 2023-08-25T09:05:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.828+0000 7f817ffff700 10 client.25738 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.858+0000 7f817ffff700 1 -- 172.21.15.99:0/3224800178 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f6a7eb4910 con 0x55f6a7fd7040 2023-08-25T09:05:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.858+0000 7f817ffff700 10 client.25738 handle_command_reply: tid=0 2023-08-25T09:05:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.858+0000 7f81ba4beb80 1 client.25738 shutdown 2023-08-25T09:05:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 -- 172.21.15.99:0/3224800178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 msgr2=0x55f6a7f6adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6adc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8194005930 tx=0x7f81940299a0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 -- 172.21.15.99:0/3224800178 shutdown_connections 2023-08-25T09:05:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6a7f6f860 0x55f6a7f6b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f6a7fd7040 0x55f6a7fd9460 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6a7f69990 0x55f6a7f6a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f817ffff700 0 client.25738 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 --2- 172.21.15.99:0/3224800178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6a7f6ef40 0x55f6a7f6adc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 -- 172.21.15.99:0/3224800178 >> 172.21.15.99:0/3224800178 conn(0x55f6a7ede080 msgr2=0x55f6a7f6caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 -- 172.21.15.99:0/3224800178 shutdown_connections 2023-08-25T09:05:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 1 -- 172.21.15.99:0/3224800178 wait complete. 2023-08-25T09:05:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.859+0000 7f81ba4beb80 20 client.25738 trim_cache size 0 max 16384 2023-08-25T09:05:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:05:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:05:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:05:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:05:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 -- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f81900a00c0 msgr2=0x7f81900a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f81900a00c0 0x7f81900a2580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81980c89e0 tx=0x7f8198009040 comp rx=0 tx=0).stop 2023-08-25T09:05:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 -- 172.21.15.99:0/2759812140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a01068b0 msgr2=0x7f81a0196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a01068b0 0x7f81a0196780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f819800efc0 tx=0x7f819800c3a0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 -- 172.21.15.99:0/2759812140 shutdown_connections 2023-08-25T09:05:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f81900a00c0 0x7f81900a2580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a01068b0 0x7f81a0196780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a01008a0 0x7f81a019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 --2- 172.21.15.99:0/2759812140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81a0101260 0x7f81a019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 -- 172.21.15.99:0/2759812140 >> 172.21.15.99:0/2759812140 conn(0x7f81a00fc370 msgr2=0x7f81a00fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 -- 172.21.15.99:0/2759812140 shutdown_connections 2023-08-25T09:05:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:03.860+0000 7f81a6bff700 1 -- 172.21.15.99:0/2759812140 wait complete. 2023-08-25T09:05:03.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 147236 2.25 MB/sec execute 189 sec latency 45.484 ms 2023-08-25T09:05:04.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 149289 2.31 MB/sec execute 190 sec latency 13.085 ms 2023-08-25T09:05:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:05 smithi181 ceph-mon[131296]: pgmap v180: 97 pgs: 97 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 595 KiB/s wr, 66 op/s 2023-08-25T09:05:05.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:05 smithi099 ceph-mon[97641]: pgmap v180: 97 pgs: 97 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 595 KiB/s wr, 66 op/s 2023-08-25T09:05:05.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:05 smithi157 ceph-mon[130879]: pgmap v180: 97 pgs: 97 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 595 KiB/s wr, 66 op/s 2023-08-25T09:05:05.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 150519 2.33 MB/sec execute 191 sec latency 18.599 ms 2023-08-25T09:05:06.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 152491 2.38 MB/sec execute 192 sec latency 14.815 ms 2023-08-25T09:05:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:07 smithi157 ceph-mon[130879]: pgmap v181: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-08-25T09:05:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:07 smithi181 ceph-mon[131296]: pgmap v181: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-08-25T09:05:07.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:07 smithi099 ceph-mon[97641]: pgmap v181: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-08-25T09:05:07.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 153619 2.39 MB/sec execute 193 sec latency 12.281 ms 2023-08-25T09:05:08.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 154951 2.40 MB/sec execute 194 sec latency 40.751 ms 2023-08-25T09:05:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:09 smithi157 ceph-mon[130879]: pgmap v182: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2023-08-25T09:05:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:09 smithi181 ceph-mon[131296]: pgmap v182: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2023-08-25T09:05:09.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:09 smithi099 ceph-mon[97641]: pgmap v182: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2023-08-25T09:05:09.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 156547 2.45 MB/sec execute 195 sec latency 15.246 ms 2023-08-25T09:05:10.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 158234 2.47 MB/sec execute 196 sec latency 44.125 ms 2023-08-25T09:05:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:11 smithi157 ceph-mon[130879]: pgmap v183: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2023-08-25T09:05:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:11 smithi181 ceph-mon[131296]: pgmap v183: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2023-08-25T09:05:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:11 smithi099 ceph-mon[97641]: pgmap v183: 97 pgs: 97 active+clean; 64 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2023-08-25T09:05:11.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 160048 2.52 MB/sec execute 197 sec latency 11.215 ms 2023-08-25T09:05:12.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 161403 2.54 MB/sec execute 198 sec latency 21.427 ms 2023-08-25T09:05:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:13 smithi157 ceph-mon[130879]: pgmap v184: 97 pgs: 97 active+clean; 71 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-08-25T09:05:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:13 smithi181 ceph-mon[131296]: pgmap v184: 97 pgs: 97 active+clean; 71 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-08-25T09:05:13.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:13 smithi099 ceph-mon[97641]: pgmap v184: 97 pgs: 97 active+clean; 71 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-08-25T09:05:13.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 163471 2.59 MB/sec execute 199 sec latency 15.876 ms 2023-08-25T09:05:14.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 164671 2.61 MB/sec execute 200 sec latency 23.069 ms 2023-08-25T09:05:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:15 smithi181 ceph-mon[131296]: pgmap v185: 97 pgs: 97 active+clean; 71 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2023-08-25T09:05:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:15 smithi099 ceph-mon[97641]: pgmap v185: 97 pgs: 97 active+clean; 71 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2023-08-25T09:05:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:15 smithi157 ceph-mon[130879]: pgmap v185: 97 pgs: 97 active+clean; 71 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2023-08-25T09:05:15.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 166790 2.66 MB/sec execute 201 sec latency 20.706 ms 2023-08-25T09:05:16.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 167939 2.66 MB/sec execute 202 sec latency 21.808 ms 2023-08-25T09:05:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 -- 172.21.15.99:0/109752608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa24c10d0e0 msgr2=0x7fa24c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 --2- 172.21.15.99:0/109752608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa24c10d0e0 0x7fa24c10f550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b0a0 tx=0x7fa24400b3b0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 -- 172.21.15.99:0/109752608 shutdown_connections 2023-08-25T09:05:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 --2- 172.21.15.99:0/109752608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa24c10d0e0 0x7fa24c10f550 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 --2- 172.21.15.99:0/109752608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa24c072d50 0x7fa24c073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 --2- 172.21.15.99:0/109752608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24c073670 0x7fa24c10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 -- 172.21.15.99:0/109752608 >> 172.21.15.99:0/109752608 conn(0x7fa24c0fc370 msgr2=0x7fa24c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 -- 172.21.15.99:0/109752608 shutdown_connections 2023-08-25T09:05:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.011+0000 7fa251dff700 1 -- 172.21.15.99:0/109752608 wait complete. 2023-08-25T09:05:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.012+0000 7fa251dff700 1 Processor -- start 2023-08-25T09:05:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.013+0000 7fa251dff700 1 -- start start 2023-08-25T09:05:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.013+0000 7fa251dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa24c072d50 0x7fa24c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.013+0000 7fa251dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24c073670 0x7fa24c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.013+0000 7fa251dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa24c10d0e0 0x7fa24c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.013+0000 7fa251dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c199130 con 0x7fa24c072d50 2023-08-25T09:05:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.013+0000 7fa251dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c1992a0 con 0x7fa24c073670 2023-08-25T09:05:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.013+0000 7fa251dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c199460 con 0x7fa24c10d0e0 2023-08-25T09:05:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa24c072d50 0x7fa24c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24c073670 0x7fa24c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa24c072d50 0x7fa24c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38958/0 (socket says 172.21.15.99:38958) 2023-08-25T09:05:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24b7fe700 1 -- 172.21.15.99:0/2043456475 learned_addr learned my addr 172.21.15.99:0/2043456475 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24bfff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa24c10d0e0 0x7fa24c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24affd700 1 -- 172.21.15.99:0/2043456475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa24c10d0e0 msgr2=0x7fa24c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24affd700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa24c10d0e0 0x7fa24c198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24affd700 1 -- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa24c072d50 msgr2=0x7fa24c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24affd700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa24c072d50 0x7fa24c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24affd700 1 -- 172.21.15.99:0/2043456475 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244009d00 con 0x7fa24c073670 2023-08-25T09:05:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.014+0000 7fa24affd700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24c073670 0x7fa24c19f1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9f0 tx=0x7fa24000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.015+0000 7fa248ff9700 1 -- 172.21.15.99:0/2043456475 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000c9c0 con 0x7fa24c073670 2023-08-25T09:05:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.015+0000 7fa248ff9700 1 -- 172.21.15.99:0/2043456475 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa240010430 con 0x7fa24c073670 2023-08-25T09:05:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.015+0000 7fa251dff700 1 -- 172.21.15.99:0/2043456475 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24c199740 con 0x7fa24c073670 2023-08-25T09:05:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.015+0000 7fa248ff9700 1 -- 172.21.15.99:0/2043456475 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240018680 con 0x7fa24c073670 2023-08-25T09:05:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.015+0000 7fa251dff700 1 -- 172.21.15.99:0/2043456475 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c1a5c50 con 0x7fa24c073670 2023-08-25T09:05:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.018+0000 7fa248ff9700 1 -- 172.21.15.99:0/2043456475 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa240010860 con 0x7fa24c073670 2023-08-25T09:05:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.019+0000 7fa265670b80 1 Processor -- start 2023-08-25T09:05:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa265670b80 1 -- start start 2023-08-25T09:05:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa265670b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa265670b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa265670b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa265670b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb702750 con 0x55c1fb7a0860 2023-08-25T09:05:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa265670b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb6f3720 con 0x55c1fb79a990 2023-08-25T09:05:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa265670b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb6f1040 con 0x55c1fb79ff40 2023-08-25T09:05:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa248ff9700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2340a00c0 0x7fa2340a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa24affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa24bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa24affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:37258/0 (socket says 172.21.15.99:37258) 2023-08-25T09:05:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.020+0000 7fa24affd700 1 -- 172.21.15.99:0/1360117561 learned_addr learned my addr 172.21.15.99:0/1360117561 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24bfff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2340a00c0 0x7fa2340a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24affd700 1 -- 172.21.15.99:0/1360117561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 msgr2=0x55c1fb7a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24affd700 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24affd700 1 -- 172.21.15.99:0/1360117561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 msgr2=0x55c1fb79a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24affd700 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24affd700 1 -- 172.21.15.99:0/1360117561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1fb56c5d0 con 0x55c1fb79a990 2023-08-25T09:05:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa248ff9700 1 -- 172.21.15.99:0/2043456475 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa240014070 con 0x7fa24c073670 2023-08-25T09:05:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24affd700 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79add0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa240095850 tx=0x7fa240010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=429384033b32fcc1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa24bfff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2340a00c0 0x7fa2340a2580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa244009cd0 tx=0x7fa24400b720 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa231ffb700 1 -- 172.21.15.99:0/1360117561 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2400994e0 con 0x55c1fb79a990 2023-08-25T09:05:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa231ffb700 1 -- 172.21.15.99:0/1360117561 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa240064d80 con 0x55c1fb79a990 2023-08-25T09:05:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.021+0000 7fa231ffb700 1 -- 172.21.15.99:0/1360117561 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2400106e0 con 0x55c1fb79a990 2023-08-25T09:05:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 -- 172.21.15.99:0/1360117561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 msgr2=0x55c1fb79add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79add0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa240095850 tx=0x7fa240010c60 comp rx=0 tx=0).stop 2023-08-25T09:05:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 -- 172.21.15.99:0/1360117561 shutdown_connections 2023-08-25T09:05:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79add0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 --2- 172.21.15.99:0/1360117561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 -- 172.21.15.99:0/1360117561 >> 172.21.15.99:0/1360117561 conn(0x55c1fb70f080 msgr2=0x55c1fb794d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 -- 172.21.15.99:0/1360117561 shutdown_connections 2023-08-25T09:05:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.022+0000 7fa265670b80 1 -- 172.21.15.99:0/1360117561 wait complete. 2023-08-25T09:05:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.023+0000 7fa265670b80 1 -- start start 2023-08-25T09:05:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.023+0000 7fa265670b80 1 Processor -- start 2023-08-25T09:05:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa265670b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb7a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa265670b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa265670b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb7a4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa265670b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb6ec930 con 0x55c1fb7a0860 2023-08-25T09:05:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa265670b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb68ab50 con 0x55c1fb79a990 2023-08-25T09:05:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa265670b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb68a850 con 0x55c1fb79ff40 2023-08-25T09:05:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa265670b80 10 client.0 _ll_register_callbacks cb 0x55c1fb5d4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58488/0 (socket says 172.21.15.99:58488) 2023-08-25T09:05:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 -- 172.21.15.99:0/1437327863 learned_addr learned my addr 172.21.15.99:0/1437327863 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24bfff700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb7a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb7a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 -- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 msgr2=0x55c1fb7a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb7a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 -- 172.21.15.99:0/1437327863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 msgr2=0x55c1fb7a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb7a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 -- 172.21.15.99:0/1437327863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c1a5c50 con 0x55c1fb79ff40 2023-08-25T09:05:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24bfff700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb7a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb7a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa24affd700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa2400952e0 tx=0x7fa240099590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8dd3d74266a3296 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.024+0000 7fa22affd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:05:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa22affd700 1 -- 172.21.15.99:0/1437327863 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240098de0 con 0x55c1fb79ff40 2023-08-25T09:05:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 1 -- 172.21.15.99:0/1437327863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1fb76a160 con 0x55c1fb79ff40 2023-08-25T09:05:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa22affd700 1 -- 172.21.15.99:0/1437327863 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa2400a0bf0 con 0x55c1fb79ff40 2023-08-25T09:05:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa22affd700 1 -- 172.21.15.99:0/1437327863 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa240099040 con 0x55c1fb79ff40 2023-08-25T09:05:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 10 client.25750 fetch_fsmap learned FSMap version 28 2023-08-25T09:05:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 1 -- 172.21.15.99:0/1437327863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c1fb518d80 con 0x55c1fb79ff40 2023-08-25T09:05:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa22affd700 1 -- 172.21.15.99:0/1437327863 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1fb76a160 con 0x55c1fb79ff40 2023-08-25T09:05:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa22affd700 1 -- 172.21.15.99:0/1437327863 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa2400a0760 con 0x55c1fb79ff40 2023-08-25T09:05:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 10 client.25750 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:05:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 20 client.25750 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c1fb7b3e40 0x55c1fb7b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 4 client.25750 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:05:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.025+0000 7fa265670b80 1 -- 172.21.15.99:0/1437327863 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1fb692b40 con 0x55c1fb7b3e40 2023-08-25T09:05:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.026+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c1fb7b3e40 0x55c1fb7b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.026+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c1fb7b3e40 0x55c1fb7b6260 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcdb8067aa2bc919 server_cookie=bcf2389d910006ad in_seq=0 out_seq=0 2023-08-25T09:05:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.026+0000 7fa22affd700 10 client.25750 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa22affd700 1 -- 172.21.15.99:0/1437327863 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c1fb692b40 con 0x55c1fb7b3e40 2023-08-25T09:05:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa22affd700 10 client.25750 handle_command_reply: tid=0 2023-08-25T09:05:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 client.25750 shutdown 2023-08-25T09:05:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 -- 172.21.15.99:0/1437327863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 msgr2=0x55c1fb7a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa2400952e0 tx=0x7fa240099590 comp rx=0 tx=0).stop 2023-08-25T09:05:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 -- 172.21.15.99:0/1437327863 shutdown_connections 2023-08-25T09:05:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb7a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c1fb7b3e40 0x55c1fb7b6260 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb7a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 --2- 172.21.15.99:0/1437327863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa22affd700 0 client.25750 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 -- 172.21.15.99:0/1437327863 >> 172.21.15.99:0/1437327863 conn(0x55c1fb70f080 msgr2=0x55c1fb78f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 -- 172.21.15.99:0/1437327863 shutdown_connections 2023-08-25T09:05:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.027+0000 7fa265670b80 1 -- 172.21.15.99:0/1437327863 wait complete. 2023-08-25T09:05:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.028+0000 7fa265670b80 20 client.25750 trim_cache size 0 max 16384 2023-08-25T09:05:17.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa265670b80 1 Processor -- start 2023-08-25T09:05:17.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa265670b80 1 -- start start 2023-08-25T09:05:17.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa265670b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa265670b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa265670b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb7a0860 0x55c1fb7ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa265670b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb56e8c0 con 0x55c1fb79ff40 2023-08-25T09:05:17.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa265670b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb7a0320 con 0x55c1fb79a990 2023-08-25T09:05:17.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa265670b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb796060 con 0x55c1fb7a0860 2023-08-25T09:05:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa24b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa24affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa24bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb7a0860 0x55c1fb7ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa24b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:37276/0 (socket says 172.21.15.99:37276) 2023-08-25T09:05:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.045+0000 7fa24b7fe700 1 -- 172.21.15.99:0/1048791604 learned_addr learned my addr 172.21.15.99:0/1048791604 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.046+0000 7fa24b7fe700 1 -- 172.21.15.99:0/1048791604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb7a0860 msgr2=0x55c1fb7ab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.046+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1048791604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb7a0860 0x55c1fb7ab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.046+0000 7fa24b7fe700 1 -- 172.21.15.99:0/1048791604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb79ff40 msgr2=0x55c1fb7a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.046+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1048791604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.046+0000 7fa24b7fe700 1 -- 172.21.15.99:0/1048791604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1fb501c00 con 0x55c1fb79a990 2023-08-25T09:05:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.046+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/1048791604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79a450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00b370 tx=0x7fa23c01cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c2ff0692df69f2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.046+0000 7fa2327fc700 1 -- 172.21.15.99:0/1048791604 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c017040 con 0x55c1fb79a990 2023-08-25T09:05:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa2327fc700 1 -- 172.21.15.99:0/1048791604 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa23c004510 con 0x55c1fb79a990 2023-08-25T09:05:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa2327fc700 1 -- 172.21.15.99:0/1048791604 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c003d60 con 0x55c1fb79a990 2023-08-25T09:05:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 -- 172.21.15.99:0/1048791604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 msgr2=0x55c1fb79a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 --2- 172.21.15.99:0/1048791604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79a450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00b370 tx=0x7fa23c01cbe0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 -- 172.21.15.99:0/1048791604 shutdown_connections 2023-08-25T09:05:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 --2- 172.21.15.99:0/1048791604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb7a0860 0x55c1fb7ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 --2- 172.21.15.99:0/1048791604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79a990 0x55c1fb79a450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 --2- 172.21.15.99:0/1048791604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb79ff40 0x55c1fb7a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 -- 172.21.15.99:0/1048791604 >> 172.21.15.99:0/1048791604 conn(0x55c1fb70f080 msgr2=0x55c1fb7a31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 -- 172.21.15.99:0/1048791604 shutdown_connections 2023-08-25T09:05:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.047+0000 7fa265670b80 1 -- 172.21.15.99:0/1048791604 wait complete. 2023-08-25T09:05:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.048+0000 7fa265670b80 1 -- start start 2023-08-25T09:05:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.048+0000 7fa265670b80 1 Processor -- start 2023-08-25T09:05:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa265670b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79a990 0x55c1fb79cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa265670b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79ff40 0x55c1fb79b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa265670b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa265670b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb79d590 con 0x55c1fb7a0860 2023-08-25T09:05:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa265670b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb79c300 con 0x55c1fb79ff40 2023-08-25T09:05:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa265670b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fb79c490 con 0x55c1fb79a990 2023-08-25T09:05:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa265670b80 10 client.0 _ll_register_callbacks cb 0x55c1fb5d4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa24b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79a990 0x55c1fb79cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa24b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79a990 0x55c1fb79cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58504/0 (socket says 172.21.15.99:58504) 2023-08-25T09:05:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa24b7fe700 1 -- 172.21.15.99:0/2266492475 learned_addr learned my addr 172.21.15.99:0/2266492475 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa24affd700 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79ff40 0x55c1fb79b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.049+0000 7fa24bfff700 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa24b7fe700 1 -- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79ff40 msgr2=0x55c1fb79b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79ff40 0x55c1fb79b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa24b7fe700 1 -- 172.21.15.99:0/2266492475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 msgr2=0x55c1fb79bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa24b7fe700 1 -- 172.21.15.99:0/2266492475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c00bb80 con 0x55c1fb79a990 2023-08-25T09:05:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa24affd700 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79ff40 0x55c1fb79b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:05:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa24b7fe700 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79a990 0x55c1fb79cd40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c00 tx=0x7fa23c01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87ebb21ce679a54 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa22affd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:05:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa22affd700 1 -- 172.21.15.99:0/2266492475 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c017070 con 0x55c1fb79a990 2023-08-25T09:05:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa22affd700 1 -- 172.21.15.99:0/2266492475 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa23c004250 con 0x55c1fb79a990 2023-08-25T09:05:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa265670b80 1 -- 172.21.15.99:0/2266492475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1fb78d600 con 0x55c1fb79a990 2023-08-25T09:05:17.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa22affd700 1 -- 172.21.15.99:0/2266492475 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa23c0132e0 con 0x55c1fb79a990 2023-08-25T09:05:17.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa265670b80 10 client.25756 fetch_fsmap learned FSMap version 28 2023-08-25T09:05:17.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.050+0000 7fa265670b80 1 -- 172.21.15.99:0/2266492475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c1fb53f1f0 con 0x55c1fb79a990 2023-08-25T09:05:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa22affd700 1 -- 172.21.15.99:0/2266492475 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1fb78d600 con 0x55c1fb79a990 2023-08-25T09:05:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa22affd700 1 -- 172.21.15.99:0/2266492475 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa23c004ce0 con 0x55c1fb79a990 2023-08-25T09:05:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa265670b80 10 client.25756 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:05:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa265670b80 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa265670b80 20 client.25756 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa265670b80 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c1fb808f30 0x55c1fb80b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa265670b80 4 client.25756 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:05:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa265670b80 1 -- 172.21.15.99:0/2266492475 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c1fb6e5910 con 0x55c1fb808f30 2023-08-25T09:05:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa24affd700 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c1fb808f30 0x55c1fb80b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa24affd700 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c1fb808f30 0x55c1fb80b350 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c990ab694cfefa server_cookie=71f1a53c17693477 in_seq=0 out_seq=0 2023-08-25T09:05:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.051+0000 7fa22affd700 10 client.25756 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa22affd700 1 -- 172.21.15.99:0/2266492475 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c1fb6e5910 con 0x55c1fb808f30 2023-08-25T09:05:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa22affd700 10 client.25756 handle_command_reply: tid=0 2023-08-25T09:05:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 client.25756 shutdown 2023-08-25T09:05:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 -- 172.21.15.99:0/2266492475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79a990 msgr2=0x55c1fb79cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79a990 0x55c1fb79cd40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c00 tx=0x7fa23c01c440 comp rx=0 tx=0).stop 2023-08-25T09:05:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 -- 172.21.15.99:0/2266492475 shutdown_connections 2023-08-25T09:05:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c1fb7a0860 0x55c1fb79bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1fb79a990 0x55c1fb79cd40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c1fb808f30 0x55c1fb80b350 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 --2- 172.21.15.99:0/2266492475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1fb79ff40 0x55c1fb79b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa22affd700 0 client.25756 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 -- 172.21.15.99:0/2266492475 >> 172.21.15.99:0/2266492475 conn(0x55c1fb70f080 msgr2=0x55c1fb7a0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.107+0000 7fa265670b80 1 -- 172.21.15.99:0/2266492475 shutdown_connections 2023-08-25T09:05:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.108+0000 7fa265670b80 1 -- 172.21.15.99:0/2266492475 wait complete. 2023-08-25T09:05:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.108+0000 7fa265670b80 20 client.25756 trim_cache size 0 max 16384 2023-08-25T09:05:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:05:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:05:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:05:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:05:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.108+0000 7fa251dff700 1 -- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2340a00c0 msgr2=0x7fa2340a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.108+0000 7fa251dff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2340a00c0 0x7fa2340a2580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa244009cd0 tx=0x7fa24400b720 comp rx=0 tx=0).stop 2023-08-25T09:05:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.108+0000 7fa251dff700 1 -- 172.21.15.99:0/2043456475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24c073670 msgr2=0x7fa24c19f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.108+0000 7fa251dff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24c073670 0x7fa24c19f1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9f0 tx=0x7fa24000ed00 comp rx=0 tx=0).stop 2023-08-25T09:05:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.109+0000 7fa251dff700 1 -- 172.21.15.99:0/2043456475 shutdown_connections 2023-08-25T09:05:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.109+0000 7fa251dff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2340a00c0 0x7fa2340a2580 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.109+0000 7fa251dff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa24c10d0e0 0x7fa24c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.109+0000 7fa251dff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa24c072d50 0x7fa24c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.109+0000 7fa251dff700 1 --2- 172.21.15.99:0/2043456475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24c073670 0x7fa24c19f1a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.109+0000 7fa251dff700 1 -- 172.21.15.99:0/2043456475 >> 172.21.15.99:0/2043456475 conn(0x7fa24c0fc370 msgr2=0x7fa24c104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.109+0000 7fa251dff700 1 -- 172.21.15.99:0/2043456475 shutdown_connections 2023-08-25T09:05:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:17.109+0000 7fa251dff700 1 -- 172.21.15.99:0/2043456475 wait complete. 2023-08-25T09:05:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:17 smithi157 ceph-mon[130879]: pgmap v186: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2023-08-25T09:05:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:17 smithi181 ceph-mon[131296]: pgmap v186: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2023-08-25T09:05:17.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:17 smithi099 ceph-mon[97641]: pgmap v186: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2023-08-25T09:05:17.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 169162 2.67 MB/sec execute 203 sec latency 63.035 ms 2023-08-25T09:05:18.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 170727 2.72 MB/sec execute 204 sec latency 16.330 ms 2023-08-25T09:05:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:19 smithi157 ceph-mon[130879]: pgmap v187: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-08-25T09:05:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:19 smithi181 ceph-mon[131296]: pgmap v187: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-08-25T09:05:19.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:19 smithi099 ceph-mon[97641]: pgmap v187: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-08-25T09:05:19.917 INFO:tasks.workunit.client.0.smithi099.stdout: 1 172099 2.73 MB/sec execute 205 sec latency 22.336 ms 2023-08-25T09:05:20.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 174141 2.78 MB/sec execute 206 sec latency 14.614 ms 2023-08-25T09:05:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:21 smithi157 ceph-mon[130879]: pgmap v188: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-08-25T09:05:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:21 smithi181 ceph-mon[131296]: pgmap v188: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-08-25T09:05:21.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:21 smithi099 ceph-mon[97641]: pgmap v188: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-08-25T09:05:21.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 175352 2.79 MB/sec execute 207 sec latency 48.145 ms 2023-08-25T09:05:22.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 177575 2.85 MB/sec execute 208 sec latency 28.475 ms 2023-08-25T09:05:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:23 smithi157 ceph-mon[130879]: pgmap v189: 97 pgs: 97 active+clean; 71 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 276 op/s 2023-08-25T09:05:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:23 smithi181 ceph-mon[131296]: pgmap v189: 97 pgs: 97 active+clean; 71 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 276 op/s 2023-08-25T09:05:23.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:23 smithi099 ceph-mon[97641]: pgmap v189: 97 pgs: 97 active+clean; 71 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 276 op/s 2023-08-25T09:05:23.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 178766 2.86 MB/sec execute 209 sec latency 23.454 ms 2023-08-25T09:05:24.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 180337 2.88 MB/sec execute 210 sec latency 36.272 ms 2023-08-25T09:05:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:25 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:05:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:25 smithi181 ceph-mon[131296]: pgmap v190: 97 pgs: 97 active+clean; 71 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-08-25T09:05:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:25 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:05:25.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:25 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:05:25.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:25 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:05:25.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:25 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:05:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:25 smithi099 ceph-mon[97641]: pgmap v190: 97 pgs: 97 active+clean; 71 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-08-25T09:05:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:25 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:05:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:25 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:05:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:25 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:05:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:25 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:05:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:25 smithi157 ceph-mon[130879]: pgmap v190: 97 pgs: 97 active+clean; 71 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-08-25T09:05:25.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:25 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:05:25.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:25 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:05:25.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:25 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:05:25.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 181623 2.90 MB/sec execute 211 sec latency 21.304 ms 2023-08-25T09:05:26.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 183320 2.92 MB/sec execute 212 sec latency 29.571 ms 2023-08-25T09:05:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:27 smithi157 ceph-mon[130879]: pgmap v191: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 293 op/s 2023-08-25T09:05:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:27 smithi181 ceph-mon[131296]: pgmap v191: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 293 op/s 2023-08-25T09:05:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:27 smithi099 ceph-mon[97641]: pgmap v191: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 293 op/s 2023-08-25T09:05:27.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 184913 2.96 MB/sec execute 213 sec latency 26.436 ms 2023-08-25T09:05:28.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 186656 2.97 MB/sec execute 214 sec latency 23.016 ms 2023-08-25T09:05:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:29 smithi157 ceph-mon[130879]: pgmap v192: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2023-08-25T09:05:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:29 smithi181 ceph-mon[131296]: pgmap v192: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2023-08-25T09:05:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:29 smithi099 ceph-mon[97641]: pgmap v192: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2023-08-25T09:05:29.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 188407 3.02 MB/sec execute 215 sec latency 14.127 ms 2023-08-25T09:05:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.261+0000 7f0f193ff700 1 -- 172.21.15.99:0/3848199165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f141068d0 msgr2=0x7f0f14106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.261+0000 7f0f193ff700 1 --2- 172.21.15.99:0/3848199165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f141068d0 0x7f0f14106cb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f040099c0 tx=0x7f0f04009cd0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.262+0000 7f0f193ff700 1 -- 172.21.15.99:0/3848199165 shutdown_connections 2023-08-25T09:05:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.262+0000 7f0f193ff700 1 --2- 172.21.15.99:0/3848199165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f140686d0 0x7f0f1410d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.262+0000 7f0f193ff700 1 --2- 172.21.15.99:0/3848199165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f141068d0 0x7f0f14106cb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.262+0000 7f0f193ff700 1 --2- 172.21.15.99:0/3848199165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f14067d10 0x7f0f14068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.262+0000 7f0f193ff700 1 -- 172.21.15.99:0/3848199165 >> 172.21.15.99:0/3848199165 conn(0x7f0f14075250 msgr2=0x7f0f14075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.262+0000 7f0f193ff700 1 -- 172.21.15.99:0/3848199165 shutdown_connections 2023-08-25T09:05:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.262+0000 7f0f193ff700 1 -- 172.21.15.99:0/3848199165 wait complete. 2023-08-25T09:05:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.263+0000 7f0f193ff700 1 Processor -- start 2023-08-25T09:05:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.263+0000 7f0f193ff700 1 -- start start 2023-08-25T09:05:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.263+0000 7f0f193ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f14067d10 0x7f0f140ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.263+0000 7f0f193ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f140686d0 0x7f0f14100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.263+0000 7f0f193ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f141068d0 0x7f0f141a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.263+0000 7f0f193ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a5140 con 0x7f0f141068d0 2023-08-25T09:05:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.263+0000 7f0f193ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a5280 con 0x7f0f140686d0 2023-08-25T09:05:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.263+0000 7f0f193ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a53f0 con 0x7f0f14067d10 2023-08-25T09:05:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f14067d10 0x7f0f140ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f14067d10 0x7f0f140ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:56014/0 (socket says 172.21.15.99:56014) 2023-08-25T09:05:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1827074104 learned_addr learned my addr 172.21.15.99:0/1827074104 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f127fc700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f140686d0 0x7f0f14100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f137fe700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f141068d0 0x7f0f141a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1827074104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f140686d0 msgr2=0x7f0f14100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f140686d0 0x7f0f14100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f141068d0 msgr2=0x7f0f141a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f141068d0 0x7f0f141a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1827074104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f040095f0 con 0x7f0f14067d10 2023-08-25T09:05:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f137fe700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f141068d0 0x7f0f141a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.264+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f14067d10 0x7f0f140ffd30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f040036c0 tx=0x7f0f0400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.265+0000 7f0efbfff700 1 -- 172.21.15.99:0/1827074104 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0401d070 con 0x7f0f14067d10 2023-08-25T09:05:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.265+0000 7f0efbfff700 1 -- 172.21.15.99:0/1827074104 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f04005050 con 0x7f0f14067d10 2023-08-25T09:05:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.265+0000 7f0f193ff700 1 -- 172.21.15.99:0/1827074104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f141a5670 con 0x7f0f14067d10 2023-08-25T09:05:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.265+0000 7f0efbfff700 1 -- 172.21.15.99:0/1827074104 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04004a00 con 0x7f0f14067d10 2023-08-25T09:05:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.265+0000 7f0f193ff700 1 -- 172.21.15.99:0/1827074104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f141a5a90 con 0x7f0f14067d10 2023-08-25T09:05:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.268+0000 7f0efbfff700 1 -- 172.21.15.99:0/1827074104 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f0f040051c0 con 0x7f0f14067d10 2023-08-25T09:05:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0f2cc57b80 1 Processor -- start 2023-08-25T09:05:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0f2cc57b80 1 -- start start 2023-08-25T09:05:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0efbfff700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0efc0a00c0 0x7f0efc0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0efbfff700 1 -- 172.21.15.99:0/1827074104 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0f040cc440 con 0x7f0f14067d10 2023-08-25T09:05:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a7e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7e860 0x55e612a78450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a78dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6129e0750 con 0x55e612a7df40 2023-08-25T09:05:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6129d1720 con 0x55e612a7e860 2023-08-25T09:05:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.270+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6129cf040 con 0x55e612a78990 2023-08-25T09:05:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7e860 0x55e612a78450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a7e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7e860 0x55e612a78450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47454/0 (socket says 172.21.15.99:47454) 2023-08-25T09:05:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 -- 172.21.15.99:0/2423279017 learned_addr learned my addr 172.21.15.99:0/2423279017 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f137fe700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0efc0a00c0 0x7f0efc0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 -- 172.21.15.99:0/2423279017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 msgr2=0x55e612a78dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 --2- 172.21.15.99:0/2423279017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a78dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 -- 172.21.15.99:0/2423279017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 msgr2=0x55e612a7e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 --2- 172.21.15.99:0/2423279017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a7e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 -- 172.21.15.99:0/2423279017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e61284a5d0 con 0x55e612a7e860 2023-08-25T09:05:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f127fc700 1 --2- 172.21.15.99:0/2423279017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7e860 0x55e612a78450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009d20 tx=0x7f0f08011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c47103001d3737fd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0f137fe700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0efc0a00c0 0x7f0efc0a2580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009410 tx=0x7f0f0c009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.271+0000 7f0ef97fa700 1 -- 172.21.15.99:0/2423279017 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08019630 con 0x55e612a7e860 2023-08-25T09:05:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0ef97fa700 1 -- 172.21.15.99:0/2423279017 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f08019c50 con 0x55e612a7e860 2023-08-25T09:05:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0ef97fa700 1 -- 172.21.15.99:0/2423279017 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0800ec40 con 0x55e612a7e860 2023-08-25T09:05:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/2423279017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7e860 msgr2=0x55e612a78450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/2423279017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7e860 0x55e612a78450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009d20 tx=0x7f0f08011720 comp rx=0 tx=0).stop 2023-08-25T09:05:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/2423279017 shutdown_connections 2023-08-25T09:05:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/2423279017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a78dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/2423279017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a7e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/2423279017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7e860 0x55e612a78450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.272+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/2423279017 >> 172.21.15.99:0/2423279017 conn(0x55e6129ed080 msgr2=0x55e612a72d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.273+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/2423279017 shutdown_connections 2023-08-25T09:05:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.273+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/2423279017 wait complete. 2023-08-25T09:05:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.273+0000 7f0f2cc57b80 1 -- start start 2023-08-25T09:05:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.273+0000 7f0f2cc57b80 1 Processor -- start 2023-08-25T09:05:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7df40 0x55e612a81950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7e860 0x55e612a82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6129ca930 con 0x55e612a7e860 2023-08-25T09:05:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e612968b50 con 0x55e612a7df40 2023-08-25T09:05:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e612968850 con 0x55e612a78990 2023-08-25T09:05:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f2cc57b80 10 client.0 _ll_register_callbacks cb 0x55e6128b2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a846d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:56028/0 (socket says 172.21.15.99:56028) 2023-08-25T09:05:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1879298319 learned_addr learned my addr 172.21.15.99:0/1879298319 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f127fc700 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7df40 0x55e612a81950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.274+0000 7f0f137fe700 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7e860 0x55e612a82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1879298319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7df40 msgr2=0x55e612a81950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7df40 0x55e612a81950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1879298319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7e860 msgr2=0x55e612a82090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7e860 0x55e612a82090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1879298319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f08009980 con 0x55e612a78990 2023-08-25T09:05:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a846d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0f040960a0 tx=0x7f0f0409a3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6391dab46b3cc1f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0ef6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:05:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/1879298319 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0409d9d0 con 0x55e612a78990 2023-08-25T09:05:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/1879298319 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0f040a1920 con 0x55e612a78990 2023-08-25T09:05:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1879298319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e612a48160 con 0x55e612a78990 2023-08-25T09:05:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/1879298319 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f0409a010 con 0x55e612a78990 2023-08-25T09:05:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f2cc57b80 10 client.25765 fetch_fsmap learned FSMap version 28 2023-08-25T09:05:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.275+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1879298319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e6127f6d80 con 0x55e612a78990 2023-08-25T09:05:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/1879298319 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e612a48160 con 0x55e612a78990 2023-08-25T09:05:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/1879298319 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0f0409abb0 con 0x55e612a78990 2023-08-25T09:05:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0f2cc57b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:05:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0f2cc57b80 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0f2cc57b80 20 client.25765 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e612a91e40 0x55e612a94230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0f2cc57b80 4 client.25765 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:05:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1879298319 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e612970b40 con 0x55e612a91e40 2023-08-25T09:05:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.276+0000 7f0f127fc700 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e612a91e40 0x55e612a94230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.277+0000 7f0f127fc700 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e612a91e40 0x55e612a94230 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88109b7f546146d2 server_cookie=96a14d3d95293254 in_seq=0 out_seq=0 2023-08-25T09:05:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.277+0000 7f0ef6ff5700 10 client.25765 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.277+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/1879298319 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e612970b40 con 0x55e612a91e40 2023-08-25T09:05:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.277+0000 7f0ef6ff5700 10 client.25765 handle_command_reply: tid=0 2023-08-25T09:05:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.277+0000 7f0f2cc57b80 1 client.25765 shutdown 2023-08-25T09:05:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1879298319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 msgr2=0x55e612a846d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a846d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0f040960a0 tx=0x7f0f0409a3e0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1879298319 shutdown_connections 2023-08-25T09:05:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7e860 0x55e612a82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a78990 0x55e612a846d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e612a91e40 0x55e612a94230 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1879298319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a7df40 0x55e612a81950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0ef6ff5700 0 client.25765 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1879298319 >> 172.21.15.99:0/1879298319 conn(0x55e6129ed080 msgr2=0x55e612a6d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1879298319 shutdown_connections 2023-08-25T09:05:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1879298319 wait complete. 2023-08-25T09:05:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.278+0000 7f0f2cc57b80 20 client.25765 trim_cache size 0 max 16384 2023-08-25T09:05:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.295+0000 7f0f2cc57b80 1 Processor -- start 2023-08-25T09:05:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.295+0000 7f0f2cc57b80 1 -- start start 2023-08-25T09:05:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a78450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a89270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 0x55e612a72c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61284c8c0 con 0x55e612a7df40 2023-08-25T09:05:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e612a82690 con 0x55e612a78990 2023-08-25T09:05:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e612a897b0 con 0x55e612a7e860 2023-08-25T09:05:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a78450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f127fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a89270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a78450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47476/0 (socket says 172.21.15.99:47476) 2023-08-25T09:05:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1061929594 learned_addr learned my addr 172.21.15.99:0/1061929594 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f137fe700 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 0x55e612a72c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1061929594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 msgr2=0x55e612a72c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 0x55e612a72c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1061929594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 msgr2=0x55e612a89270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a89270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f12ffd700 1 -- 172.21.15.99:0/1061929594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6127dfc00 con 0x55e612a78990 2023-08-25T09:05:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.296+0000 7f0f127fc700 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a89270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:05:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.297+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a78450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f04096610 tx=0x7f0f040cc3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a659bff6033958e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.297+0000 7f0ef9ffb700 1 -- 172.21.15.99:0/1061929594 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f040a1920 con 0x55e612a78990 2023-08-25T09:05:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.297+0000 7f0ef9ffb700 1 -- 172.21.15.99:0/1061929594 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f0409d9d0 con 0x55e612a78990 2023-08-25T09:05:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.297+0000 7f0ef9ffb700 1 -- 172.21.15.99:0/1061929594 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04099090 con 0x55e612a78990 2023-08-25T09:05:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.297+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1061929594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 msgr2=0x55e612a78450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.297+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a78450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f04096610 tx=0x7f0f040cc3b0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.298+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1061929594 shutdown_connections 2023-08-25T09:05:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.298+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 0x55e612a72c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.298+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a78450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.298+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/1061929594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a89270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.298+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1061929594 >> 172.21.15.99:0/1061929594 conn(0x55e6129ed080 msgr2=0x55e612a98b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.298+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1061929594 shutdown_connections 2023-08-25T09:05:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.298+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/1061929594 wait complete. 2023-08-25T09:05:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.298+0000 7f0f2cc57b80 1 -- start start 2023-08-25T09:05:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f2cc57b80 1 Processor -- start 2023-08-25T09:05:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a79880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a79dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f2cc57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 0x55e612a7a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e612a89640 con 0x55e612a7df40 2023-08-25T09:05:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e612a7a840 con 0x55e612a78990 2023-08-25T09:05:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f2cc57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e612a7aa00 con 0x55e612a7e860 2023-08-25T09:05:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f2cc57b80 10 client.0 _ll_register_callbacks cb 0x7f0f04099500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a79880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a79880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47482/0 (socket says 172.21.15.99:47482) 2023-08-25T09:05:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f12ffd700 1 -- 172.21.15.99:0/3246136678 learned_addr learned my addr 172.21.15.99:0/3246136678 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f137fe700 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 0x55e612a7a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.299+0000 7f0f127fc700 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a79dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f12ffd700 1 -- 172.21.15.99:0/3246136678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 msgr2=0x55e612a7a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 0x55e612a7a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f12ffd700 1 -- 172.21.15.99:0/3246136678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 msgr2=0x55e612a79dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a79dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f12ffd700 1 -- 172.21.15.99:0/3246136678 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e61284a5d0 con 0x55e612a78990 2023-08-25T09:05:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f127fc700 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a79dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f12ffd700 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a79880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0f04097ef0 tx=0x7f0f0409ade0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496739530a2d5ce2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0ef6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:05:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/3246136678 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f040a15b0 con 0x55e612a78990 2023-08-25T09:05:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/3246136678 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0f040a1920 con 0x55e612a78990 2023-08-25T09:05:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/3246136678 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e612a6b600 con 0x55e612a78990 2023-08-25T09:05:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/3246136678 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f0400fcd0 con 0x55e612a78990 2023-08-25T09:05:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f2cc57b80 10 client.25787 fetch_fsmap learned FSMap version 28 2023-08-25T09:05:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/3246136678 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e6127f6d80 con 0x55e612a78990 2023-08-25T09:05:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.300+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/3246136678 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f0407e080 con 0x55e612a78990 2023-08-25T09:05:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/3246136678 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0f04095990 con 0x55e612a78990 2023-08-25T09:05:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0f2cc57b80 10 client.25787 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T09:05:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0f2cc57b80 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0f2cc57b80 20 client.25787 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e612ae60b0 0x55e612ae84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0f2cc57b80 4 client.25787 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:05:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/3246136678 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e6129c3910 con 0x55e612ae60b0 2023-08-25T09:05:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0f127fc700 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e612ae60b0 0x55e612ae84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0ef6ff5700 10 client.25787 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.301+0000 7f0f127fc700 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e612ae60b0 0x55e612ae84a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa408a495d6f6894 server_cookie=59ea480c60989bf1 in_seq=0 out_seq=0 2023-08-25T09:05:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.411+0000 7f0ef6ff5700 1 -- 172.21.15.99:0/3246136678 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e6129c3910 con 0x55e612ae60b0 2023-08-25T09:05:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.411+0000 7f0ef6ff5700 10 client.25787 handle_command_reply: tid=0 2023-08-25T09:05:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.411+0000 7f0f2cc57b80 1 client.25787 shutdown 2023-08-25T09:05:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/3246136678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 msgr2=0x55e612a79880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a79880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0f04097ef0 tx=0x7f0f0409ade0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/3246136678 shutdown_connections 2023-08-25T09:05:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e612a7e860 0x55e612a7a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e612ae60b0 0x55e612ae84a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e612a78990 0x55e612a79880 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0ef6ff5700 0 client.25787 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 --2- 172.21.15.99:0/3246136678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e612a7df40 0x55e612a79dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/3246136678 >> 172.21.15.99:0/3246136678 conn(0x55e6129ed080 msgr2=0x55e612a72bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/3246136678 shutdown_connections 2023-08-25T09:05:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 1 -- 172.21.15.99:0/3246136678 wait complete. 2023-08-25T09:05:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.412+0000 7f0f2cc57b80 20 client.25787 trim_cache size 0 max 16384 2023-08-25T09:05:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:05:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:05:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:05:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:05:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 -- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0efc0a00c0 msgr2=0x7f0efc0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0efc0a00c0 0x7f0efc0a2580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009410 tx=0x7f0f0c009920 comp rx=0 tx=0).stop 2023-08-25T09:05:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 -- 172.21.15.99:0/1827074104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f14067d10 msgr2=0x7f0f140ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f14067d10 0x7f0f140ffd30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f040036c0 tx=0x7f0f0400f690 comp rx=0 tx=0).stop 2023-08-25T09:05:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 -- 172.21.15.99:0/1827074104 shutdown_connections 2023-08-25T09:05:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f0efc0a00c0 0x7f0efc0a2580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f141068d0 0x7f0f141a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f14067d10 0x7f0f140ffd30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 --2- 172.21.15.99:0/1827074104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f140686d0 0x7f0f14100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 -- 172.21.15.99:0/1827074104 >> 172.21.15.99:0/1827074104 conn(0x7f0f14075250 msgr2=0x7f0f140fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 -- 172.21.15.99:0/1827074104 shutdown_connections 2023-08-25T09:05:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:30.413+0000 7f0f193ff700 1 -- 172.21.15.99:0/1827074104 wait complete. 2023-08-25T09:05:30.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 189515 3.03 MB/sec execute 216 sec latency 65.419 ms 2023-08-25T09:05:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:31 smithi157 ceph-mon[130879]: pgmap v193: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2023-08-25T09:05:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:31 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:05:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:31 smithi181 ceph-mon[131296]: pgmap v193: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2023-08-25T09:05:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:31 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:05:31.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:31 smithi099 ceph-mon[97641]: pgmap v193: 97 pgs: 97 active+clean; 79 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2023-08-25T09:05:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:31 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:05:31.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 191363 3.06 MB/sec execute 217 sec latency 14.310 ms 2023-08-25T09:05:32.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 192490 3.07 MB/sec execute 218 sec latency 41.493 ms 2023-08-25T09:05:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:33 smithi157 ceph-mon[130879]: pgmap v194: 97 pgs: 97 active+clean; 78 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2023-08-25T09:05:33.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:33 smithi181 ceph-mon[131296]: pgmap v194: 97 pgs: 97 active+clean; 78 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2023-08-25T09:05:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:33 smithi099 ceph-mon[97641]: pgmap v194: 97 pgs: 97 active+clean; 78 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2023-08-25T09:05:33.918 INFO:tasks.workunit.client.0.smithi099.stdout: 1 194520 3.11 MB/sec execute 219 sec latency 41.167 ms 2023-08-25T09:05:34.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 195807 3.13 MB/sec execute 220 sec latency 46.070 ms 2023-08-25T09:05:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:35 smithi181 ceph-mon[131296]: pgmap v195: 97 pgs: 97 active+clean; 78 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 203 op/s 2023-08-25T09:05:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:35 smithi099 ceph-mon[97641]: pgmap v195: 97 pgs: 97 active+clean; 78 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 203 op/s 2023-08-25T09:05:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:35 smithi157 ceph-mon[130879]: pgmap v195: 97 pgs: 97 active+clean; 78 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 203 op/s 2023-08-25T09:05:35.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 197011 3.14 MB/sec execute 221 sec latency 70.761 ms 2023-08-25T09:05:36.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 198955 3.18 MB/sec execute 222 sec latency 14.656 ms 2023-08-25T09:05:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:37 smithi157 ceph-mon[130879]: pgmap v196: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 287 op/s 2023-08-25T09:05:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:37 smithi181 ceph-mon[131296]: pgmap v196: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 287 op/s 2023-08-25T09:05:37.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:37 smithi099 ceph-mon[97641]: pgmap v196: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 287 op/s 2023-08-25T09:05:37.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 200233 3.19 MB/sec execute 223 sec latency 28.755 ms 2023-08-25T09:05:38.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 202395 3.24 MB/sec execute 224 sec latency 48.574 ms 2023-08-25T09:05:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:39 smithi157 ceph-mon[130879]: pgmap v197: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-08-25T09:05:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:39 smithi181 ceph-mon[131296]: pgmap v197: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-08-25T09:05:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:39 smithi099 ceph-mon[97641]: pgmap v197: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-08-25T09:05:39.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 203571 3.25 MB/sec execute 225 sec latency 18.566 ms 2023-08-25T09:05:40.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 204954 3.26 MB/sec execute 226 sec latency 17.256 ms 2023-08-25T09:05:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:41 smithi157 ceph-mon[130879]: pgmap v198: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-08-25T09:05:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:41 smithi181 ceph-mon[131296]: pgmap v198: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-08-25T09:05:41.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:41 smithi099 ceph-mon[97641]: pgmap v198: 97 pgs: 97 active+clean; 84 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-08-25T09:05:41.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 206296 3.28 MB/sec execute 227 sec latency 10.775 ms 2023-08-25T09:05:42.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 207870 3.29 MB/sec execute 228 sec latency 18.223 ms 2023-08-25T09:05:43.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 -- 172.21.15.99:0/540392165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d01008a0 msgr2=0x7f86d0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 --2- 172.21.15.99:0/540392165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d01008a0 0x7f86d0100d20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86c40098f0 tx=0x7f86c4009c00 comp rx=0 tx=0).stop 2023-08-25T09:05:43.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 -- 172.21.15.99:0/540392165 shutdown_connections 2023-08-25T09:05:43.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 --2- 172.21.15.99:0/540392165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d0101260 0x7f86d010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 --2- 172.21.15.99:0/540392165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d01068b0 0x7f86d0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 --2- 172.21.15.99:0/540392165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d01008a0 0x7f86d0100d20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 -- 172.21.15.99:0/540392165 >> 172.21.15.99:0/540392165 conn(0x7f86d00fc370 msgr2=0x7f86d00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 -- 172.21.15.99:0/540392165 shutdown_connections 2023-08-25T09:05:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.567+0000 7f86d51ff700 1 -- 172.21.15.99:0/540392165 wait complete. 2023-08-25T09:05:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.569+0000 7f86d51ff700 1 Processor -- start 2023-08-25T09:05:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.569+0000 7f86d51ff700 1 -- start start 2023-08-25T09:05:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.569+0000 7f86d51ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d01008a0 0x7f86d019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.569+0000 7f86d51ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d0101260 0x7f86d019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.569+0000 7f86d51ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d01068b0 0x7f86d0196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.569+0000 7f86d51ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0196e80 con 0x7f86d01008a0 2023-08-25T09:05:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.569+0000 7f86d51ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0196ff0 con 0x7f86d0101260 2023-08-25T09:05:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.569+0000 7f86d51ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01971b0 con 0x7f86d01068b0 2023-08-25T09:05:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d01008a0 0x7f86d019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d0101260 0x7f86d019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d0101260 0x7f86d019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51222/0 (socket says 172.21.15.99:51222) 2023-08-25T09:05:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 -- 172.21.15.99:0/3034103688 learned_addr learned my addr 172.21.15.99:0/3034103688 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86cf59e700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d01068b0 0x7f86d0196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 -- 172.21.15.99:0/3034103688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d01068b0 msgr2=0x7f86d0196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d01068b0 0x7f86d0196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 -- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d01008a0 msgr2=0x7f86d019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d01008a0 0x7f86d019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 -- 172.21.15.99:0/3034103688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c00095f0 con 0x7f86d0101260 2023-08-25T09:05:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ced9d700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d01008a0 0x7f86d019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:05:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.570+0000 7f86ce59c700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d0101260 0x7f86d019cef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009810 tx=0x7f86c400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.571+0000 7f86b7fff700 1 -- 172.21.15.99:0/3034103688 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c401c070 con 0x7f86d0101260 2023-08-25T09:05:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.571+0000 7f86b7fff700 1 -- 172.21.15.99:0/3034103688 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c400bb20 con 0x7f86d0101260 2023-08-25T09:05:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.571+0000 7f86d51ff700 1 -- 172.21.15.99:0/3034103688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c4009520 con 0x7f86d0101260 2023-08-25T09:05:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.571+0000 7f86b7fff700 1 -- 172.21.15.99:0/3034103688 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4004930 con 0x7f86d0101260 2023-08-25T09:05:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.571+0000 7f86d51ff700 1 -- 172.21.15.99:0/3034103688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d01a37c0 con 0x7f86d0101260 2023-08-25T09:05:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.574+0000 7f86b7fff700 1 -- 172.21.15.99:0/3034103688 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f86c4020070 con 0x7f86d0101260 2023-08-25T09:05:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.575+0000 7f86e8a70b80 1 Processor -- start 2023-08-25T09:05:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.575+0000 7f86e8a70b80 1 -- start start 2023-08-25T09:05:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.575+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd63320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.575+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.575+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fcc5750 con 0x55e46fd5d990 2023-08-25T09:05:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fcb6720 con 0x55e46fd62f40 2023-08-25T09:05:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fcb4040 con 0x55e46fd63860 2023-08-25T09:05:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86b7fff700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f86b80a00c0 0x7f86b80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57054/0 (socket says 172.21.15.99:57054) 2023-08-25T09:05:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ced9d700 1 -- 172.21.15.99:0/2459662097 learned_addr learned my addr 172.21.15.99:0/2459662097 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ce59c700 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86cf59e700 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd63320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86cf59e700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f86b80a00c0 0x7f86b80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86b7fff700 1 -- 172.21.15.99:0/3034103688 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f86c40cc3f0 con 0x7f86d0101260 2023-08-25T09:05:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ced9d700 1 -- 172.21.15.99:0/2459662097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 msgr2=0x55e46fd63320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ced9d700 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd63320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ced9d700 1 -- 172.21.15.99:0/2459662097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 msgr2=0x55e46fd5ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ced9d700 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ced9d700 1 -- 172.21.15.99:0/2459662097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46fb2f5d0 con 0x55e46fd63860 2023-08-25T09:05:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86ce59c700 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.576+0000 7f86cf59e700 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd63320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.577+0000 7f86ced9d700 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5d450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86c0011780 tx=0x7f86c0011a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68b7ab67c1cf3613 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.577+0000 7f86cf59e700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f86b80a00c0 0x7f86b80a2580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f86d019d6b0 tx=0x7f86c800b040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.577+0000 7f86b57fa700 1 -- 172.21.15.99:0/2459662097 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0004020 con 0x55e46fd63860 2023-08-25T09:05:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.577+0000 7f86b57fa700 1 -- 172.21.15.99:0/2459662097 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c0003680 con 0x55e46fd63860 2023-08-25T09:05:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.577+0000 7f86b57fa700 1 -- 172.21.15.99:0/2459662097 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000f8e0 con 0x55e46fd63860 2023-08-25T09:05:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.577+0000 7f86e8a70b80 1 -- 172.21.15.99:0/2459662097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 msgr2=0x55e46fd5d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.577+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5d450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86c0011780 tx=0x7f86c0011a90 comp rx=0 tx=0).stop 2023-08-25T09:05:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 -- 172.21.15.99:0/2459662097 shutdown_connections 2023-08-25T09:05:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd63320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/2459662097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5d450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 -- 172.21.15.99:0/2459662097 >> 172.21.15.99:0/2459662097 conn(0x55e46fcd2080 msgr2=0x55e46fd57d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 -- 172.21.15.99:0/2459662097 shutdown_connections 2023-08-25T09:05:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 -- 172.21.15.99:0/2459662097 wait complete. 2023-08-25T09:05:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 -- start start 2023-08-25T09:05:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.578+0000 7f86e8a70b80 1 Processor -- start 2023-08-25T09:05:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd5d990 0x55e46fd695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd66850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd63860 0x55e46fd66e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fcaf930 con 0x55e46fd63860 2023-08-25T09:05:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fc4db50 con 0x55e46fd62f40 2023-08-25T09:05:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fc4d850 con 0x55e46fd5d990 2023-08-25T09:05:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86e8a70b80 10 client.0 _ll_register_callbacks cb 0x55e46fb97a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd5d990 0x55e46fd695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ce59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd66850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd5d990 0x55e46fd695d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57060/0 (socket says 172.21.15.99:57060) 2023-08-25T09:05:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ced9d700 1 -- 172.21.15.99:0/3733492353 learned_addr learned my addr 172.21.15.99:0/3733492353 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ce59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd66850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51244/0 (socket says 172.21.15.99:51244) 2023-08-25T09:05:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86cf59e700 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd63860 0x55e46fd66e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ce59c700 1 -- 172.21.15.99:0/3733492353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd5d990 msgr2=0x55e46fd695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ce59c700 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd5d990 0x55e46fd695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ce59c700 1 -- 172.21.15.99:0/3733492353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd63860 msgr2=0x55e46fd66e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ce59c700 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd63860 0x55e46fd66e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ce59c700 1 -- 172.21.15.99:0/3733492353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c0011440 con 0x55e46fd62f40 2023-08-25T09:05:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.579+0000 7f86ced9d700 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd5d990 0x55e46fd695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:05:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86ce59c700 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd66850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86c40c9aa0 tx=0x7f86c4095f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ec3c2b7513af339 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:05:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86b2ff5700 1 -- 172.21.15.99:0/3733492353 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4099f50 con 0x55e46fd62f40 2023-08-25T09:05:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86e8a70b80 1 -- 172.21.15.99:0/3733492353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e46fd2d160 con 0x55e46fd62f40 2023-08-25T09:05:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86b2ff5700 1 -- 172.21.15.99:0/3733492353 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f86c4099af0 con 0x55e46fd62f40 2023-08-25T09:05:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86b2ff5700 1 -- 172.21.15.99:0/3733492353 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c40c9f30 con 0x55e46fd62f40 2023-08-25T09:05:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86e8a70b80 10 client.25799 fetch_fsmap learned FSMap version 29 2023-08-25T09:05:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86e8a70b80 1 -- 172.21.15.99:0/3733492353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e46fadbd80 con 0x55e46fd62f40 2023-08-25T09:05:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86b2ff5700 1 -- 172.21.15.99:0/3733492353 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e46fd2d160 con 0x55e46fd62f40 2023-08-25T09:05:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.580+0000 7f86b2ff5700 1 -- 172.21.15.99:0/3733492353 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f86c40a13e0 con 0x55e46fd62f40 2023-08-25T09:05:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86e8a70b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 29 2023-08-25T09:05:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86e8a70b80 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86e8a70b80 20 client.25799 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e46fd76e10 0x55e46fd79230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86e8a70b80 4 client.25799 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:05:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86e8a70b80 1 -- 172.21.15.99:0/3733492353 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e46fc55b40 con 0x55e46fd76e10 2023-08-25T09:05:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86ced9d700 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e46fd76e10 0x55e46fd79230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86b2ff5700 10 client.25799 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.581+0000 7f86ced9d700 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e46fd76e10 0x55e46fd79230 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eadff7e080994b74 server_cookie=d64f0ead676072f8 in_seq=0 out_seq=0 2023-08-25T09:05:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.582+0000 7f86b2ff5700 1 -- 172.21.15.99:0/3733492353 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e46fc55b40 con 0x55e46fd76e10 2023-08-25T09:05:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.582+0000 7f86b2ff5700 10 client.25799 handle_command_reply: tid=0 2023-08-25T09:05:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.582+0000 7f86e8a70b80 1 client.25799 shutdown 2023-08-25T09:05:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.582+0000 7f86e8a70b80 1 -- 172.21.15.99:0/3733492353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 msgr2=0x55e46fd66850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.582+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd66850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86c40c9aa0 tx=0x7f86c4095f60 comp rx=0 tx=0).stop 2023-08-25T09:05:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.582+0000 7f86e8a70b80 1 -- 172.21.15.99:0/3733492353 shutdown_connections 2023-08-25T09:05:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd63860 0x55e46fd66e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd5d990 0x55e46fd695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e46fd76e10 0x55e46fd79230 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/3733492353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd62f40 0x55e46fd66850 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86b2ff5700 0 client.25799 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86e8a70b80 1 -- 172.21.15.99:0/3733492353 >> 172.21.15.99:0/3733492353 conn(0x55e46fcd2080 msgr2=0x55e46fd52820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86e8a70b80 1 -- 172.21.15.99:0/3733492353 shutdown_connections 2023-08-25T09:05:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86e8a70b80 1 -- 172.21.15.99:0/3733492353 wait complete. 2023-08-25T09:05:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.583+0000 7f86e8a70b80 20 client.25799 trim_cache size 0 max 16384 2023-08-25T09:05:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.598+0000 7f86e8a70b80 1 Processor -- start 2023-08-25T09:05:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.598+0000 7f86e8a70b80 1 -- start start 2023-08-25T09:05:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.598+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.598+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd62f40 0x55e46fd64350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.598+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd63860 0x55e46fd6e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.598+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fb318c0 con 0x55e46fd5d990 2023-08-25T09:05:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.598+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fd54700 con 0x55e46fd63860 2023-08-25T09:05:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.598+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fd503f0 con 0x55e46fd62f40 2023-08-25T09:05:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ce59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd62f40 0x55e46fd64350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86cf59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd63860 0x55e46fd6e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55622/0 (socket says 172.21.15.99:55622) 2023-08-25T09:05:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86cf59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd63860 0x55e46fd6e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51254/0 (socket says 172.21.15.99:51254) 2023-08-25T09:05:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 -- 172.21.15.99:0/454348655 learned_addr learned my addr 172.21.15.99:0/454348655 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ce59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd62f40 0x55e46fd64350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57072/0 (socket says 172.21.15.99:57072) 2023-08-25T09:05:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 -- 172.21.15.99:0/454348655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd62f40 msgr2=0x55e46fd64350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 --2- 172.21.15.99:0/454348655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd62f40 0x55e46fd64350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 -- 172.21.15.99:0/454348655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd63860 msgr2=0x55e46fd6e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 --2- 172.21.15.99:0/454348655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd63860 0x55e46fd6e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 -- 172.21.15.99:0/454348655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46fac4c00 con 0x55e46fd5d990 2023-08-25T09:05:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.599+0000 7f86ced9d700 1 --2- 172.21.15.99:0/454348655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5d450 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f86c00094b0 tx=0x7f86c00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e519d7d27679aa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86b5ffb700 1 -- 172.21.15.99:0/454348655 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0029b20 con 0x55e46fd5d990 2023-08-25T09:05:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86b5ffb700 1 -- 172.21.15.99:0/454348655 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c002d070 con 0x55e46fd5d990 2023-08-25T09:05:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86b5ffb700 1 -- 172.21.15.99:0/454348655 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00039e0 con 0x55e46fd5d990 2023-08-25T09:05:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 -- 172.21.15.99:0/454348655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 msgr2=0x55e46fd5d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/454348655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5d450 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f86c00094b0 tx=0x7f86c00058e0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 -- 172.21.15.99:0/454348655 shutdown_connections 2023-08-25T09:05:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/454348655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd63860 0x55e46fd6e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/454348655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd5d990 0x55e46fd5d450 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/454348655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd62f40 0x55e46fd64350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 -- 172.21.15.99:0/454348655 >> 172.21.15.99:0/454348655 conn(0x55e46fcd2080 msgr2=0x55e46fd63c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 -- 172.21.15.99:0/454348655 shutdown_connections 2023-08-25T09:05:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.600+0000 7f86e8a70b80 1 -- 172.21.15.99:0/454348655 wait complete. 2023-08-25T09:05:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 1 -- start start 2023-08-25T09:05:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 1 Processor -- start 2023-08-25T09:05:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd5d990 0x55e46fd5fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd62f40 0x55e46fd60210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fd5ef00 con 0x55e46fd62f40 2023-08-25T09:05:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fd5f070 con 0x55e46fd5d990 2023-08-25T09:05:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46fd5f230 con 0x55e46fd63860 2023-08-25T09:05:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86e8a70b80 10 client.0 _ll_register_callbacks cb 0x7f86c40998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.601+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd5d990 0x55e46fd5fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ced9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd5d990 0x55e46fd5fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51264/0 (socket says 172.21.15.99:51264) 2023-08-25T09:05:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86cf59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ced9d700 1 -- 172.21.15.99:0/583429401 learned_addr learned my addr 172.21.15.99:0/583429401 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ce59c700 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd62f40 0x55e46fd60210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ced9d700 1 -- 172.21.15.99:0/583429401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 msgr2=0x55e46fd5e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ced9d700 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ced9d700 1 -- 172.21.15.99:0/583429401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd62f40 msgr2=0x55e46fd60210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ced9d700 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd62f40 0x55e46fd60210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ced9d700 1 -- 172.21.15.99:0/583429401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e46fb2f5d0 con 0x55e46fd5d990 2023-08-25T09:05:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86ced9d700 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd5d990 0x55e46fd5fcd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86c0003fd0 tx=0x7f86c0029b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93deaada2fcd543c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:05:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86b2ff5700 1 -- 172.21.15.99:0/583429401 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0029d10 con 0x55e46fd5d990 2023-08-25T09:05:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86e8a70b80 1 -- 172.21.15.99:0/583429401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e46fd50600 con 0x55e46fd5d990 2023-08-25T09:05:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.602+0000 7f86b2ff5700 1 -- 172.21.15.99:0/583429401 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f86c0029150 con 0x55e46fd5d990 2023-08-25T09:05:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86b2ff5700 1 -- 172.21.15.99:0/583429401 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c0029e70 con 0x55e46fd5d990 2023-08-25T09:05:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86e8a70b80 10 client.25805 fetch_fsmap learned FSMap version 29 2023-08-25T09:05:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86e8a70b80 1 -- 172.21.15.99:0/583429401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e46fb021f0 con 0x55e46fd5d990 2023-08-25T09:05:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86b2ff5700 1 -- 172.21.15.99:0/583429401 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c0040020 con 0x55e46fd5d990 2023-08-25T09:05:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86b2ff5700 1 -- 172.21.15.99:0/583429401 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f86c0005340 con 0x55e46fd5d990 2023-08-25T09:05:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86e8a70b80 10 client.25805 fetch_fsmap finished waiting for FSMap version 29 2023-08-25T09:05:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86e8a70b80 10 client.25805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86e8a70b80 20 client.25805 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e46fdcfcf0 0x55e46fdd20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86e8a70b80 4 client.25805 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:05:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86e8a70b80 1 -- 172.21.15.99:0/583429401 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e46fca8910 con 0x55e46fdcfcf0 2023-08-25T09:05:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.603+0000 7f86ce59c700 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e46fdcfcf0 0x55e46fdd20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.604+0000 7f86ce59c700 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e46fdcfcf0 0x55e46fdd20e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4462e88661cc799 server_cookie=43fecd748acb4d85 in_seq=0 out_seq=0 2023-08-25T09:05:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.604+0000 7f86b2ff5700 10 client.25805 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:43.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:43 smithi157 ceph-mon[130879]: pgmap v199: 97 pgs: 97 active+clean; 89 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-08-25T09:05:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86b2ff5700 1 -- 172.21.15.99:0/583429401 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e46fca8910 con 0x55e46fdcfcf0 2023-08-25T09:05:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86b2ff5700 10 client.25805 handle_command_reply: tid=0 2023-08-25T09:05:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 client.25805 shutdown 2023-08-25T09:05:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 -- 172.21.15.99:0/583429401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd5d990 msgr2=0x55e46fd5fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd5d990 0x55e46fd5fcd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86c0003fd0 tx=0x7f86c0029b20 comp rx=0 tx=0).stop 2023-08-25T09:05:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 -- 172.21.15.99:0/583429401 shutdown_connections 2023-08-25T09:05:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e46fd63860 0x55e46fd5e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e46fdcfcf0 0x55e46fdd20e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e46fd5d990 0x55e46fd5fcd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 --2- 172.21.15.99:0/583429401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46fd62f40 0x55e46fd60210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86b2ff5700 0 client.25805 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 -- 172.21.15.99:0/583429401 >> 172.21.15.99:0/583429401 conn(0x55e46fcd2080 msgr2=0x55e46fd51f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.658+0000 7f86e8a70b80 1 -- 172.21.15.99:0/583429401 shutdown_connections 2023-08-25T09:05:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.659+0000 7f86e8a70b80 1 -- 172.21.15.99:0/583429401 wait complete. 2023-08-25T09:05:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.659+0000 7f86e8a70b80 20 client.25805 trim_cache size 0 max 16384 2023-08-25T09:05:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:05:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:05:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:05:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:05:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.659+0000 7f86d51ff700 1 -- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f86b80a00c0 msgr2=0x7f86b80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f86b80a00c0 0x7f86b80a2580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f86d019d6b0 tx=0x7f86c800b040 comp rx=0 tx=0).stop 2023-08-25T09:05:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 -- 172.21.15.99:0/3034103688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d0101260 msgr2=0x7f86d019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d0101260 0x7f86d019cef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009810 tx=0x7f86c400fa00 comp rx=0 tx=0).stop 2023-08-25T09:05:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 -- 172.21.15.99:0/3034103688 shutdown_connections 2023-08-25T09:05:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f86b80a00c0 0x7f86b80a2580 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d01068b0 0x7f86d0196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d01008a0 0x7f86d019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 --2- 172.21.15.99:0/3034103688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d0101260 0x7f86d019cef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 -- 172.21.15.99:0/3034103688 >> 172.21.15.99:0/3034103688 conn(0x7f86d00fc370 msgr2=0x7f86d00fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 -- 172.21.15.99:0/3034103688 shutdown_connections 2023-08-25T09:05:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:43.660+0000 7f86d51ff700 1 -- 172.21.15.99:0/3034103688 wait complete. 2023-08-25T09:05:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:43 smithi181 ceph-mon[131296]: pgmap v199: 97 pgs: 97 active+clean; 89 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-08-25T09:05:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:43 smithi099 ceph-mon[97641]: pgmap v199: 97 pgs: 97 active+clean; 89 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-08-25T09:05:43.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 209558 3.34 MB/sec execute 229 sec latency 38.459 ms 2023-08-25T09:05:44.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 210837 3.35 MB/sec execute 230 sec latency 18.373 ms 2023-08-25T09:05:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:45 smithi181 ceph-mon[131296]: pgmap v200: 97 pgs: 97 active+clean; 89 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-08-25T09:05:45.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:45 smithi099 ceph-mon[97641]: pgmap v200: 97 pgs: 97 active+clean; 89 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-08-25T09:05:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:45 smithi157 ceph-mon[130879]: pgmap v200: 97 pgs: 97 active+clean; 89 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-08-25T09:05:45.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 213060 3.39 MB/sec execute 231 sec latency 14.813 ms 2023-08-25T09:05:46.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 214221 3.40 MB/sec execute 232 sec latency 21.101 ms 2023-08-25T09:05:47.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:47 smithi157 ceph-mon[130879]: pgmap v201: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2023-08-25T09:05:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:47 smithi181 ceph-mon[131296]: pgmap v201: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2023-08-25T09:05:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:47 smithi099 ceph-mon[97641]: pgmap v201: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2023-08-25T09:05:47.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 215656 3.41 MB/sec execute 233 sec latency 15.402 ms 2023-08-25T09:05:48.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 216986 3.43 MB/sec execute 234 sec latency 55.630 ms 2023-08-25T09:05:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:49 smithi157 ceph-mon[130879]: pgmap v202: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-08-25T09:05:49.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:49 smithi181 ceph-mon[131296]: pgmap v202: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-08-25T09:05:49.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:49 smithi099 ceph-mon[97641]: pgmap v202: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-08-25T09:05:49.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 218853 3.45 MB/sec execute 235 sec latency 22.608 ms 2023-08-25T09:05:50.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 220472 3.48 MB/sec execute 236 sec latency 11.186 ms 2023-08-25T09:05:51.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:51 smithi157 ceph-mon[130879]: pgmap v203: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-08-25T09:05:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:51 smithi181 ceph-mon[131296]: pgmap v203: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-08-25T09:05:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:51 smithi099 ceph-mon[97641]: pgmap v203: 97 pgs: 97 active+clean; 83 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-08-25T09:05:51.919 INFO:tasks.workunit.client.0.smithi099.stdout: 1 221791 3.49 MB/sec execute 237 sec latency 20.845 ms 2023-08-25T09:05:52.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 223759 3.53 MB/sec execute 238 sec latency 13.061 ms 2023-08-25T09:05:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:53 smithi157 ceph-mon[130879]: pgmap v204: 97 pgs: 97 active+clean; 89 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 320 op/s 2023-08-25T09:05:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:53 smithi181 ceph-mon[131296]: pgmap v204: 97 pgs: 97 active+clean; 89 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 320 op/s 2023-08-25T09:05:53.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:53 smithi099 ceph-mon[97641]: pgmap v204: 97 pgs: 97 active+clean; 89 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 320 op/s 2023-08-25T09:05:53.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 224966 3.54 MB/sec execute 239 sec latency 33.731 ms 2023-08-25T09:05:54.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 227202 3.58 MB/sec execute 240 sec latency 15.092 ms 2023-08-25T09:05:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:55 smithi181 ceph-mon[131296]: pgmap v205: 97 pgs: 97 active+clean; 89 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2023-08-25T09:05:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:55 smithi099 ceph-mon[97641]: pgmap v205: 97 pgs: 97 active+clean; 89 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2023-08-25T09:05:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:55 smithi157 ceph-mon[130879]: pgmap v205: 97 pgs: 97 active+clean; 89 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2023-08-25T09:05:55.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 228441 3.59 MB/sec execute 241 sec latency 13.352 ms 2023-08-25T09:05:56.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:56 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:05:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:56 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:05:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:56 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:05:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.807+0000 7f001b7ff700 1 -- 172.21.15.99:0/3749200669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014108ba0 msgr2=0x7f0014108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.807+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3749200669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014108ba0 0x7f0014108f80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0010009990 tx=0x7f0010009ca0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.807+0000 7f001b7ff700 1 -- 172.21.15.99:0/3749200669 shutdown_connections 2023-08-25T09:05:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.807+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3749200669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00141034c0 0x7f001410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.807+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3749200669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014108ba0 0x7f0014108f80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.808+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3749200669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0014102b00 0x7f0014102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.808+0000 7f001b7ff700 1 -- 172.21.15.99:0/3749200669 >> 172.21.15.99:0/3749200669 conn(0x7f00140fe5d0 msgr2=0x7f00141009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.808+0000 7f001b7ff700 1 -- 172.21.15.99:0/3749200669 shutdown_connections 2023-08-25T09:05:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.808+0000 7f001b7ff700 1 -- 172.21.15.99:0/3749200669 wait complete. 2023-08-25T09:05:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001b7ff700 1 Processor -- start 2023-08-25T09:05:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001b7ff700 1 -- start start 2023-08-25T09:05:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001b7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014102b00 0x7f001419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001b7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00141034c0 0x7f001419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001b7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0014108ba0 0x7f00141989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001b7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0014199050 con 0x7f0014102b00 2023-08-25T09:05:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001b7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00141991c0 con 0x7f00141034c0 2023-08-25T09:05:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001b7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0014199380 con 0x7f0014108ba0 2023-08-25T09:05:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001959b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014102b00 0x7f001419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001959b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014102b00 0x7f001419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45688/0 (socket says 172.21.15.99:45688) 2023-08-25T09:05:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.809+0000 7f001959b700 1 -- 172.21.15.99:0/3020039032 learned_addr learned my addr 172.21.15.99:0/3020039032 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f0018d9a700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00141034c0 0x7f001419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f0019d9c700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0014108ba0 0x7f00141989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f001959b700 1 -- 172.21.15.99:0/3020039032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0014108ba0 msgr2=0x7f00141989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f001959b700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0014108ba0 0x7f00141989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f001959b700 1 -- 172.21.15.99:0/3020039032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00141034c0 msgr2=0x7f001419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f001959b700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00141034c0 0x7f001419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f001959b700 1 -- 172.21.15.99:0/3020039032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00100095f0 con 0x7f0014102b00 2023-08-25T09:05:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f0019d9c700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0014108ba0 0x7f00141989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:05:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f001959b700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014102b00 0x7f001419eb80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0010005910 tx=0x7f0010004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f000a7fc700 1 -- 172.21.15.99:0/3020039032 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001001d070 con 0x7f0014102b00 2023-08-25T09:05:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f001b7ff700 1 -- 172.21.15.99:0/3020039032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0014199600 con 0x7f0014102b00 2023-08-25T09:05:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f000a7fc700 1 -- 172.21.15.99:0/3020039032 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0010022470 con 0x7f0014102b00 2023-08-25T09:05:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.810+0000 7f000a7fc700 1 -- 172.21.15.99:0/3020039032 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010017650 con 0x7f0014102b00 2023-08-25T09:05:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.811+0000 7f001b7ff700 1 -- 172.21.15.99:0/3020039032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00141a5b70 con 0x7f0014102b00 2023-08-25T09:05:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.814+0000 7f000a7fc700 1 -- 172.21.15.99:0/3020039032 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f00100225e0 con 0x7f0014102b00 2023-08-25T09:05:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.815+0000 7f002f108b80 1 Processor -- start 2023-08-25T09:05:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.815+0000 7f000a7fc700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f00000a00c0 0x7f00000a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.815+0000 7f000a7fc700 1 -- 172.21.15.99:0/3020039032 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f00100cc920 con 0x7f0014102b00 2023-08-25T09:05:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.815+0000 7f002f108b80 1 -- start start 2023-08-25T09:05:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.815+0000 7f0018d9a700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f00000a00c0 0x7f00000a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f002f108b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3daf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f002f108b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3da9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f002f108b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f002f108b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3d11750 con 0x5570c3daef40 2023-08-25T09:05:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f002f108b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3d02720 con 0x5570c3da9990 2023-08-25T09:05:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f002f108b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3d00040 con 0x5570c3daf860 2023-08-25T09:05:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0018d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3da9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f001959b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3daf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0018d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3da9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57532/0 (socket says 172.21.15.99:57532) 2023-08-25T09:05:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0019d9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0018d9a700 1 -- 172.21.15.99:0/3630408118 learned_addr learned my addr 172.21.15.99:0/3630408118 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0019d9c700 1 -- 172.21.15.99:0/3630408118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 msgr2=0x5570c3da9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0019d9c700 1 --2- 172.21.15.99:0/3630408118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3da9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0019d9c700 1 -- 172.21.15.99:0/3630408118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 msgr2=0x5570c3daf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0019d9c700 1 --2- 172.21.15.99:0/3630408118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3daf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0019d9c700 1 -- 172.21.15.99:0/3630408118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c3b7b5d0 con 0x5570c3da9990 2023-08-25T09:05:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.816+0000 7f0018d9a700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f00000a00c0 0x7f00000a2580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00040115e0 tx=0x7f00040119a0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.817+0000 7f0019d9c700 1 --2- 172.21.15.99:0/3630408118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f000c009370 tx=0x7f000c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3270837627fdd90 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.817+0000 7effff7fe700 1 -- 172.21.15.99:0/3630408118 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00e070 con 0x5570c3da9990 2023-08-25T09:05:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.817+0000 7effff7fe700 1 -- 172.21.15.99:0/3630408118 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c00f040 con 0x5570c3da9990 2023-08-25T09:05:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.817+0000 7effff7fe700 1 -- 172.21.15.99:0/3630408118 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c004440 con 0x5570c3da9990 2023-08-25T09:05:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 -- 172.21.15.99:0/3630408118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 msgr2=0x5570c3da9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 --2- 172.21.15.99:0/3630408118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f000c009370 tx=0x7f000c009920 comp rx=0 tx=0).stop 2023-08-25T09:05:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 -- 172.21.15.99:0/3630408118 shutdown_connections 2023-08-25T09:05:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 --2- 172.21.15.99:0/3630408118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 --2- 172.21.15.99:0/3630408118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3daf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 --2- 172.21.15.99:0/3630408118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3da9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 -- 172.21.15.99:0/3630408118 >> 172.21.15.99:0/3630408118 conn(0x5570c3d1e080 msgr2=0x5570c3da3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 -- 172.21.15.99:0/3630408118 shutdown_connections 2023-08-25T09:05:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 -- 172.21.15.99:0/3630408118 wait complete. 2023-08-25T09:05:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.818+0000 7f002f108b80 1 -- start start 2023-08-25T09:05:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f002f108b80 1 Processor -- start 2023-08-25T09:05:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f002f108b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f002f108b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f002f108b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3db2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f002f108b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3cfb930 con 0x5570c3daef40 2023-08-25T09:05:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f002f108b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3c99b50 con 0x5570c3da9990 2023-08-25T09:05:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f002f108b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3c99850 con 0x5570c3daf860 2023-08-25T09:05:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f002f108b80 10 client.0 _ll_register_callbacks cb 0x5570c3be3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f001959b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f001959b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40082/0 (socket says 172.21.15.99:40082) 2023-08-25T09:05:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f001959b700 1 -- 172.21.15.99:0/883940679 learned_addr learned my addr 172.21.15.99:0/883940679 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f0018d9a700 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.819+0000 7f0019d9c700 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3db2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f001959b700 1 -- 172.21.15.99:0/883940679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 msgr2=0x5570c3db2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f001959b700 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3db2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f001959b700 1 -- 172.21.15.99:0/883940679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 msgr2=0x5570c3db2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f001959b700 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f001959b700 1 -- 172.21.15.99:0/883940679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c009050 con 0x5570c3da9990 2023-08-25T09:05:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f001959b700 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db5580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0010009350 tx=0x7f0010099be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9fdb5bfd794279 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7efffcff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:05:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7efffcff9700 1 -- 172.21.15.99:0/883940679 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100b1070 con 0x5570c3da9990 2023-08-25T09:05:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f002f108b80 1 -- 172.21.15.99:0/883940679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570c3d79160 con 0x5570c3da9990 2023-08-25T09:05:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7efffcff9700 1 -- 172.21.15.99:0/883940679 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f001009a020 con 0x5570c3da9990 2023-08-25T09:05:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7efffcff9700 1 -- 172.21.15.99:0/883940679 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f001009de50 con 0x5570c3da9990 2023-08-25T09:05:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f002f108b80 10 client.25817 fetch_fsmap learned FSMap version 30 2023-08-25T09:05:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.820+0000 7f002f108b80 1 -- 172.21.15.99:0/883940679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5570c3b27d80 con 0x5570c3da9990 2023-08-25T09:05:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7efffcff9700 1 -- 172.21.15.99:0/883940679 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570c3d79160 con 0x5570c3da9990 2023-08-25T09:05:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7efffcff9700 1 -- 172.21.15.99:0/883940679 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f00100a1910 con 0x5570c3da9990 2023-08-25T09:05:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7f002f108b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 30 2023-08-25T09:05:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7f002f108b80 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7f002f108b80 20 client.25817 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7f002f108b80 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570c3dc2dd0 0x5570c3dc51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7f002f108b80 4 client.25817 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:05:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7f002f108b80 1 -- 172.21.15.99:0/883940679 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570c3ca1b40 con 0x5570c3dc2dd0 2023-08-25T09:05:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.821+0000 7f0018d9a700 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570c3dc2dd0 0x5570c3dc51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.822+0000 7efffcff9700 10 client.25817 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.822+0000 7f0018d9a700 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570c3dc2dd0 0x5570c3dc51f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd98c32f3a30d33 server_cookie=d81df3a04c56fc11 in_seq=0 out_seq=0 2023-08-25T09:05:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.822+0000 7efffcff9700 1 -- 172.21.15.99:0/883940679 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5570c3ca1b40 con 0x5570c3dc2dd0 2023-08-25T09:05:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.822+0000 7efffcff9700 10 client.25817 handle_command_reply: tid=0 2023-08-25T09:05:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.822+0000 7f002f108b80 1 client.25817 shutdown 2023-08-25T09:05:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 -- 172.21.15.99:0/883940679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 msgr2=0x5570c3db5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db5580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0010009350 tx=0x7f0010099be0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 -- 172.21.15.99:0/883940679 shutdown_connections 2023-08-25T09:05:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3db2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570c3dc2dd0 0x5570c3dc51f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db5580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7efffcff9700 0 client.25817 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 --2- 172.21.15.99:0/883940679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 -- 172.21.15.99:0/883940679 >> 172.21.15.99:0/883940679 conn(0x5570c3d1e080 msgr2=0x5570c3d9e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 -- 172.21.15.99:0/883940679 shutdown_connections 2023-08-25T09:05:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 1 -- 172.21.15.99:0/883940679 wait complete. 2023-08-25T09:05:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.823+0000 7f002f108b80 20 client.25817 trim_cache size 0 max 16384 2023-08-25T09:05:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.838+0000 7f002f108b80 1 Processor -- start 2023-08-25T09:05:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.838+0000 7f002f108b80 1 -- start start 2023-08-25T09:05:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.838+0000 7f002f108b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.838+0000 7f002f108b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daef40 0x5570c3dba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.838+0000 7f002f108b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daf860 0x5570c3d9e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.838+0000 7f002f108b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3b7d8c0 con 0x5570c3daf860 2023-08-25T09:05:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.838+0000 7f002f108b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3da4fb0 con 0x5570c3da9990 2023-08-25T09:05:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.838+0000 7f002f108b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3da4a60 con 0x5570c3daef40 2023-08-25T09:05:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daf860 0x5570c3d9e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daf860 0x5570c3d9e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45718/0 (socket says 172.21.15.99:45718) 2023-08-25T09:05:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 -- 172.21.15.99:0/50378920 learned_addr learned my addr 172.21.15.99:0/50378920 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0018d9a700 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daef40 0x5570c3dba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f001959b700 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 -- 172.21.15.99:0/50378920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daef40 msgr2=0x5570c3dba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daef40 0x5570c3dba2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 -- 172.21.15.99:0/50378920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 msgr2=0x5570c3da9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 -- 172.21.15.99:0/50378920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c3b10c00 con 0x5570c3daf860 2023-08-25T09:05:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0018d9a700 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daef40 0x5570c3dba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:05:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.839+0000 7f0019d9c700 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daf860 0x5570c3d9e6a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f000c009340 tx=0x7f000c009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa4840a5523006d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7efffffff700 1 -- 172.21.15.99:0/50378920 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00e040 con 0x5570c3daf860 2023-08-25T09:05:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7efffffff700 1 -- 172.21.15.99:0/50378920 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c005040 con 0x5570c3daf860 2023-08-25T09:05:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7efffffff700 1 -- 172.21.15.99:0/50378920 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0256a0 con 0x5570c3daf860 2023-08-25T09:05:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 -- 172.21.15.99:0/50378920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daf860 msgr2=0x5570c3d9e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daf860 0x5570c3d9e6a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f000c009340 tx=0x7f000c009600 comp rx=0 tx=0).stop 2023-08-25T09:05:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 -- 172.21.15.99:0/50378920 shutdown_connections 2023-08-25T09:05:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daf860 0x5570c3d9e6a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3da9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 --2- 172.21.15.99:0/50378920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daef40 0x5570c3dba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 -- 172.21.15.99:0/50378920 >> 172.21.15.99:0/50378920 conn(0x5570c3d1e080 msgr2=0x5570c3d9df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 -- 172.21.15.99:0/50378920 shutdown_connections 2023-08-25T09:05:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.840+0000 7f002f108b80 1 -- 172.21.15.99:0/50378920 wait complete. 2023-08-25T09:05:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.841+0000 7f002f108b80 1 -- start start 2023-08-25T09:05:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.841+0000 7f002f108b80 1 Processor -- start 2023-08-25T09:05:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f002f108b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f002f108b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f002f108b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3db0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f002f108b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3db16e0 con 0x5570c3daef40 2023-08-25T09:05:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f002f108b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3db1850 con 0x5570c3da9990 2023-08-25T09:05:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f002f108b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c3db1a10 con 0x5570c3daf860 2023-08-25T09:05:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f002f108b80 10 client.0 _ll_register_callbacks cb 0x5570c3be3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:05:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f0018d9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f001959b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f001959b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40096/0 (socket says 172.21.15.99:40096) 2023-08-25T09:05:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f001959b700 1 -- 172.21.15.99:0/3269296004 learned_addr learned my addr 172.21.15.99:0/3269296004 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:05:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f0018d9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45720/0 (socket says 172.21.15.99:45720) 2023-08-25T09:05:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.842+0000 7f0019d9c700 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3db0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f001959b700 1 -- 172.21.15.99:0/3269296004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 msgr2=0x5570c3db0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f001959b700 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3db0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f001959b700 1 -- 172.21.15.99:0/3269296004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 msgr2=0x5570c3db26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:05:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f001959b700 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f001959b700 1 -- 172.21.15.99:0/3269296004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00040042a0 con 0x5570c3da9990 2023-08-25T09:05:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f0018d9a700 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:05:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f001959b700 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db21a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f00100a1c70 tx=0x7f0010099740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a14ada1bed483a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:05:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7efffcff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:05:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7efffcff9700 1 -- 172.21.15.99:0/3269296004 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100b1070 con 0x5570c3da9990 2023-08-25T09:05:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7efffcff9700 1 -- 172.21.15.99:0/3269296004 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f001009a020 con 0x5570c3da9990 2023-08-25T09:05:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f002f108b80 1 -- 172.21.15.99:0/3269296004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570c3d9c600 con 0x5570c3da9990 2023-08-25T09:05:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7efffcff9700 1 -- 172.21.15.99:0/3269296004 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f001009de50 con 0x5570c3da9990 2023-08-25T09:05:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f002f108b80 10 client.25823 fetch_fsmap learned FSMap version 30 2023-08-25T09:05:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.843+0000 7f002f108b80 1 -- 172.21.15.99:0/3269296004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5570c3b7b5d0 con 0x5570c3da9990 2023-08-25T09:05:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7efffcff9700 1 -- 172.21.15.99:0/3269296004 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570c3d9c600 con 0x5570c3da9990 2023-08-25T09:05:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7efffcff9700 1 -- 172.21.15.99:0/3269296004 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f001000fd20 con 0x5570c3da9990 2023-08-25T09:05:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7f002f108b80 10 client.25823 fetch_fsmap finished waiting for FSMap version 30 2023-08-25T09:05:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7f002f108b80 10 client.25823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:05:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7f002f108b80 20 client.25823 populate_metadata read hostname 'smithi099' 2023-08-25T09:05:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7f002f108b80 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570c3e17810 0x5570c3e19c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:05:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7f002f108b80 4 client.25823 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:05:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7f002f108b80 1 -- 172.21.15.99:0/3269296004 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5570c3cf4910 con 0x5570c3e17810 2023-08-25T09:05:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7f0018d9a700 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570c3e17810 0x5570c3e19c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:05:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7f0018d9a700 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570c3e17810 0x5570c3e19c00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a3433aee9c089b server_cookie=afb1ceaa4629be32 in_seq=0 out_seq=0 2023-08-25T09:05:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.844+0000 7efffcff9700 10 client.25823 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7efffcff9700 1 -- 172.21.15.99:0/3269296004 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5570c3cf4910 con 0x5570c3e17810 2023-08-25T09:05:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7efffcff9700 10 client.25823 handle_command_reply: tid=0 2023-08-25T09:05:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 client.25823 shutdown 2023-08-25T09:05:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 -- 172.21.15.99:0/3269296004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 msgr2=0x5570c3db21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db21a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f00100a1c70 tx=0x7f0010099740 comp rx=0 tx=0).stop 2023-08-25T09:05:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 -- 172.21.15.99:0/3269296004 shutdown_connections 2023-08-25T09:05:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5570c3daf860 0x5570c3db0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5570c3e17810 0x5570c3e19c00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5570c3da9990 0x5570c3db21a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 --2- 172.21.15.99:0/3269296004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570c3daef40 0x5570c3db26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7efffcff9700 0 client.25823 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:05:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.907+0000 7f002f108b80 1 -- 172.21.15.99:0/3269296004 >> 172.21.15.99:0/3269296004 conn(0x5570c3d1e080 msgr2=0x5570c3db0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f002f108b80 1 -- 172.21.15.99:0/3269296004 shutdown_connections 2023-08-25T09:05:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f002f108b80 1 -- 172.21.15.99:0/3269296004 wait complete. 2023-08-25T09:05:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f002f108b80 20 client.25823 trim_cache size 0 max 16384 2023-08-25T09:05:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:05:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:05:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:05:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:05:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 -- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f00000a00c0 msgr2=0x7f00000a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f00000a00c0 0x7f00000a2580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00040115e0 tx=0x7f00040119a0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 -- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014102b00 msgr2=0x7f001419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:05:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014102b00 0x7f001419eb80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0010005910 tx=0x7f0010004710 comp rx=0 tx=0).stop 2023-08-25T09:05:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 -- 172.21.15.99:0/3020039032 shutdown_connections 2023-08-25T09:05:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f00000a00c0 0x7f00000a2580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0014108ba0 0x7f00141989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0014102b00 0x7f001419eb80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 --2- 172.21.15.99:0/3020039032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00141034c0 0x7f001419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:05:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.908+0000 7f001b7ff700 1 -- 172.21.15.99:0/3020039032 >> 172.21.15.99:0/3020039032 conn(0x7f00140fe5d0 msgr2=0x7f00140feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:05:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.909+0000 7f001b7ff700 1 -- 172.21.15.99:0/3020039032 shutdown_connections 2023-08-25T09:05:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:05:56.909+0000 7f001b7ff700 1 -- 172.21.15.99:0/3020039032 wait complete. 2023-08-25T09:05:56.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 228748 3.58 MB/sec execute 242 sec latency 763.215 ms 2023-08-25T09:05:57.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:57 smithi181 ceph-mon[131296]: pgmap v206: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 318 op/s 2023-08-25T09:05:57.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:57 smithi099 ceph-mon[97641]: pgmap v206: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 318 op/s 2023-08-25T09:05:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:57 smithi157 ceph-mon[130879]: pgmap v206: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 318 op/s 2023-08-25T09:05:57.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 228946 3.56 MB/sec execute 243 sec latency 1666.705 ms 2023-08-25T09:05:58.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 230912 3.60 MB/sec execute 244 sec latency 33.501 ms 2023-08-25T09:05:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:05:59 smithi157 ceph-mon[130879]: pgmap v207: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:05:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:05:59 smithi181 ceph-mon[131296]: pgmap v207: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:05:59.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:05:59 smithi099 ceph-mon[97641]: pgmap v207: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:05:59.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 232263 3.61 MB/sec execute 245 sec latency 23.151 ms 2023-08-25T09:06:00.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 232296 3.60 MB/sec execute 246 sec latency 972.406 ms 2023-08-25T09:06:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:01 smithi157 ceph-mon[130879]: pgmap v208: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:06:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:01 smithi181 ceph-mon[131296]: pgmap v208: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:06:01.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:01 smithi099 ceph-mon[97641]: pgmap v208: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:06:01.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 232296 3.58 MB/sec execute 247 sec latency 1972.514 ms 2023-08-25T09:06:02.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 232479 3.57 MB/sec execute 248 sec latency 2871.564 ms 2023-08-25T09:06:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:03 smithi157 ceph-mon[130879]: pgmap v209: 97 pgs: 97 active+clean; 89 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2023-08-25T09:06:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:03 smithi181 ceph-mon[131296]: pgmap v209: 97 pgs: 97 active+clean; 89 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2023-08-25T09:06:03.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:03 smithi099 ceph-mon[97641]: pgmap v209: 97 pgs: 97 active+clean; 89 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2023-08-25T09:06:03.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 234156 3.60 MB/sec execute 249 sec latency 24.742 ms 2023-08-25T09:06:04.920 INFO:tasks.workunit.client.0.smithi099.stdout: 1 235255 3.60 MB/sec execute 250 sec latency 22.030 ms 2023-08-25T09:06:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:05 smithi181 ceph-mon[131296]: pgmap v210: 97 pgs: 97 active+clean; 89 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2023-08-25T09:06:05.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:05 smithi099 ceph-mon[97641]: pgmap v210: 97 pgs: 97 active+clean; 89 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2023-08-25T09:06:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:05 smithi157 ceph-mon[130879]: pgmap v210: 97 pgs: 97 active+clean; 89 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2023-08-25T09:06:05.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 235839 3.60 MB/sec execute 251 sec latency 662.922 ms 2023-08-25T09:06:06.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 235839 3.59 MB/sec execute 252 sec latency 1663.037 ms 2023-08-25T09:06:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:07 smithi157 ceph-mon[130879]: pgmap v211: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2023-08-25T09:06:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:07 smithi181 ceph-mon[131296]: pgmap v211: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2023-08-25T09:06:07.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:07 smithi099 ceph-mon[97641]: pgmap v211: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2023-08-25T09:06:07.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 235956 3.57 MB/sec execute 253 sec latency 2567.680 ms 2023-08-25T09:06:08.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 237510 3.59 MB/sec execute 254 sec latency 27.999 ms 2023-08-25T09:06:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:09 smithi157 ceph-mon[130879]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-08-25T09:06:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:09 smithi157 ceph-mon[130879]: osdmap e89: 12 total, 12 up, 12 in 2023-08-25T09:06:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:09 smithi181 ceph-mon[131296]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-08-25T09:06:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:09 smithi181 ceph-mon[131296]: osdmap e89: 12 total, 12 up, 12 in 2023-08-25T09:06:09.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:09 smithi099 ceph-mon[97641]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-08-25T09:06:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:09 smithi099 ceph-mon[97641]: osdmap e89: 12 total, 12 up, 12 in 2023-08-25T09:06:09.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 238570 3.60 MB/sec execute 255 sec latency 23.121 ms 2023-08-25T09:06:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 -- 172.21.15.99:0/1797602880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 msgr2=0x7f7608073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 --2- 172.21.15.99:0/1797602880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 0x7f7608073130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009990 tx=0x7f75f8009ca0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 -- 172.21.15.99:0/1797602880 shutdown_connections 2023-08-25T09:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 --2- 172.21.15.99:0/1797602880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760810d0f0 0x7f760810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 --2- 172.21.15.99:0/1797602880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 0x7f7608073130 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 --2- 172.21.15.99:0/1797602880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7608073670 0x7f760810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 -- 172.21.15.99:0/1797602880 >> 172.21.15.99:0/1797602880 conn(0x7f76080fc380 msgr2=0x7f76080fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 -- 172.21.15.99:0/1797602880 shutdown_connections 2023-08-25T09:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.060+0000 7f760c9ff700 1 -- 172.21.15.99:0/1797602880 wait complete. 2023-08-25T09:06:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.061+0000 7f760c9ff700 1 Processor -- start 2023-08-25T09:06:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.061+0000 7f760c9ff700 1 -- start start 2023-08-25T09:06:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.061+0000 7f760c9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 0x7f760819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.061+0000 7f760c9ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7608073670 0x7f760819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.061+0000 7f760c9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760810d0f0 0x7f7608198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.061+0000 7f760c9ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7608199100 con 0x7f7608073670 2023-08-25T09:06:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.061+0000 7f760c9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7608199270 con 0x7f760810d0f0 2023-08-25T09:06:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.061+0000 7f760c9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7608199430 con 0x7f7608072d50 2023-08-25T09:06:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 0x7f760819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f7606d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760810d0f0 0x7f7608198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 0x7f760819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:47078/0 (socket says 172.21.15.99:47078) 2023-08-25T09:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 -- 172.21.15.99:0/3790451298 learned_addr learned my addr 172.21.15.99:0/3790451298 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f7605d9b700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7608073670 0x7f760819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 -- 172.21.15.99:0/3790451298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760810d0f0 msgr2=0x7f7608198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760810d0f0 0x7f7608198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 -- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7608073670 msgr2=0x7f760819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7608073670 0x7f760819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 -- 172.21.15.99:0/3790451298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc009520 con 0x7f7608072d50 2023-08-25T09:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f760659c700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 0x7f760819ec30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f75f8005910 tx=0x7f75f8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.062+0000 7f75f77fe700 1 -- 172.21.15.99:0/3790451298 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f801d070 con 0x7f7608072d50 2023-08-25T09:06:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.063+0000 7f760c9ff700 1 -- 172.21.15.99:0/3790451298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f80095f0 con 0x7f7608072d50 2023-08-25T09:06:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.063+0000 7f75f77fe700 1 -- 172.21.15.99:0/3790451298 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75f8022470 con 0x7f7608072d50 2023-08-25T09:06:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.063+0000 7f75f77fe700 1 -- 172.21.15.99:0/3790451298 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8017650 con 0x7f7608072d50 2023-08-25T09:06:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.063+0000 7f760c9ff700 1 -- 172.21.15.99:0/3790451298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76081a5a40 con 0x7f7608072d50 2023-08-25T09:06:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.066+0000 7f75f77fe700 1 -- 172.21.15.99:0/3790451298 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f75f80225e0 con 0x7f7608072d50 2023-08-25T09:06:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.067+0000 7f76203a5b80 1 Processor -- start 2023-08-25T09:06:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.067+0000 7f75f77fe700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75f00a00c0 0x7f75f00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.067+0000 7f75f77fe700 1 -- 172.21.15.99:0/3790451298 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f75f80cca80 con 0x7f7608072d50 2023-08-25T09:06:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.067+0000 7f76203a5b80 1 -- start start 2023-08-25T09:06:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.067+0000 7f7606d9d700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75f00a00c0 0x7f75f00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c3f40 0x5647c68c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68be450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68be990 0x5647c68bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f76203a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6826750 con 0x5647c68c3f40 2023-08-25T09:06:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f76203a5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6817720 con 0x5647c68be990 2023-08-25T09:06:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f76203a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6815040 con 0x5647c68c4860 2023-08-25T09:06:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f760659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c3f40 0x5647c68c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f760659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c3f40 0x5647c68c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43720/0 (socket says 172.21.15.99:43720) 2023-08-25T09:06:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f760659c700 1 -- 172.21.15.99:0/1295348460 learned_addr learned my addr 172.21.15.99:0/1295348460 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f7605d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68be450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f7606d9d700 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68be990 0x5647c68bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f7606d9d700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75f00a00c0 0x7f75f00a2580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f760819f930 tx=0x7f760000b040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.068+0000 7f760659c700 1 -- 172.21.15.99:0/1295348460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 msgr2=0x5647c68be450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f760659c700 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68be450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f760659c700 1 -- 172.21.15.99:0/1295348460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68be990 msgr2=0x5647c68bedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f760659c700 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68be990 0x5647c68bedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f760659c700 1 -- 172.21.15.99:0/1295348460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c66905d0 con 0x5647c68c3f40 2023-08-25T09:06:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f7605d9b700 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68be450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:06:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f760659c700 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c3f40 0x5647c68c4320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f75f8096b90 tx=0x7f75f8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5e51658aed277a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f75f4ff9700 1 -- 172.21.15.99:0/1295348460 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f809c030 con 0x5647c68c3f40 2023-08-25T09:06:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f75f4ff9700 1 -- 172.21.15.99:0/1295348460 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75f8099a80 con 0x5647c68c3f40 2023-08-25T09:06:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.069+0000 7f75f4ff9700 1 -- 172.21.15.99:0/1295348460 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f809a380 con 0x5647c68c3f40 2023-08-25T09:06:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 -- 172.21.15.99:0/1295348460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c3f40 msgr2=0x5647c68c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c3f40 0x5647c68c4320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f75f8096b90 tx=0x7f75f8005f10 comp rx=0 tx=0).stop 2023-08-25T09:06:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 -- 172.21.15.99:0/1295348460 shutdown_connections 2023-08-25T09:06:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68be990 0x5647c68bedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c3f40 0x5647c68c4320 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1295348460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68be450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 -- 172.21.15.99:0/1295348460 >> 172.21.15.99:0/1295348460 conn(0x5647c6833080 msgr2=0x5647c68b8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 -- 172.21.15.99:0/1295348460 shutdown_connections 2023-08-25T09:06:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 -- 172.21.15.99:0/1295348460 wait complete. 2023-08-25T09:06:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.070+0000 7f76203a5b80 1 -- start start 2023-08-25T09:06:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f76203a5b80 1 Processor -- start 2023-08-25T09:06:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f76203a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6810930 con 0x5647c68be990 2023-08-25T09:06:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f76203a5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c67aeb50 con 0x5647c68c3f40 2023-08-25T09:06:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f76203a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c67ae850 con 0x5647c68c4860 2023-08-25T09:06:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f76203a5b80 10 client.0 _ll_register_callbacks cb 0x5647c66f8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:06:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f760659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f7605d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f760659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43734/0 (socket says 172.21.15.99:43734) 2023-08-25T09:06:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f760659c700 1 -- 172.21.15.99:0/2494262677 learned_addr learned my addr 172.21.15.99:0/2494262677 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.071+0000 7f7606d9d700 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f760659c700 1 -- 172.21.15.99:0/2494262677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 msgr2=0x5647c68c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f760659c700 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f760659c700 1 -- 172.21.15.99:0/2494262677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 msgr2=0x5647c68c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f760659c700 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f760659c700 1 -- 172.21.15.99:0/2494262677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc009a70 con 0x5647c68be990 2023-08-25T09:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f7605d9b700 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f760659c700 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68ca6b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f75f80ca650 tx=0x7f75f8022af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20298e560fd9cb26 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f75edffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f75edffb700 1 -- 172.21.15.99:0/2494262677 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f809c040 con 0x5647c68be990 2023-08-25T09:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f76203a5b80 1 -- 172.21.15.99:0/2494262677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647c688e160 con 0x5647c68be990 2023-08-25T09:06:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f75edffb700 1 -- 172.21.15.99:0/2494262677 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f75f809d070 con 0x5647c68be990 2023-08-25T09:06:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f75edffb700 1 -- 172.21.15.99:0/2494262677 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75f8066250 con 0x5647c68be990 2023-08-25T09:06:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f76203a5b80 10 client.15783 fetch_fsmap learned FSMap version 30 2023-08-25T09:06:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.072+0000 7f76203a5b80 1 -- 172.21.15.99:0/2494262677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f76081a5a40 con 0x5647c68be990 2023-08-25T09:06:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f75edffb700 1 -- 172.21.15.99:0/2494262677 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647c688e160 con 0x5647c68be990 2023-08-25T09:06:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f75edffb700 1 -- 172.21.15.99:0/2494262677 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75f80a1a10 con 0x5647c68be990 2023-08-25T09:06:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f76203a5b80 10 client.15783 fetch_fsmap finished waiting for FSMap version 30 2023-08-25T09:06:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f76203a5b80 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:06:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f76203a5b80 20 client.15783 populate_metadata read hostname 'smithi099' 2023-08-25T09:06:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f76203a5b80 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5647c68d7970 0x5647c68d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f76203a5b80 4 client.15783 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:06:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f76203a5b80 1 -- 172.21.15.99:0/2494262677 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647c67b6b40 con 0x5647c68d7970 2023-08-25T09:06:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.073+0000 7f7605d9b700 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5647c68d7970 0x5647c68d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.074+0000 7f75edffb700 10 client.15783 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.074+0000 7f7605d9b700 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5647c68d7970 0x5647c68d9d90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268892630663a471 server_cookie=9a0ae5d91ac726ed in_seq=0 out_seq=0 2023-08-25T09:06:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.074+0000 7f75edffb700 1 -- 172.21.15.99:0/2494262677 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647c67b6b40 con 0x5647c68d7970 2023-08-25T09:06:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.074+0000 7f75edffb700 10 client.15783 handle_command_reply: tid=0 2023-08-25T09:06:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.074+0000 7f76203a5b80 1 client.15783 shutdown 2023-08-25T09:06:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 -- 172.21.15.99:0/2494262677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 msgr2=0x5647c68ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68ca6b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f75f80ca650 tx=0x7f75f8022af0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 -- 172.21.15.99:0/2494262677 shutdown_connections 2023-08-25T09:06:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c4860 0x5647c68c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68ca6b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5647c68d7970 0x5647c68d9d90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 --2- 172.21.15.99:0/2494262677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f75edffb700 0 client.15783 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 -- 172.21.15.99:0/2494262677 >> 172.21.15.99:0/2494262677 conn(0x5647c6833080 msgr2=0x5647c68b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 -- 172.21.15.99:0/2494262677 shutdown_connections 2023-08-25T09:06:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 1 -- 172.21.15.99:0/2494262677 wait complete. 2023-08-25T09:06:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.075+0000 7f76203a5b80 20 client.15783 trim_cache size 0 max 16384 2023-08-25T09:06:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.090+0000 7f76203a5b80 1 Processor -- start 2023-08-25T09:06:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.090+0000 7f76203a5b80 1 -- start start 2023-08-25T09:06:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68be990 0x5647c68ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68ca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c4860 0x5647c68c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f76203a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c66928c0 con 0x5647c68c4860 2023-08-25T09:06:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f76203a5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c68be7e0 con 0x5647c68c3f40 2023-08-25T09:06:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f76203a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c68cf410 con 0x5647c68be990 2023-08-25T09:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f760659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68be990 0x5647c68ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f7605d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f760659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68be990 0x5647c68ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:47114/0 (socket says 172.21.15.99:47114) 2023-08-25T09:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f760659c700 1 -- 172.21.15.99:0/3015912805 learned_addr learned my addr 172.21.15.99:0/3015912805 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f7606d9d700 1 --2- 172.21.15.99:0/3015912805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c4860 0x5647c68c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f760659c700 1 -- 172.21.15.99:0/3015912805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 msgr2=0x5647c68ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f760659c700 1 --2- 172.21.15.99:0/3015912805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f760659c700 1 -- 172.21.15.99:0/3015912805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c4860 msgr2=0x5647c68c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f760659c700 1 --2- 172.21.15.99:0/3015912805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c4860 0x5647c68c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.091+0000 7f760659c700 1 -- 172.21.15.99:0/3015912805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c663cd80 con 0x5647c68be990 2023-08-25T09:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f760659c700 1 --2- 172.21.15.99:0/3015912805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68be990 0x5647c68ceed0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f75f80095c0 tx=0x7f75f8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4c1cfe54d7919 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f75f57fa700 1 -- 172.21.15.99:0/3015912805 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f809c030 con 0x5647c68be990 2023-08-25T09:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f75f57fa700 1 -- 172.21.15.99:0/3015912805 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75f8066250 con 0x5647c68be990 2023-08-25T09:06:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f75f57fa700 1 -- 172.21.15.99:0/3015912805 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80a1ea0 con 0x5647c68be990 2023-08-25T09:06:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 -- 172.21.15.99:0/3015912805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68be990 msgr2=0x5647c68ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 --2- 172.21.15.99:0/3015912805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68be990 0x5647c68ceed0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f75f80095c0 tx=0x7f75f8005850 comp rx=0 tx=0).stop 2023-08-25T09:06:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 -- 172.21.15.99:0/3015912805 shutdown_connections 2023-08-25T09:06:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 --2- 172.21.15.99:0/3015912805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68c4860 0x5647c68c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 --2- 172.21.15.99:0/3015912805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68be990 0x5647c68ceed0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 --2- 172.21.15.99:0/3015912805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c3f40 0x5647c68ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 -- 172.21.15.99:0/3015912805 >> 172.21.15.99:0/3015912805 conn(0x5647c6833080 msgr2=0x5647c68ba670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 -- 172.21.15.99:0/3015912805 shutdown_connections 2023-08-25T09:06:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.092+0000 7f76203a5b80 1 -- 172.21.15.99:0/3015912805 wait complete. 2023-08-25T09:06:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.093+0000 7f76203a5b80 1 -- start start 2023-08-25T09:06:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.093+0000 7f76203a5b80 1 Processor -- start 2023-08-25T09:06:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c3f40 0x5647c68bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f76203a5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c4860 0x5647c68c0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f76203a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c68c0840 con 0x5647c68be990 2023-08-25T09:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f76203a5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c68c09b0 con 0x5647c68c4860 2023-08-25T09:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f76203a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c68c0b70 con 0x5647c68c3f40 2023-08-25T09:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f76203a5b80 10 client.0 _ll_register_callbacks cb 0x7f760000f320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f760659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7605d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c3f40 0x5647c68bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7606d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c4860 0x5647c68c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7605d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c3f40 0x5647c68bfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:47122/0 (socket says 172.21.15.99:47122) 2023-08-25T09:06:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f760659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68bf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43752/0 (socket says 172.21.15.99:43752) 2023-08-25T09:06:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7605d9b700 1 -- 172.21.15.99:0/1528428814 learned_addr learned my addr 172.21.15.99:0/1528428814 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7605d9b700 1 -- 172.21.15.99:0/1528428814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c4860 msgr2=0x5647c68c0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7605d9b700 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c4860 0x5647c68c0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7605d9b700 1 -- 172.21.15.99:0/1528428814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 msgr2=0x5647c68bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7605d9b700 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.094+0000 7f7605d9b700 1 -- 172.21.15.99:0/1528428814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76081a5a40 con 0x5647c68c3f40 2023-08-25T09:06:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.095+0000 7f7605d9b700 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c3f40 0x5647c68bfdc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5647c68b9c40 tx=0x7f75fc012e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfb7a401040550d8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.095+0000 7f75edffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:06:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.095+0000 7f75edffb700 1 -- 172.21.15.99:0/1528428814 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc017040 con 0x5647c68c3f40 2023-08-25T09:06:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.095+0000 7f76203a5b80 1 -- 172.21.15.99:0/1528428814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647c68b1600 con 0x5647c68c3f40 2023-08-25T09:06:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.095+0000 7f75edffb700 1 -- 172.21.15.99:0/1528428814 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f75fc021e20 con 0x5647c68c3f40 2023-08-25T09:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.095+0000 7f75edffb700 1 -- 172.21.15.99:0/1528428814 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75fc0212b0 con 0x5647c68c3f40 2023-08-25T09:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.095+0000 7f76203a5b80 10 client.25825 fetch_fsmap learned FSMap version 30 2023-08-25T09:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.095+0000 7f76203a5b80 1 -- 172.21.15.99:0/1528428814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5647c6625c00 con 0x5647c68c3f40 2023-08-25T09:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f75edffb700 1 -- 172.21.15.99:0/1528428814 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647c68b1600 con 0x5647c68c3f40 2023-08-25T09:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f75edffb700 1 -- 172.21.15.99:0/1528428814 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75fc0038e0 con 0x5647c68c3f40 2023-08-25T09:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f76203a5b80 10 client.25825 fetch_fsmap finished waiting for FSMap version 30 2023-08-25T09:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f76203a5b80 10 client.25825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f76203a5b80 20 client.25825 populate_metadata read hostname 'smithi099' 2023-08-25T09:06:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5647c692c260 0x5647c692e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f76203a5b80 4 client.25825 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:06:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f76203a5b80 1 -- 172.21.15.99:0/1528428814 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647c6809910 con 0x5647c692c260 2023-08-25T09:06:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.096+0000 7f760659c700 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5647c692c260 0x5647c692e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.097+0000 7f760659c700 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5647c692c260 0x5647c692e680 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b44bc8e0e5c8f43 server_cookie=b0d8c53a0b997a0 in_seq=0 out_seq=0 2023-08-25T09:06:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.097+0000 7f75edffb700 10 client.25825 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.145+0000 7f75edffb700 1 -- 172.21.15.99:0/1528428814 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647c6809910 con 0x5647c692c260 2023-08-25T09:06:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.145+0000 7f75edffb700 10 client.25825 handle_command_reply: tid=0 2023-08-25T09:06:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.145+0000 7f76203a5b80 1 client.25825 shutdown 2023-08-25T09:06:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.145+0000 7f76203a5b80 1 -- 172.21.15.99:0/1528428814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c3f40 msgr2=0x5647c68bfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.145+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c3f40 0x5647c68bfdc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5647c68b9c40 tx=0x7f75fc012e40 comp rx=0 tx=0).stop 2023-08-25T09:06:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 1 -- 172.21.15.99:0/1528428814 shutdown_connections 2023-08-25T09:06:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5647c692c260 0x5647c692e680 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647c68c4860 0x5647c68c0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f75edffb700 0 client.25825 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647c68be990 0x5647c68bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 1 --2- 172.21.15.99:0/1528428814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647c68c3f40 0x5647c68bfdc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 1 -- 172.21.15.99:0/1528428814 >> 172.21.15.99:0/1528428814 conn(0x5647c6833080 msgr2=0x5647c68c4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 1 -- 172.21.15.99:0/1528428814 shutdown_connections 2023-08-25T09:06:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 1 -- 172.21.15.99:0/1528428814 wait complete. 2023-08-25T09:06:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.146+0000 7f76203a5b80 20 client.25825 trim_cache size 0 max 16384 2023-08-25T09:06:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:06:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:06:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:06:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:06:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 -- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75f00a00c0 msgr2=0x7f75f00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75f00a00c0 0x7f75f00a2580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f760819f930 tx=0x7f760000b040 comp rx=0 tx=0).stop 2023-08-25T09:06:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 -- 172.21.15.99:0/3790451298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 msgr2=0x7f760819ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 0x7f760819ec30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f75f8005910 tx=0x7f75f8005720 comp rx=0 tx=0).stop 2023-08-25T09:06:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 -- 172.21.15.99:0/3790451298 shutdown_connections 2023-08-25T09:06:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f75f00a00c0 0x7f75f00a2580 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760810d0f0 0x7f7608198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608072d50 0x7f760819ec30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 --2- 172.21.15.99:0/3790451298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7608073670 0x7f760819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 -- 172.21.15.99:0/3790451298 >> 172.21.15.99:0/3790451298 conn(0x7f76080fc380 msgr2=0x7f760810f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 -- 172.21.15.99:0/3790451298 shutdown_connections 2023-08-25T09:06:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:10.147+0000 7f760c9ff700 1 -- 172.21.15.99:0/3790451298 wait complete. 2023-08-25T09:06:10.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 239388 3.60 MB/sec execute 256 sec latency 454.347 ms 2023-08-25T09:06:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:11 smithi157 ceph-mon[130879]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.1 MiB/s wr, 220 op/s 2023-08-25T09:06:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:11 smithi181 ceph-mon[131296]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.1 MiB/s wr, 220 op/s 2023-08-25T09:06:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:11 smithi099 ceph-mon[97641]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.1 MiB/s wr, 220 op/s 2023-08-25T09:06:11.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 239388 3.59 MB/sec execute 257 sec latency 1454.448 ms 2023-08-25T09:06:12.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 239460 3.58 MB/sec execute 258 sec latency 2418.647 ms 2023-08-25T09:06:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:13 smithi157 ceph-mon[130879]: pgmap v215: 97 pgs: 97 active+clean; 83 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2023-08-25T09:06:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:13 smithi181 ceph-mon[131296]: pgmap v215: 97 pgs: 97 active+clean; 83 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2023-08-25T09:06:13.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:13 smithi099 ceph-mon[97641]: pgmap v215: 97 pgs: 97 active+clean; 83 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2023-08-25T09:06:13.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 241537 3.62 MB/sec execute 259 sec latency 14.783 ms 2023-08-25T09:06:14.921 INFO:tasks.workunit.client.0.smithi099.stdout: 1 242691 3.62 MB/sec execute 260 sec latency 32.344 ms 2023-08-25T09:06:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:15 smithi181 ceph-mon[131296]: osdmap e90: 12 total, 12 up, 12 in 2023-08-25T09:06:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:15 smithi181 ceph-mon[131296]: pgmap v217: 97 pgs: 97 active+clean; 83 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2023-08-25T09:06:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:15 smithi099 ceph-mon[97641]: osdmap e90: 12 total, 12 up, 12 in 2023-08-25T09:06:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:15 smithi099 ceph-mon[97641]: pgmap v217: 97 pgs: 97 active+clean; 83 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2023-08-25T09:06:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:15 smithi157 ceph-mon[130879]: osdmap e90: 12 total, 12 up, 12 in 2023-08-25T09:06:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:15 smithi157 ceph-mon[130879]: pgmap v217: 97 pgs: 97 active+clean; 83 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2023-08-25T09:06:15.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 242931 3.61 MB/sec execute 261 sec latency 843.195 ms 2023-08-25T09:06:16.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 242931 3.60 MB/sec execute 262 sec latency 1843.327 ms 2023-08-25T09:06:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:17 smithi157 ceph-mon[130879]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-08-25T09:06:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:17 smithi181 ceph-mon[131296]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-08-25T09:06:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:17 smithi099 ceph-mon[97641]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-08-25T09:06:17.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 243101 3.58 MB/sec execute 263 sec latency 2752.300 ms 2023-08-25T09:06:18.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 245159 3.62 MB/sec execute 264 sec latency 12.870 ms 2023-08-25T09:06:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:19 smithi157 ceph-mon[130879]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2023-08-25T09:06:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:19 smithi181 ceph-mon[131296]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2023-08-25T09:06:19.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:19 smithi099 ceph-mon[97641]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2023-08-25T09:06:19.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 246478 3.63 MB/sec execute 265 sec latency 44.158 ms 2023-08-25T09:06:20.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 246478 3.61 MB/sec execute 266 sec latency 1005.017 ms 2023-08-25T09:06:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:21 smithi157 ceph-mon[130879]: pgmap v220: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-08-25T09:06:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:21 smithi181 ceph-mon[131296]: pgmap v220: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-08-25T09:06:21.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:21 smithi099 ceph-mon[97641]: pgmap v220: 97 pgs: 97 active+clean; 85 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-08-25T09:06:21.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 246478 3.60 MB/sec execute 267 sec latency 2005.157 ms 2023-08-25T09:06:22.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 246701 3.59 MB/sec execute 268 sec latency 2906.361 ms 2023-08-25T09:06:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 -- 172.21.15.99:0/3659809930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b40686d0 msgr2=0x7f71b410d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 --2- 172.21.15.99:0/3659809930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b40686d0 0x7f71b410d590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b0d0 tx=0x7f71ac00b3e0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 -- 172.21.15.99:0/3659809930 shutdown_connections 2023-08-25T09:06:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 --2- 172.21.15.99:0/3659809930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b40686d0 0x7f71b410d590 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 --2- 172.21.15.99:0/3659809930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71b41068a0 0x7f71b4106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 --2- 172.21.15.99:0/3659809930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b4067d10 0x7f71b4068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 -- 172.21.15.99:0/3659809930 >> 172.21.15.99:0/3659809930 conn(0x7f71b4075050 msgr2=0x7f71b4075460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 -- 172.21.15.99:0/3659809930 shutdown_connections 2023-08-25T09:06:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.309+0000 7f71b999d700 1 -- 172.21.15.99:0/3659809930 wait complete. 2023-08-25T09:06:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.310+0000 7f71b999d700 1 Processor -- start 2023-08-25T09:06:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b999d700 1 -- start start 2023-08-25T09:06:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b999d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b4067d10 0x7f71b406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b999d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71b40686d0 0x7f71b4070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b999d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b41068a0 0x7f71b406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b999d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b406b420 con 0x7f71b41068a0 2023-08-25T09:06:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b999d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b406b590 con 0x7f71b4067d10 2023-08-25T09:06:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b999d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b406b750 con 0x7f71b40686d0 2023-08-25T09:06:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b899b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b4067d10 0x7f71b406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b899b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b4067d10 0x7f71b406fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50246/0 (socket says 172.21.15.99:50246) 2023-08-25T09:06:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b899b700 1 -- 172.21.15.99:0/136184908 learned_addr learned my addr 172.21.15.99:0/136184908 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b3fff700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71b40686d0 0x7f71b4070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.311+0000 7f71b919c700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b41068a0 0x7f71b406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b899b700 1 -- 172.21.15.99:0/136184908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71b40686d0 msgr2=0x7f71b4070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b899b700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71b40686d0 0x7f71b4070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b899b700 1 -- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b41068a0 msgr2=0x7f71b406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b899b700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b41068a0 0x7f71b406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b899b700 1 -- 172.21.15.99:0/136184908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac009d00 con 0x7f71b4067d10 2023-08-25T09:06:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b3fff700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71b40686d0 0x7f71b4070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:06:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b899b700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b4067d10 0x7f71b406fbf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71a800d980 tx=0x7f71a800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b1ffb700 1 -- 172.21.15.99:0/136184908 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8004220 con 0x7f71b4067d10 2023-08-25T09:06:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b1ffb700 1 -- 172.21.15.99:0/136184908 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a8007640 con 0x7f71b4067d10 2023-08-25T09:06:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b1ffb700 1 -- 172.21.15.99:0/136184908 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8005690 con 0x7f71b4067d10 2023-08-25T09:06:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b999d700 1 -- 172.21.15.99:0/136184908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b406ba30 con 0x7f71b4067d10 2023-08-25T09:06:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.312+0000 7f71b999d700 1 -- 172.21.15.99:0/136184908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b406eb30 con 0x7f71b4067d10 2023-08-25T09:06:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.316+0000 7f71b1ffb700 1 -- 172.21.15.99:0/136184908 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f71a8004750 con 0x7f71b4067d10 2023-08-25T09:06:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71ce469b80 1 Processor -- start 2023-08-25T09:06:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71ce469b80 1 -- start start 2023-08-25T09:06:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71b1ffb700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71a40a00c0 0x7f71a40a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71b1ffb700 1 -- 172.21.15.99:0/136184908 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f71a80cb3f0 con 0x7f71b4067d10 2023-08-25T09:06:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf903e0 0x55f2ecf907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf90d00 0x55f2ecf8a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf8b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71ce469b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2eceee750 con 0x55f2ecf8ada0 2023-08-25T09:06:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71b919c700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71a40a00c0 0x7f71a40a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71ce469b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ecedf720 con 0x55f2ecf90d00 2023-08-25T09:06:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.318+0000 7f71ce469b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ecedd040 con 0x55f2ecf903e0 2023-08-25T09:06:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf90d00 0x55f2ecf8a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf90d00 0x55f2ecf8a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50254/0 (socket says 172.21.15.99:50254) 2023-08-25T09:06:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b3fff700 1 -- 172.21.15.99:0/1309554777 learned_addr learned my addr 172.21.15.99:0/1309554777 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b899b700 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf903e0 0x55f2ecf907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b919c700 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf8b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b899b700 1 -- 172.21.15.99:0/1309554777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf90d00 msgr2=0x55f2ecf8a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b899b700 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf90d00 0x55f2ecf8a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b899b700 1 -- 172.21.15.99:0/1309554777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 msgr2=0x55f2ecf8b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b899b700 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf8b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b899b700 1 -- 172.21.15.99:0/1309554777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2ecd585d0 con 0x55f2ecf903e0 2023-08-25T09:06:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b3fff700 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf90d00 0x55f2ecf8a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:06:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b899b700 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf903e0 0x55f2ecf907c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71a80c8ad0 tx=0x7f71a8099190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d53a6c5888607cb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.319+0000 7f71b919c700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71a40a00c0 0x7f71a40a2580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b770 tx=0x7f71ac008cf0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.320+0000 7f719effd700 1 -- 172.21.15.99:0/1309554777 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a809c8c0 con 0x55f2ecf903e0 2023-08-25T09:06:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.320+0000 7f719effd700 1 -- 172.21.15.99:0/1309554777 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a80a08a0 con 0x55f2ecf903e0 2023-08-25T09:06:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.320+0000 7f719effd700 1 -- 172.21.15.99:0/1309554777 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8097f80 con 0x55f2ecf903e0 2023-08-25T09:06:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.320+0000 7f71ce469b80 1 -- 172.21.15.99:0/1309554777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf903e0 msgr2=0x55f2ecf907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.320+0000 7f71ce469b80 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf903e0 0x55f2ecf907c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71a80c8ad0 tx=0x7f71a8099190 comp rx=0 tx=0).stop 2023-08-25T09:06:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.321+0000 7f71ce469b80 1 -- 172.21.15.99:0/1309554777 shutdown_connections 2023-08-25T09:06:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.321+0000 7f71ce469b80 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf8b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.321+0000 7f71ce469b80 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf903e0 0x55f2ecf907c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.321+0000 7f71ce469b80 1 --2- 172.21.15.99:0/1309554777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf90d00 0x55f2ecf8a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.321+0000 7f71ce469b80 1 -- 172.21.15.99:0/1309554777 >> 172.21.15.99:0/1309554777 conn(0x55f2ecefb080 msgr2=0x55f2ecf80fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.321+0000 7f71ce469b80 1 -- 172.21.15.99:0/1309554777 shutdown_connections 2023-08-25T09:06:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.321+0000 7f71ce469b80 1 -- 172.21.15.99:0/1309554777 wait complete. 2023-08-25T09:06:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.321+0000 7f71ce469b80 1 -- start start 2023-08-25T09:06:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.322+0000 7f71ce469b80 1 Processor -- start 2023-08-25T09:06:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.322+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf96cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf903e0 0x55f2ecf93f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71ce469b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2eced8930 con 0x55f2ecf903e0 2023-08-25T09:06:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71ce469b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ece76b50 con 0x55f2ecf8ada0 2023-08-25T09:06:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71ce469b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ece76850 con 0x55f2ecf90d00 2023-08-25T09:06:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71ce469b80 10 client.0 _ll_register_callbacks cb 0x55f2ecdc0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:06:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf96cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf96cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50262/0 (socket says 172.21.15.99:50262) 2023-08-25T09:06:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 -- 172.21.15.99:0/2972677226 learned_addr learned my addr 172.21.15.99:0/2972677226 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b919c700 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b3fff700 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf903e0 0x55f2ecf93f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 -- 172.21.15.99:0/2972677226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 msgr2=0x55f2ecf94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 -- 172.21.15.99:0/2972677226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf903e0 msgr2=0x55f2ecf93f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf903e0 0x55f2ecf93f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 -- 172.21.15.99:0/2972677226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a0009d10 con 0x55f2ecf8ada0 2023-08-25T09:06:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b3fff700 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf903e0 0x55f2ecf93f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:06:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f71b899b700 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf96cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71a80989d0 tx=0x7f71a809c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520ee3efd6f30e25 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.323+0000 7f7193fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:06:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f7193fff700 1 -- 172.21.15.99:0/2972677226 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8097f80 con 0x55f2ecf8ada0 2023-08-25T09:06:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f7193fff700 1 -- 172.21.15.99:0/2972677226 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f71a80980e0 con 0x55f2ecf8ada0 2023-08-25T09:06:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 1 -- 172.21.15.99:0/2972677226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2ecf56160 con 0x55f2ecf8ada0 2023-08-25T09:06:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f7193fff700 1 -- 172.21.15.99:0/2972677226 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a8098cf0 con 0x55f2ecf8ada0 2023-08-25T09:06:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 10 client.25853 fetch_fsmap learned FSMap version 30 2023-08-25T09:06:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 1 -- 172.21.15.99:0/2972677226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f71b406eb30 con 0x55f2ecf8ada0 2023-08-25T09:06:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f7193fff700 1 -- 172.21.15.99:0/2972677226 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2ecf56160 con 0x55f2ecf8ada0 2023-08-25T09:06:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f7193fff700 1 -- 172.21.15.99:0/2972677226 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71a80b0070 con 0x55f2ecf8ada0 2023-08-25T09:06:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 10 client.25853 fetch_fsmap finished waiting for FSMap version 30 2023-08-25T09:06:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 10 client.25853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:06:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 20 client.25853 populate_metadata read hostname 'smithi099' 2023-08-25T09:06:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f2ecfa4380 0x55f2ecfa67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 4 client.25853 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:06:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.324+0000 7f71ce469b80 1 -- 172.21.15.99:0/2972677226 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2ece7eb40 con 0x55f2ecfa4380 2023-08-25T09:06:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.325+0000 7f71b3fff700 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f2ecfa4380 0x55f2ecfa67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.325+0000 7f7193fff700 10 client.25853 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.325+0000 7f71b3fff700 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f2ecfa4380 0x55f2ecfa67a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eee60470c2de38e server_cookie=112a68be29dbd42e in_seq=0 out_seq=0 2023-08-25T09:06:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.326+0000 7f7193fff700 1 -- 172.21.15.99:0/2972677226 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f2ece7eb40 con 0x55f2ecfa4380 2023-08-25T09:06:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.326+0000 7f7193fff700 10 client.25853 handle_command_reply: tid=0 2023-08-25T09:06:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.326+0000 7f71ce469b80 1 client.25853 shutdown 2023-08-25T09:06:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 -- 172.21.15.99:0/2972677226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf8ada0 msgr2=0x55f2ecf96cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf96cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71a80989d0 tx=0x7f71a809c970 comp rx=0 tx=0).stop 2023-08-25T09:06:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 -- 172.21.15.99:0/2972677226 shutdown_connections 2023-08-25T09:06:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f2ecfa4380 0x55f2ecfa67a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf96cf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 --2- 172.21.15.99:0/2972677226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf903e0 0x55f2ecf93f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 -- 172.21.15.99:0/2972677226 >> 172.21.15.99:0/2972677226 conn(0x55f2ecefb080 msgr2=0x55f2ecf85440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f7193fff700 0 client.25853 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 -- 172.21.15.99:0/2972677226 shutdown_connections 2023-08-25T09:06:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 1 -- 172.21.15.99:0/2972677226 wait complete. 2023-08-25T09:06:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.327+0000 7f71ce469b80 20 client.25853 trim_cache size 0 max 16384 2023-08-25T09:06:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71ce469b80 1 Processor -- start 2023-08-25T09:06:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71ce469b80 1 -- start start 2023-08-25T09:06:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf9b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf96820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf80bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71ce469b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ecd5a8c0 con 0x55f2ecf8ada0 2023-08-25T09:06:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71ce469b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ecf81d20 con 0x55f2ecf903e0 2023-08-25T09:06:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71ce469b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ecf8a860 con 0x55f2ecf90d00 2023-08-25T09:06:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71b919c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf80bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.343+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf96820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b919c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf80bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:49620/0 (socket says 172.21.15.99:49620) 2023-08-25T09:06:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf96820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50270/0 (socket says 172.21.15.99:50270) 2023-08-25T09:06:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b919c700 1 -- 172.21.15.99:0/3387686342 learned_addr learned my addr 172.21.15.99:0/3387686342 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b899b700 1 --2- 172.21.15.99:0/3387686342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf9b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b919c700 1 -- 172.21.15.99:0/3387686342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 msgr2=0x55f2ecf96820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b919c700 1 --2- 172.21.15.99:0/3387686342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf96820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b919c700 1 -- 172.21.15.99:0/3387686342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 msgr2=0x55f2ecf9b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b919c700 1 --2- 172.21.15.99:0/3387686342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf9b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b919c700 1 -- 172.21.15.99:0/3387686342 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2ecd04d80 con 0x55f2ecf90d00 2023-08-25T09:06:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.344+0000 7f71b919c700 1 --2- 172.21.15.99:0/3387686342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf80bd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0093e0 tx=0x7f71ac028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f2d10c3d29be8c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.345+0000 7f719f7fe700 1 -- 172.21.15.99:0/3387686342 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00e040 con 0x55f2ecf90d00 2023-08-25T09:06:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.345+0000 7f719f7fe700 1 -- 172.21.15.99:0/3387686342 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71ac0076a0 con 0x55f2ecf90d00 2023-08-25T09:06:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.345+0000 7f719f7fe700 1 -- 172.21.15.99:0/3387686342 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac01e460 con 0x55f2ecf90d00 2023-08-25T09:06:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 -- 172.21.15.99:0/3387686342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 msgr2=0x55f2ecf80bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3387686342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf80bd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0093e0 tx=0x7f71ac028000 comp rx=0 tx=0).stop 2023-08-25T09:06:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 -- 172.21.15.99:0/3387686342 shutdown_connections 2023-08-25T09:06:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3387686342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecf80bd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3387686342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf9b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3387686342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf96820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 -- 172.21.15.99:0/3387686342 >> 172.21.15.99:0/3387686342 conn(0x55f2ecefb080 msgr2=0x55f2ecf9ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 -- 172.21.15.99:0/3387686342 shutdown_connections 2023-08-25T09:06:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.346+0000 7f71ce469b80 1 -- 172.21.15.99:0/3387686342 wait complete. 2023-08-25T09:06:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.347+0000 7f71ce469b80 1 -- start start 2023-08-25T09:06:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.347+0000 7f71ce469b80 1 Processor -- start 2023-08-25T09:06:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf9aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf9afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71ce469b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecff7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71ce469b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ecf9b8f0 con 0x55f2ecf8ada0 2023-08-25T09:06:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71ce469b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ecff76b0 con 0x55f2ecf903e0 2023-08-25T09:06:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71ce469b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ecff77f0 con 0x55f2ecf90d00 2023-08-25T09:06:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71ce469b80 10 client.0 _ll_register_callbacks cb 0x7f71a0009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:06:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf9afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b899b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf9aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf9afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50280/0 (socket says 172.21.15.99:50280) 2023-08-25T09:06:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b3fff700 1 -- 172.21.15.99:0/3073935763 learned_addr learned my addr 172.21.15.99:0/3073935763 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b919c700 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecff7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b3fff700 1 -- 172.21.15.99:0/3073935763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 msgr2=0x55f2ecff7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b3fff700 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecff7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b3fff700 1 -- 172.21.15.99:0/3073935763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 msgr2=0x55f2ecf9aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.348+0000 7f71b3fff700 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf9aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f71b3fff700 1 -- 172.21.15.99:0/3073935763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac027050 con 0x55f2ecf903e0 2023-08-25T09:06:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f71b3fff700 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf9afe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71a001f640 tx=0x7f71a001fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e301beb78334dd8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f7193fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:06:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f7193fff700 1 -- 172.21.15.99:0/3073935763 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0004030 con 0x55f2ecf903e0 2023-08-25T09:06:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f71ce469b80 1 -- 172.21.15.99:0/3073935763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2ecf79600 con 0x55f2ecf903e0 2023-08-25T09:06:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f7193fff700 1 -- 172.21.15.99:0/3073935763 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f71a0004190 con 0x55f2ecf903e0 2023-08-25T09:06:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f7193fff700 1 -- 172.21.15.99:0/3073935763 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a0004350 con 0x55f2ecf903e0 2023-08-25T09:06:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f71ce469b80 10 client.25859 fetch_fsmap learned FSMap version 30 2023-08-25T09:06:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.349+0000 7f71ce469b80 1 -- 172.21.15.99:0/3073935763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f2eccedc00 con 0x55f2ecf903e0 2023-08-25T09:06:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f7193fff700 1 -- 172.21.15.99:0/3073935763 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2ecf79600 con 0x55f2ecf903e0 2023-08-25T09:06:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f7193fff700 1 -- 172.21.15.99:0/3073935763 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71a000f710 con 0x55f2ecf903e0 2023-08-25T09:06:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f71ce469b80 10 client.25859 fetch_fsmap finished waiting for FSMap version 30 2023-08-25T09:06:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f71ce469b80 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:06:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f71ce469b80 20 client.25859 populate_metadata read hostname 'smithi099' 2023-08-25T09:06:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f2ecff9040 0x55f2ecffb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f71ce469b80 4 client.25859 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:06:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f71ce469b80 1 -- 172.21.15.99:0/3073935763 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2eced1910 con 0x55f2ecff9040 2023-08-25T09:06:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f71b899b700 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f2ecff9040 0x55f2ecffb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f71b899b700 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f2ecff9040 0x55f2ecffb460 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9832fe0996beb2e8 server_cookie=780510b6c9063d41 in_seq=0 out_seq=0 2023-08-25T09:06:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.350+0000 7f7193fff700 10 client.25859 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.404+0000 7f7193fff700 1 -- 172.21.15.99:0/3073935763 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2eced1910 con 0x55f2ecff9040 2023-08-25T09:06:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f7193fff700 10 client.25859 handle_command_reply: tid=0 2023-08-25T09:06:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 client.25859 shutdown 2023-08-25T09:06:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 -- 172.21.15.99:0/3073935763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 msgr2=0x55f2ecf9afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf9afe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71a001f640 tx=0x7f71a001fcd0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 -- 172.21.15.99:0/3073935763 shutdown_connections 2023-08-25T09:06:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2ecf90d00 0x55f2ecff7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ecf8ada0 0x55f2ecf9aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f2ecff9040 0x55f2ecffb460 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 --2- 172.21.15.99:0/3073935763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2ecf903e0 0x55f2ecf9afe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f7193fff700 0 client.25859 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 -- 172.21.15.99:0/3073935763 >> 172.21.15.99:0/3073935763 conn(0x55f2ecefb080 msgr2=0x55f2ecf80bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 -- 172.21.15.99:0/3073935763 shutdown_connections 2023-08-25T09:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.405+0000 7f71ce469b80 1 -- 172.21.15.99:0/3073935763 wait complete. 2023-08-25T09:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71ce469b80 20 client.25859 trim_cache size 0 max 16384 2023-08-25T09:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 -- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71a40a00c0 msgr2=0x7f71a40a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71a40a00c0 0x7f71a40a2580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b770 tx=0x7f71ac008cf0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 -- 172.21.15.99:0/136184908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b4067d10 msgr2=0x7f71b406fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b4067d10 0x7f71b406fbf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71a800d980 tx=0x7f71a800dc90 comp rx=0 tx=0).stop 2023-08-25T09:06:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 -- 172.21.15.99:0/136184908 shutdown_connections 2023-08-25T09:06:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71a40a00c0 0x7f71a40a2580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b41068a0 0x7f71b406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b4067d10 0x7f71b406fbf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 --2- 172.21.15.99:0/136184908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71b40686d0 0x7f71b4070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 -- 172.21.15.99:0/136184908 >> 172.21.15.99:0/136184908 conn(0x7f71b4075050 msgr2=0x7f71b40fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 -- 172.21.15.99:0/136184908 shutdown_connections 2023-08-25T09:06:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:23.406+0000 7f71b999d700 1 -- 172.21.15.99:0/136184908 wait complete. 2023-08-25T09:06:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:23 smithi157 ceph-mon[130879]: pgmap v221: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2023-08-25T09:06:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:23 smithi181 ceph-mon[131296]: pgmap v221: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2023-08-25T09:06:23.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:23 smithi099 ceph-mon[97641]: pgmap v221: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2023-08-25T09:06:23.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 248633 3.62 MB/sec execute 269 sec latency 16.888 ms 2023-08-25T09:06:24.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 249852 3.63 MB/sec execute 270 sec latency 37.585 ms 2023-08-25T09:06:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:25 smithi181 ceph-mon[131296]: pgmap v222: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2023-08-25T09:06:25.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:25 smithi099 ceph-mon[97641]: pgmap v222: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2023-08-25T09:06:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:25 smithi157 ceph-mon[130879]: pgmap v222: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2023-08-25T09:06:25.922 INFO:tasks.workunit.client.0.smithi099.stdout: 1 250021 3.62 MB/sec execute 271 sec latency 896.113 ms 2023-08-25T09:06:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:26 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:06:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:26 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:06:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:26 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:06:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:26 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:06:26.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:26 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:06:26.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:26 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:06:26.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:26 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:06:26.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:26 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:06:26.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:26 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:06:26.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:26 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:06:26.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:26 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:06:26.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:26 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:06:26.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 250021 3.60 MB/sec execute 272 sec latency 1896.181 ms 2023-08-25T09:06:27.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:27 smithi181 ceph-mon[131296]: pgmap v223: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-08-25T09:06:27.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:27 smithi099 ceph-mon[97641]: pgmap v223: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-08-25T09:06:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:27 smithi157 ceph-mon[130879]: pgmap v223: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-08-25T09:06:27.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 250219 3.59 MB/sec execute 273 sec latency 2804.757 ms 2023-08-25T09:06:28.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 252246 3.63 MB/sec execute 274 sec latency 13.973 ms 2023-08-25T09:06:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:29 smithi157 ceph-mon[130879]: pgmap v224: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-08-25T09:06:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:29 smithi181 ceph-mon[131296]: pgmap v224: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-08-25T09:06:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:29 smithi099 ceph-mon[97641]: pgmap v224: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-08-25T09:06:29.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 253539 3.63 MB/sec execute 275 sec latency 37.544 ms 2023-08-25T09:06:30.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 253572 3.62 MB/sec execute 276 sec latency 977.239 ms 2023-08-25T09:06:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:31 smithi157 ceph-mon[130879]: pgmap v225: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-08-25T09:06:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:31 smithi181 ceph-mon[131296]: pgmap v225: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-08-25T09:06:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:31 smithi099 ceph-mon[97641]: pgmap v225: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-08-25T09:06:31.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 253572 3.61 MB/sec execute 277 sec latency 1977.377 ms 2023-08-25T09:06:32.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 253754 3.60 MB/sec execute 278 sec latency 2877.356 ms 2023-08-25T09:06:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:33 smithi157 ceph-mon[130879]: pgmap v226: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2023-08-25T09:06:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:33 smithi181 ceph-mon[131296]: pgmap v226: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2023-08-25T09:06:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:33 smithi099 ceph-mon[97641]: pgmap v226: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2023-08-25T09:06:33.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 255783 3.63 MB/sec execute 279 sec latency 14.191 ms 2023-08-25T09:06:34.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 257113 3.64 MB/sec execute 280 sec latency 20.740 ms 2023-08-25T09:06:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:35 smithi181 ceph-mon[131296]: pgmap v227: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2023-08-25T09:06:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:35 smithi099 ceph-mon[97641]: pgmap v227: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2023-08-25T09:06:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:35 smithi157 ceph-mon[130879]: pgmap v227: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2023-08-25T09:06:35.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 257113 3.62 MB/sec execute 281 sec latency 1000.190 ms 2023-08-25T09:06:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 -- 172.21.15.99:0/1629968465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d60101260 msgr2=0x7f9d6010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 --2- 172.21.15.99:0/1629968465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d60101260 0x7f9d6010d770 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800b0a0 tx=0x7f9d5800b3b0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 -- 172.21.15.99:0/1629968465 shutdown_connections 2023-08-25T09:06:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 --2- 172.21.15.99:0/1629968465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d60101260 0x7f9d6010d770 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 --2- 172.21.15.99:0/1629968465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d601068b0 0x7f9d60106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 --2- 172.21.15.99:0/1629968465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d601008a0 0x7f9d60100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 -- 172.21.15.99:0/1629968465 >> 172.21.15.99:0/1629968465 conn(0x7f9d600fc370 msgr2=0x7f9d600fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 -- 172.21.15.99:0/1629968465 shutdown_connections 2023-08-25T09:06:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.561+0000 7f9d66bff700 1 -- 172.21.15.99:0/1629968465 wait complete. 2023-08-25T09:06:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.562+0000 7f9d66bff700 1 Processor -- start 2023-08-25T09:06:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.562+0000 7f9d66bff700 1 -- start start 2023-08-25T09:06:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.562+0000 7f9d66bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d601008a0 0x7f9d6019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d66bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d60101260 0x7f9d6019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d66bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d601068b0 0x7f9d60198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d66bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60198fa0 con 0x7f9d601008a0 2023-08-25T09:06:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d66bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60199110 con 0x7f9d60101260 2023-08-25T09:06:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d66bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d601992d0 con 0x7f9d601068b0 2023-08-25T09:06:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d601008a0 0x7f9d6019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d601008a0 0x7f9d6019ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54260/0 (socket says 172.21.15.99:54260) 2023-08-25T09:06:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 -- 172.21.15.99:0/2524357322 learned_addr learned my addr 172.21.15.99:0/2524357322 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6519c700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d601068b0 0x7f9d60198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d5ffff700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d60101260 0x7f9d6019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 -- 172.21.15.99:0/2524357322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d601068b0 msgr2=0x7f9d60198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d601068b0 0x7f9d60198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 -- 172.21.15.99:0/2524357322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d60101260 msgr2=0x7f9d6019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d60101260 0x7f9d6019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 -- 172.21.15.99:0/2524357322 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d58009d00 con 0x7f9d601008a0 2023-08-25T09:06:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.563+0000 7f9d6499b700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d601008a0 0x7f9d6019ead0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e9a0 tx=0x7f9d5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.564+0000 7f9d5dffb700 1 -- 172.21.15.99:0/2524357322 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54009790 con 0x7f9d601008a0 2023-08-25T09:06:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.564+0000 7f9d5dffb700 1 -- 172.21.15.99:0/2524357322 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d54010430 con 0x7f9d601008a0 2023-08-25T09:06:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.564+0000 7f9d5dffb700 1 -- 172.21.15.99:0/2524357322 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54018680 con 0x7f9d601008a0 2023-08-25T09:06:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.564+0000 7f9d66bff700 1 -- 172.21.15.99:0/2524357322 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d601995b0 con 0x7f9d601008a0 2023-08-25T09:06:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.564+0000 7f9d66bff700 1 -- 172.21.15.99:0/2524357322 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d601a5ac0 con 0x7f9d601008a0 2023-08-25T09:06:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.567+0000 7f9d5dffb700 1 -- 172.21.15.99:0/2524357322 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9d540105a0 con 0x7f9d601008a0 2023-08-25T09:06:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.568+0000 7f9d7a4a1b80 1 Processor -- start 2023-08-25T09:06:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.568+0000 7f9d5dffb700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9d500a00c0 0x7f9d500a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.568+0000 7f9d7a4a1b80 1 -- start start 2023-08-25T09:06:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.568+0000 7f9d5dffb700 1 -- 172.21.15.99:0/2524357322 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9d54014070 con 0x7f9d601008a0 2023-08-25T09:06:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6519c700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9d500a00c0 0x7f9d500a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c46320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c40450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17ba8750 con 0x55ab17c40990 2023-08-25T09:06:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17b99720 con 0x55ab17c45f40 2023-08-25T09:06:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17b97040 con 0x55ab17c46860 2023-08-25T09:06:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c46320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6519c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c46320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57646/0 (socket says 172.21.15.99:57646) 2023-08-25T09:06:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6499b700 1 -- 172.21.15.99:0/2316384493 learned_addr learned my addr 172.21.15.99:0/2316384493 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d5ffff700 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c40450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6499b700 1 -- 172.21.15.99:0/2316384493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 msgr2=0x55ab17c40450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6499b700 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c40450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6499b700 1 -- 172.21.15.99:0/2316384493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 msgr2=0x55ab17c40dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6499b700 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6499b700 1 -- 172.21.15.99:0/2316384493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab17a125d0 con 0x55ab17c45f40 2023-08-25T09:06:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.569+0000 7f9d6519c700 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:06:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.570+0000 7f9d6519c700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9d500a00c0 0x7f9d500a2580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800b780 tx=0x7f9d5800bed0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.570+0000 7f9d6499b700 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c46320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009230 tx=0x7f9d54099590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9f51b536e3d56 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.570+0000 7f9d4affd700 1 -- 172.21.15.99:0/2316384493 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54099ea0 con 0x55ab17c45f40 2023-08-25T09:06:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.570+0000 7f9d4affd700 1 -- 172.21.15.99:0/2316384493 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d540a0ca0 con 0x55ab17c45f40 2023-08-25T09:06:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.570+0000 7f9d4affd700 1 -- 172.21.15.99:0/2316384493 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54098380 con 0x55ab17c45f40 2023-08-25T09:06:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/2316384493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 msgr2=0x55ab17c46320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c46320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009230 tx=0x7f9d54099590 comp rx=0 tx=0).stop 2023-08-25T09:06:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/2316384493 shutdown_connections 2023-08-25T09:06:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c46320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/2316384493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c40450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/2316384493 >> 172.21.15.99:0/2316384493 conn(0x55ab17bb5080 msgr2=0x55ab17c3ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/2316384493 shutdown_connections 2023-08-25T09:06:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/2316384493 wait complete. 2023-08-25T09:06:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.571+0000 7f9d7a4a1b80 1 -- start start 2023-08-25T09:06:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d7a4a1b80 1 Processor -- start 2023-08-25T09:06:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c40990 0x55ab17c4c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c45f40 0x55ab17c49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17b92930 con 0x55ab17c45f40 2023-08-25T09:06:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17b30b50 con 0x55ab17c40990 2023-08-25T09:06:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17b30850 con 0x55ab17c46860 2023-08-25T09:06:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d7a4a1b80 10 client.0 _ll_register_callbacks cb 0x55ab17a7aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:06:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d5ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c45f40 0x55ab17c49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d5ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c45f40 0x55ab17c49960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54272/0 (socket says 172.21.15.99:54272) 2023-08-25T09:06:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c40990 0x55ab17c4c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d5ffff700 1 -- 172.21.15.99:0/3922945515 learned_addr learned my addr 172.21.15.99:0/3922945515 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.572+0000 7f9d6519c700 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d6499b700 1 -- 172.21.15.99:0/3922945515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 msgr2=0x55ab17c4a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d6499b700 1 -- 172.21.15.99:0/3922945515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c45f40 msgr2=0x55ab17c49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c45f40 0x55ab17c49960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d6499b700 1 -- 172.21.15.99:0/3922945515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d601a5ac0 con 0x55ab17c40990 2023-08-25T09:06:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d6519c700 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:06:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c40990 0x55ab17c4c6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55ab17c4cda0 tx=0x7f9d54010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a00318b1d6b1e56 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:06:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3922945515 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54099ea0 con 0x55ab17c40990 2023-08-25T09:06:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3922945515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab17c10160 con 0x55ab17c40990 2023-08-25T09:06:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3922945515 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9d54099300 con 0x55ab17c40990 2023-08-25T09:06:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3922945515 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d540a0ca0 con 0x55ab17c40990 2023-08-25T09:06:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d7a4a1b80 10 client.25871 fetch_fsmap learned FSMap version 30 2023-08-25T09:06:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.573+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3922945515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ab179bed80 con 0x55ab17c40990 2023-08-25T09:06:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3922945515 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab17c10160 con 0x55ab17c40990 2023-08-25T09:06:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3922945515 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d54097df0 con 0x55ab17c40990 2023-08-25T09:06:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d7a4a1b80 10 client.25871 fetch_fsmap finished waiting for FSMap version 30 2023-08-25T09:06:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d7a4a1b80 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:06:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d7a4a1b80 20 client.25871 populate_metadata read hostname 'smithi099' 2023-08-25T09:06:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ab17c59e20 0x55ab17c5c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d7a4a1b80 4 client.25871 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:06:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3922945515 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab17b38b40 con 0x55ab17c59e20 2023-08-25T09:06:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.574+0000 7f9d5ffff700 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ab17c59e20 0x55ab17c5c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.575+0000 7f9d3ffff700 10 client.25871 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.575+0000 7f9d5ffff700 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ab17c59e20 0x55ab17c5c240 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae87f04c03451a2f server_cookie=2d255197258a8c5 in_seq=0 out_seq=0 2023-08-25T09:06:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3922945515 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ab17b38b40 con 0x55ab17c59e20 2023-08-25T09:06:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d3ffff700 10 client.25871 handle_command_reply: tid=0 2023-08-25T09:06:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 client.25871 shutdown 2023-08-25T09:06:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3922945515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c40990 msgr2=0x55ab17c4c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c40990 0x55ab17c4c6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55ab17c4cda0 tx=0x7f9d54010ab0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3922945515 shutdown_connections 2023-08-25T09:06:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ab17c59e20 0x55ab17c5c240 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c40990 0x55ab17c4c6e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3922945515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c45f40 0x55ab17c49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d3ffff700 0 client.25871 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3922945515 >> 172.21.15.99:0/3922945515 conn(0x55ab17bb5080 msgr2=0x55ab17c35820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3922945515 shutdown_connections 2023-08-25T09:06:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3922945515 wait complete. 2023-08-25T09:06:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.576+0000 7f9d7a4a1b80 20 client.25871 trim_cache size 0 max 16384 2023-08-25T09:06:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.591+0000 7f9d7a4a1b80 1 Processor -- start 2023-08-25T09:06:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d7a4a1b80 1 -- start start 2023-08-25T09:06:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c51270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17a148c0 con 0x55ab17c40990 2023-08-25T09:06:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17c3bef0 con 0x55ab17c45f40 2023-08-25T09:06:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17c3b280 con 0x55ab17c46860 2023-08-25T09:06:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54284/0 (socket says 172.21.15.99:54284) 2023-08-25T09:06:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d6499b700 1 -- 172.21.15.99:0/3276841646 learned_addr learned my addr 172.21.15.99:0/3276841646 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.592+0000 7f9d5ffff700 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c51270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d6519c700 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d6499b700 1 -- 172.21.15.99:0/3276841646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 msgr2=0x55ab17c4c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d6499b700 1 -- 172.21.15.99:0/3276841646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 msgr2=0x55ab17c51270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c51270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d6499b700 1 -- 172.21.15.99:0/3276841646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab17c41040 con 0x55ab17c40990 2023-08-25T09:06:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d54098e20 tx=0x7f9d54010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b62a780663c90fb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d4b7fe700 1 -- 172.21.15.99:0/3276841646 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54099ea0 con 0x55ab17c40990 2023-08-25T09:06:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d4b7fe700 1 -- 172.21.15.99:0/3276841646 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d540a0a80 con 0x55ab17c40990 2023-08-25T09:06:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.593+0000 7f9d4b7fe700 1 -- 172.21.15.99:0/3276841646 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d540a08e0 con 0x55ab17c40990 2023-08-25T09:06:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3276841646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 msgr2=0x55ab17c40450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d54098e20 tx=0x7f9d54010b80 comp rx=0 tx=0).stop 2023-08-25T09:06:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3276841646 shutdown_connections 2023-08-25T09:06:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c46860 0x55ab17c4c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c40990 0x55ab17c40450 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3276841646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c51270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3276841646 >> 172.21.15.99:0/3276841646 conn(0x55ab17bb5080 msgr2=0x55ab17c43a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3276841646 shutdown_connections 2023-08-25T09:06:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3276841646 wait complete. 2023-08-25T09:06:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.594+0000 7f9d7a4a1b80 1 -- start start 2023-08-25T09:06:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.595+0000 7f9d7a4a1b80 1 Processor -- start 2023-08-25T09:06:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.595+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c40990 0x55ab17c42a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.595+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c42f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.595+0000 7f9d7a4a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c46860 0x55ab17c41950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.595+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17c41e90 con 0x55ab17c46860 2023-08-25T09:06:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.595+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17c42000 con 0x55ab17c45f40 2023-08-25T09:06:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.595+0000 7f9d7a4a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab17c421c0 con 0x55ab17c40990 2023-08-25T09:06:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.595+0000 7f9d7a4a1b80 10 client.0 _ll_register_callbacks cb 0x55ab17a7aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:06:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c40990 0x55ab17c42a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6519c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c46860 0x55ab17c41950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c40990 0x55ab17c42a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:47452/0 (socket says 172.21.15.99:47452) 2023-08-25T09:06:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 -- 172.21.15.99:0/3389295409 learned_addr learned my addr 172.21.15.99:0/3389295409 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d5ffff700 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c42f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 -- 172.21.15.99:0/3389295409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 msgr2=0x55ab17c42f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c42f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 -- 172.21.15.99:0/3389295409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c46860 msgr2=0x55ab17c41950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c46860 0x55ab17c41950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 -- 172.21.15.99:0/3389295409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab179bed80 con 0x55ab17c40990 2023-08-25T09:06:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d6499b700 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c40990 0x55ab17c42a40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d540c9450 tx=0x7f9d54098920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec978681edbbff2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:06:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3389295409 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54099ea0 con 0x55ab17c40990 2023-08-25T09:06:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3389295409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab17c33600 con 0x55ab17c40990 2023-08-25T09:06:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.596+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3389295409 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9d540a0ca0 con 0x55ab17c40990 2023-08-25T09:06:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3389295409 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d540993a0 con 0x55ab17c40990 2023-08-25T09:06:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d7a4a1b80 10 client.25861 fetch_fsmap learned FSMap version 30 2023-08-25T09:06:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3389295409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ab179a7c00 con 0x55ab17c40990 2023-08-25T09:06:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3389295409 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab17c33600 con 0x55ab17c40990 2023-08-25T09:06:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3389295409 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d5400ccb0 con 0x55ab17c40990 2023-08-25T09:06:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d7a4a1b80 10 client.25861 fetch_fsmap finished waiting for FSMap version 30 2023-08-25T09:06:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d7a4a1b80 10 client.25861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:06:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d7a4a1b80 20 client.25861 populate_metadata read hostname 'smithi099' 2023-08-25T09:06:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ab17cae5b0 0x55ab17cb09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d7a4a1b80 4 client.25861 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:06:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3389295409 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab17b8b910 con 0x55ab17cae5b0 2023-08-25T09:06:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.597+0000 7f9d5ffff700 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ab17cae5b0 0x55ab17cb09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.598+0000 7f9d3ffff700 10 client.25861 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.598+0000 7f9d5ffff700 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ab17cae5b0 0x55ab17cb09a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=153180e975471763 server_cookie=1cd390f49855a50b in_seq=0 out_seq=0 2023-08-25T09:06:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d3ffff700 1 -- 172.21.15.99:0/3389295409 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab17b8b910 con 0x55ab17cae5b0 2023-08-25T09:06:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d3ffff700 10 client.25861 handle_command_reply: tid=0 2023-08-25T09:06:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 client.25861 shutdown 2023-08-25T09:06:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3389295409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c40990 msgr2=0x55ab17c42a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c40990 0x55ab17c42a40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d540c9450 tx=0x7f9d54098920 comp rx=0 tx=0).stop 2023-08-25T09:06:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3389295409 shutdown_connections 2023-08-25T09:06:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab17c46860 0x55ab17c41950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab17c40990 0x55ab17c42a40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ab17cae5b0 0x55ab17cb09a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 --2- 172.21.15.99:0/3389295409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab17c45f40 0x55ab17c42f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3389295409 >> 172.21.15.99:0/3389295409 conn(0x55ab17bb5080 msgr2=0x55ab17c49200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.645+0000 7f9d3ffff700 0 client.25861 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.646+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3389295409 shutdown_connections 2023-08-25T09:06:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.646+0000 7f9d7a4a1b80 1 -- 172.21.15.99:0/3389295409 wait complete. 2023-08-25T09:06:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.646+0000 7f9d7a4a1b80 20 client.25861 trim_cache size 0 max 16384 2023-08-25T09:06:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:06:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:06:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:06:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:06:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 -- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9d500a00c0 msgr2=0x7f9d500a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9d500a00c0 0x7f9d500a2580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800b780 tx=0x7f9d5800bed0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 -- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d601008a0 msgr2=0x7f9d6019ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d601008a0 0x7f9d6019ead0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e9a0 tx=0x7f9d5400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 -- 172.21.15.99:0/2524357322 shutdown_connections 2023-08-25T09:06:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9d500a00c0 0x7f9d500a2580 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d601068b0 0x7f9d60198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d601008a0 0x7f9d6019ead0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 --2- 172.21.15.99:0/2524357322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d60101260 0x7f9d6019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 -- 172.21.15.99:0/2524357322 >> 172.21.15.99:0/2524357322 conn(0x7f9d600fc370 msgr2=0x7f9d6010bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 -- 172.21.15.99:0/2524357322 shutdown_connections 2023-08-25T09:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:36.647+0000 7f9d66bff700 1 -- 172.21.15.99:0/2524357322 wait complete. 2023-08-25T09:06:36.923 INFO:tasks.workunit.client.0.smithi099.stdout: 1 257113 3.61 MB/sec execute 282 sec latency 2000.356 ms 2023-08-25T09:06:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:37 smithi157 ceph-mon[130879]: pgmap v228: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2023-08-25T09:06:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:37 smithi181 ceph-mon[131296]: pgmap v228: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2023-08-25T09:06:37.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:37 smithi099 ceph-mon[97641]: pgmap v228: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2023-08-25T09:06:37.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 257295 3.60 MB/sec execute 283 sec latency 2901.914 ms 2023-08-25T09:06:38.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 259325 3.63 MB/sec execute 284 sec latency 16.150 ms 2023-08-25T09:06:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:39 smithi157 ceph-mon[130879]: pgmap v229: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2023-08-25T09:06:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:39 smithi181 ceph-mon[131296]: pgmap v229: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2023-08-25T09:06:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:39 smithi099 ceph-mon[97641]: pgmap v229: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2023-08-25T09:06:39.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 260618 3.64 MB/sec execute 285 sec latency 22.273 ms 2023-08-25T09:06:40.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 260661 3.63 MB/sec execute 286 sec latency 961.045 ms 2023-08-25T09:06:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:41 smithi157 ceph-mon[130879]: pgmap v230: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2023-08-25T09:06:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:41 smithi181 ceph-mon[131296]: pgmap v230: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2023-08-25T09:06:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:41 smithi099 ceph-mon[97641]: pgmap v230: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2023-08-25T09:06:41.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 260661 3.62 MB/sec execute 287 sec latency 1961.164 ms 2023-08-25T09:06:42.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 260840 3.60 MB/sec execute 288 sec latency 2866.740 ms 2023-08-25T09:06:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:43 smithi157 ceph-mon[130879]: pgmap v231: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2023-08-25T09:06:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:43 smithi181 ceph-mon[131296]: pgmap v231: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2023-08-25T09:06:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:43 smithi099 ceph-mon[97641]: pgmap v231: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2023-08-25T09:06:43.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 262870 3.64 MB/sec execute 289 sec latency 13.412 ms 2023-08-25T09:06:44.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 264042 3.64 MB/sec execute 290 sec latency 50.442 ms 2023-08-25T09:06:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:45 smithi181 ceph-mon[131296]: pgmap v232: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2023-08-25T09:06:45.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:45 smithi099 ceph-mon[97641]: pgmap v232: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2023-08-25T09:06:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:45 smithi157 ceph-mon[130879]: pgmap v232: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2023-08-25T09:06:45.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 264204 3.63 MB/sec execute 291 sec latency 906.685 ms 2023-08-25T09:06:46.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 264204 3.62 MB/sec execute 292 sec latency 1906.842 ms 2023-08-25T09:06:47.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:47 smithi157 ceph-mon[130879]: pgmap v233: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2023-08-25T09:06:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:47 smithi181 ceph-mon[131296]: pgmap v233: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2023-08-25T09:06:47.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:47 smithi099 ceph-mon[97641]: pgmap v233: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2023-08-25T09:06:47.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 264387 3.61 MB/sec execute 293 sec latency 2813.826 ms 2023-08-25T09:06:48.924 INFO:tasks.workunit.client.0.smithi099.stdout: 1 266432 3.64 MB/sec execute 294 sec latency 15.470 ms 2023-08-25T09:06:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:49 smithi157 ceph-mon[130879]: pgmap v234: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2023-08-25T09:06:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:49 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:06:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:49 smithi181 ceph-mon[131296]: pgmap v234: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2023-08-25T09:06:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:49 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:06:49.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:49 smithi099 ceph-mon[97641]: pgmap v234: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2023-08-25T09:06:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:49 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:06:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.813+0000 7effabfff700 1 -- 172.21.15.99:0/4091393306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac100ad0 msgr2=0x7effac100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.813+0000 7effabfff700 1 --2- 172.21.15.99:0/4091393306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac100ad0 0x7effac100eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009990 tx=0x7eff9c009ca0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.813+0000 7effabfff700 1 -- 172.21.15.99:0/4091393306 shutdown_connections 2023-08-25T09:06:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.813+0000 7effabfff700 1 --2- 172.21.15.99:0/4091393306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effac102bd0 0x7effac10d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.813+0000 7effabfff700 1 --2- 172.21.15.99:0/4091393306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac100ad0 0x7effac100eb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.813+0000 7effabfff700 1 --2- 172.21.15.99:0/4091393306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac102210 0x7effac102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.813+0000 7effabfff700 1 -- 172.21.15.99:0/4091393306 >> 172.21.15.99:0/4091393306 conn(0x7effac0fc1e0 msgr2=0x7effac0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.814+0000 7effabfff700 1 -- 172.21.15.99:0/4091393306 shutdown_connections 2023-08-25T09:06:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.814+0000 7effabfff700 1 -- 172.21.15.99:0/4091393306 wait complete. 2023-08-25T09:06:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.815+0000 7effabfff700 1 Processor -- start 2023-08-25T09:06:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.815+0000 7effabfff700 1 -- start start 2023-08-25T09:06:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.815+0000 7effabfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effac100ad0 0x7effac19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.815+0000 7effabfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac102210 0x7effac19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effabfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac102bd0 0x7effac198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effabfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac198f70 con 0x7effac100ad0 2023-08-25T09:06:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effabfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac1990e0 con 0x7effac102210 2023-08-25T09:06:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effabfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac1992a0 con 0x7effac102bd0 2023-08-25T09:06:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effac100ad0 0x7effac19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaa7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac102210 0x7effac19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac102bd0 0x7effac198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effac100ad0 0x7effac19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43288/0 (socket says 172.21.15.99:43288) 2023-08-25T09:06:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaaffd700 1 -- 172.21.15.99:0/1922155989 learned_addr learned my addr 172.21.15.99:0/1922155989 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaa7fc700 1 -- 172.21.15.99:0/1922155989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac102bd0 msgr2=0x7effac198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaa7fc700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac102bd0 0x7effac198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaa7fc700 1 -- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effac100ad0 msgr2=0x7effac19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaa7fc700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effac100ad0 0x7effac19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaa7fc700 1 -- 172.21.15.99:0/1922155989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c0095f0 con 0x7effac102210 2023-08-25T09:06:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaaffd700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effac100ad0 0x7effac19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:06:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.816+0000 7effaa7fc700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac102210 0x7effac19edd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effa000e9f0 tx=0x7effa000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.817+0000 7eff93fff700 1 -- 172.21.15.99:0/1922155989 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000c9c0 con 0x7effac102210 2023-08-25T09:06:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.817+0000 7eff93fff700 1 -- 172.21.15.99:0/1922155989 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa0010430 con 0x7effac102210 2023-08-25T09:06:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.817+0000 7eff93fff700 1 -- 172.21.15.99:0/1922155989 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0018680 con 0x7effac102210 2023-08-25T09:06:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.817+0000 7effabfff700 1 -- 172.21.15.99:0/1922155989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effac199580 con 0x7effac102210 2023-08-25T09:06:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.817+0000 7effabfff700 1 -- 172.21.15.99:0/1922155989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac1a5880 con 0x7effac102210 2023-08-25T09:06:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.820+0000 7eff93fff700 1 -- 172.21.15.99:0/1922155989 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7effa0010860 con 0x7effac102210 2023-08-25T09:06:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.822+0000 7effc510bb80 1 Processor -- start 2023-08-25T09:06:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.822+0000 7eff93fff700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7eff940a0140 0x7eff940a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.822+0000 7effc510bb80 1 -- start start 2023-08-25T09:06:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.822+0000 7eff93fff700 1 -- 172.21.15.99:0/1922155989 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7effa0014070 con 0x7effac102210 2023-08-25T09:06:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effab7fe700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7eff940a0140 0x7eff940a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effc510bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423010 0x56301e4233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effc510bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423930 0x56301e41d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effc510bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 0x56301e41ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effc510bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e385750 con 0x56301e41d990 2023-08-25T09:06:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effc510bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e376720 con 0x56301e423930 2023-08-25T09:06:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effc510bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e374040 con 0x56301e423010 2023-08-25T09:06:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaa7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423930 0x56301e41d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaa7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423930 0x56301e41d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:46586/0 (socket says 172.21.15.99:46586) 2023-08-25T09:06:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaa7fc700 1 -- 172.21.15.99:0/343870656 learned_addr learned my addr 172.21.15.99:0/343870656 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaaffd700 1 --2- 172.21.15.99:0/343870656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423010 0x56301e4233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaa7fc700 1 -- 172.21.15.99:0/343870656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423010 msgr2=0x56301e4233f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaa7fc700 1 --2- 172.21.15.99:0/343870656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423010 0x56301e4233f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaa7fc700 1 -- 172.21.15.99:0/343870656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 msgr2=0x56301e41ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaa7fc700 1 --2- 172.21.15.99:0/343870656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 0x56301e41ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effaa7fc700 1 -- 172.21.15.99:0/343870656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301e1ef5d0 con 0x56301e423930 2023-08-25T09:06:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.823+0000 7effab7fe700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7eff940a0140 0x7eff940a2600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7effac19f590 tx=0x7effa400b040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.824+0000 7effaa7fc700 1 --2- 172.21.15.99:0/343870656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423930 0x56301e41d450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7effa00958b0 tx=0x7effa0098820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad93a15ab86d936 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.824+0000 7eff917fa700 1 -- 172.21.15.99:0/343870656 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa009cc70 con 0x56301e423930 2023-08-25T09:06:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.824+0000 7eff917fa700 1 -- 172.21.15.99:0/343870656 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa00a0c50 con 0x56301e423930 2023-08-25T09:06:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.824+0000 7eff917fa700 1 -- 172.21.15.99:0/343870656 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0099300 con 0x56301e423930 2023-08-25T09:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 -- 172.21.15.99:0/343870656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423930 msgr2=0x56301e41d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 --2- 172.21.15.99:0/343870656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423930 0x56301e41d450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7effa00958b0 tx=0x7effa0098820 comp rx=0 tx=0).stop 2023-08-25T09:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 -- 172.21.15.99:0/343870656 shutdown_connections 2023-08-25T09:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 --2- 172.21.15.99:0/343870656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 0x56301e41ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 --2- 172.21.15.99:0/343870656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423010 0x56301e4233f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 --2- 172.21.15.99:0/343870656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423930 0x56301e41d450 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 -- 172.21.15.99:0/343870656 >> 172.21.15.99:0/343870656 conn(0x56301e392080 msgr2=0x56301e417fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 -- 172.21.15.99:0/343870656 shutdown_connections 2023-08-25T09:06:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 -- 172.21.15.99:0/343870656 wait complete. 2023-08-25T09:06:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.825+0000 7effc510bb80 1 -- start start 2023-08-25T09:06:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effc510bb80 1 Processor -- start 2023-08-25T09:06:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effc510bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e41d990 0x56301e426ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effc510bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e4270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effc510bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423930 0x56301e427620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effc510bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e36f930 con 0x56301e423930 2023-08-25T09:06:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effc510bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e30db50 con 0x56301e423010 2023-08-25T09:06:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effc510bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e30d850 con 0x56301e41d990 2023-08-25T09:06:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effc510bb80 10 client.0 _ll_register_callbacks cb 0x56301e257a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:06:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effaa7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e4270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effaaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e41d990 0x56301e426ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effaaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e41d990 0x56301e426ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45200/0 (socket says 172.21.15.99:45200) 2023-08-25T09:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effaa7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e4270e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:46588/0 (socket says 172.21.15.99:46588) 2023-08-25T09:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effaaffd700 1 -- 172.21.15.99:0/615609480 learned_addr learned my addr 172.21.15.99:0/615609480 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.826+0000 7effab7fe700 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423930 0x56301e427620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effaa7fc700 1 -- 172.21.15.99:0/615609480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e41d990 msgr2=0x56301e426ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effaa7fc700 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e41d990 0x56301e426ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effaa7fc700 1 -- 172.21.15.99:0/615609480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423930 msgr2=0x56301e427620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effaa7fc700 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423930 0x56301e427620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effaa7fc700 1 -- 172.21.15.99:0/615609480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac1a5880 con 0x56301e423010 2023-08-25T09:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effaa7fc700 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e4270e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7effa0095340 tx=0x7effa00107d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b50e900ecb410f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7eff8eff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7eff8eff5700 1 -- 172.21.15.99:0/615609480 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa00b82d0 con 0x56301e423010 2023-08-25T09:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7eff8eff5700 1 -- 172.21.15.99:0/615609480 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7effa009cc70 con 0x56301e423010 2023-08-25T09:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effc510bb80 1 -- 172.21.15.99:0/615609480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56301e3ed160 con 0x56301e423010 2023-08-25T09:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7eff8eff5700 1 -- 172.21.15.99:0/615609480 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa00990a0 con 0x56301e423010 2023-08-25T09:06:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effc510bb80 10 client.25889 fetch_fsmap learned FSMap version 31 2023-08-25T09:06:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.827+0000 7effc510bb80 1 -- 172.21.15.99:0/615609480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56301e19bd80 con 0x56301e423010 2023-08-25T09:06:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7eff8eff5700 1 -- 172.21.15.99:0/615609480 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56301e3ed160 con 0x56301e423010 2023-08-25T09:06:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7eff8eff5700 1 -- 172.21.15.99:0/615609480 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7effa0094cc0 con 0x56301e423010 2023-08-25T09:06:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7effc510bb80 10 client.25889 fetch_fsmap finished waiting for FSMap version 31 2023-08-25T09:06:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7effc510bb80 10 client.25889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:06:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7effc510bb80 20 client.25889 populate_metadata read hostname 'smithi099' 2023-08-25T09:06:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7effc510bb80 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56301e4370e0 0x56301e439500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7effc510bb80 4 client.25889 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:06:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7effc510bb80 1 -- 172.21.15.99:0/615609480 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56301e315b40 con 0x56301e4370e0 2023-08-25T09:06:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.828+0000 7effaaffd700 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56301e4370e0 0x56301e439500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.829+0000 7effaaffd700 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56301e4370e0 0x56301e439500 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4bf2f0374561e49 server_cookie=9a015c3336956ae7 in_seq=0 out_seq=0 2023-08-25T09:06:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.829+0000 7eff8eff5700 10 client.25889 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.829+0000 7eff8eff5700 1 -- 172.21.15.99:0/615609480 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56301e315b40 con 0x56301e4370e0 2023-08-25T09:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.829+0000 7eff8eff5700 10 client.25889 handle_command_reply: tid=0 2023-08-25T09:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.829+0000 7effc510bb80 1 client.25889 shutdown 2023-08-25T09:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 -- 172.21.15.99:0/615609480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 msgr2=0x56301e4270e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e4270e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7effa0095340 tx=0x7effa00107d0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 -- 172.21.15.99:0/615609480 shutdown_connections 2023-08-25T09:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423930 0x56301e427620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e41d990 0x56301e426ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56301e4370e0 0x56301e439500 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 --2- 172.21.15.99:0/615609480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e4270e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7eff8eff5700 0 client.25889 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 -- 172.21.15.99:0/615609480 >> 172.21.15.99:0/615609480 conn(0x56301e392080 msgr2=0x56301e41c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 -- 172.21.15.99:0/615609480 shutdown_connections 2023-08-25T09:06:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 1 -- 172.21.15.99:0/615609480 wait complete. 2023-08-25T09:06:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.830+0000 7effc510bb80 20 client.25889 trim_cache size 0 max 16384 2023-08-25T09:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.845+0000 7effc510bb80 1 Processor -- start 2023-08-25T09:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effc510bb80 1 -- start start 2023-08-25T09:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effc510bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e41d990 0x56301e4274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effc510bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423010 0x56301e417bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effc510bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e418110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effc510bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e1f18c0 con 0x56301e423010 2023-08-25T09:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effc510bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e429f30 con 0x56301e41d990 2023-08-25T09:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effc510bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e4146f0 con 0x56301e423930 2023-08-25T09:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e418110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effaaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e41d990 0x56301e4274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e418110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45212/0 (socket says 172.21.15.99:45212) 2023-08-25T09:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effab7fe700 1 -- 172.21.15.99:0/1559312676 learned_addr learned my addr 172.21.15.99:0/1559312676 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.846+0000 7effaa7fc700 1 --2- 172.21.15.99:0/1559312676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423010 0x56301e417bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7effaaffd700 1 -- 172.21.15.99:0/1559312676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 msgr2=0x56301e418110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7effaaffd700 1 --2- 172.21.15.99:0/1559312676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e418110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7effaaffd700 1 -- 172.21.15.99:0/1559312676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423010 msgr2=0x56301e417bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7effaaffd700 1 --2- 172.21.15.99:0/1559312676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423010 0x56301e417bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7effaaffd700 1 -- 172.21.15.99:0/1559312676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301e411b40 con 0x56301e41d990 2023-08-25T09:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7effaaffd700 1 --2- 172.21.15.99:0/1559312676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e41d990 0x56301e4274a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x56301e4374c0 tx=0x7eff9c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbb67b3d43d33814 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7eff91ffb700 1 -- 172.21.15.99:0/1559312676 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c01d040 con 0x56301e41d990 2023-08-25T09:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7eff91ffb700 1 -- 172.21.15.99:0/1559312676 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9c01e040 con 0x56301e41d990 2023-08-25T09:06:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.847+0000 7eff91ffb700 1 -- 172.21.15.99:0/1559312676 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c005570 con 0x56301e41d990 2023-08-25T09:06:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 -- 172.21.15.99:0/1559312676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e41d990 msgr2=0x56301e4274a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 --2- 172.21.15.99:0/1559312676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e41d990 0x56301e4274a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x56301e4374c0 tx=0x7eff9c00b260 comp rx=0 tx=0).stop 2023-08-25T09:06:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 -- 172.21.15.99:0/1559312676 shutdown_connections 2023-08-25T09:06:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 --2- 172.21.15.99:0/1559312676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e418110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 --2- 172.21.15.99:0/1559312676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e41d990 0x56301e4274a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 --2- 172.21.15.99:0/1559312676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e423010 0x56301e417bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 -- 172.21.15.99:0/1559312676 >> 172.21.15.99:0/1559312676 conn(0x56301e392080 msgr2=0x56301e432b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 -- 172.21.15.99:0/1559312676 shutdown_connections 2023-08-25T09:06:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.848+0000 7effc510bb80 1 -- 172.21.15.99:0/1559312676 wait complete. 2023-08-25T09:06:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 1 -- start start 2023-08-25T09:06:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 1 Processor -- start 2023-08-25T09:06:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 0x56301e42e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e42c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e42c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e42e9a0 con 0x56301e41d990 2023-08-25T09:06:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e42ce40 con 0x56301e423010 2023-08-25T09:06:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e42cfb0 con 0x56301e423930 2023-08-25T09:06:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.849+0000 7effc510bb80 10 client.0 _ll_register_callbacks cb 0x7effa40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:06:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 0x56301e42e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e42c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effaa7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e42c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e42c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45224/0 (socket says 172.21.15.99:45224) 2023-08-25T09:06:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effaa7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e42c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:46604/0 (socket says 172.21.15.99:46604) 2023-08-25T09:06:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 -- 172.21.15.99:0/3473983488 learned_addr learned my addr 172.21.15.99:0/3473983488 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:06:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 -- 172.21.15.99:0/3473983488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 msgr2=0x56301e42c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e42c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 -- 172.21.15.99:0/3473983488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 msgr2=0x56301e42e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:06:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 0x56301e42e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 -- 172.21.15.99:0/3473983488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac10dfe0 con 0x56301e423930 2023-08-25T09:06:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effaa7fc700 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e42c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:06:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7effab7fe700 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e42c900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7effa4008b40 tx=0x7effa4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d135f61d1a4964 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:06:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.850+0000 7eff8eff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:06:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7eff8eff5700 1 -- 172.21.15.99:0/3473983488 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa400e040 con 0x56301e423930 2023-08-25T09:06:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 1 -- 172.21.15.99:0/3473983488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56301e410600 con 0x56301e423930 2023-08-25T09:06:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7eff8eff5700 1 -- 172.21.15.99:0/3473983488 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7effa401c040 con 0x56301e423930 2023-08-25T09:06:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7eff8eff5700 1 -- 172.21.15.99:0/3473983488 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa4008e50 con 0x56301e423930 2023-08-25T09:06:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 10 client.25879 fetch_fsmap learned FSMap version 31 2023-08-25T09:06:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 1 -- 172.21.15.99:0/3473983488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56301e184c00 con 0x56301e423930 2023-08-25T09:06:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7eff8eff5700 1 -- 172.21.15.99:0/3473983488 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56301e410600 con 0x56301e423930 2023-08-25T09:06:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7eff8eff5700 1 -- 172.21.15.99:0/3473983488 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7effa4029030 con 0x56301e423930 2023-08-25T09:06:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 10 client.25879 fetch_fsmap finished waiting for FSMap version 31 2023-08-25T09:06:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 10 client.25879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:06:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 20 client.25879 populate_metadata read hostname 'smithi099' 2023-08-25T09:06:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56301e48c2a0 0x56301e48e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:06:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 4 client.25879 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:06:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.851+0000 7effc510bb80 1 -- 172.21.15.99:0/3473983488 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56301e368910 con 0x56301e48c2a0 2023-08-25T09:06:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.852+0000 7effaaffd700 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56301e48c2a0 0x56301e48e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:06:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.852+0000 7effaaffd700 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56301e48c2a0 0x56301e48e6c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ad533ee1b6b6d4 server_cookie=b4cb21ac34f5c801 in_seq=0 out_seq=0 2023-08-25T09:06:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.852+0000 7eff8eff5700 10 client.25879 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.903+0000 7eff8eff5700 1 -- 172.21.15.99:0/3473983488 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56301e368910 con 0x56301e48c2a0 2023-08-25T09:06:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.903+0000 7eff8eff5700 10 client.25879 handle_command_reply: tid=0 2023-08-25T09:06:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.903+0000 7effc510bb80 1 client.25879 shutdown 2023-08-25T09:06:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.903+0000 7effc510bb80 1 -- 172.21.15.99:0/3473983488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 msgr2=0x56301e42c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.903+0000 7effc510bb80 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e42c900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7effa4008b40 tx=0x7effa4009920 comp rx=0 tx=0).stop 2023-08-25T09:06:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 1 -- 172.21.15.99:0/3473983488 shutdown_connections 2023-08-25T09:06:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56301e423930 0x56301e42c900 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56301e41d990 0x56301e42e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56301e48c2a0 0x56301e48e6c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 1 --2- 172.21.15.99:0/3473983488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56301e423010 0x56301e42c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 1 -- 172.21.15.99:0/3473983488 >> 172.21.15.99:0/3473983488 conn(0x56301e392080 msgr2=0x56301e417bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7eff8eff5700 0 client.25879 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:06:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 1 -- 172.21.15.99:0/3473983488 shutdown_connections 2023-08-25T09:06:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 1 -- 172.21.15.99:0/3473983488 wait complete. 2023-08-25T09:06:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.904+0000 7effc510bb80 20 client.25879 trim_cache size 0 max 16384 2023-08-25T09:06:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:06:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:06:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:06:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:06:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 -- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7eff940a0140 msgr2=0x7eff940a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7eff940a0140 0x7eff940a2600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7effac19f590 tx=0x7effa400b040 comp rx=0 tx=0).stop 2023-08-25T09:06:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 -- 172.21.15.99:0/1922155989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac102210 msgr2=0x7effac19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:06:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac102210 0x7effac19edd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effa000e9f0 tx=0x7effa000ed00 comp rx=0 tx=0).stop 2023-08-25T09:06:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 -- 172.21.15.99:0/1922155989 shutdown_connections 2023-08-25T09:06:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7eff940a0140 0x7eff940a2600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac102bd0 0x7effac198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effac100ad0 0x7effac19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 --2- 172.21.15.99:0/1922155989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effac102210 0x7effac19edd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:06:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 -- 172.21.15.99:0/1922155989 >> 172.21.15.99:0/1922155989 conn(0x7effac0fc1e0 msgr2=0x7effac105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:06:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 -- 172.21.15.99:0/1922155989 shutdown_connections 2023-08-25T09:06:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:06:49.905+0000 7effabfff700 1 -- 172.21.15.99:0/1922155989 wait complete. 2023-08-25T09:06:49.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 267619 3.65 MB/sec execute 295 sec latency 37.489 ms 2023-08-25T09:06:50.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 269819 3.68 MB/sec execute 296 sec latency 15.603 ms 2023-08-25T09:06:51.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:51 smithi157 ceph-mon[130879]: pgmap v235: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2023-08-25T09:06:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:51 smithi181 ceph-mon[131296]: pgmap v235: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2023-08-25T09:06:51.827 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:51 smithi099 ceph-mon[97641]: pgmap v235: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2023-08-25T09:06:51.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 271012 3.69 MB/sec execute 297 sec latency 19.712 ms 2023-08-25T09:06:52.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 272369 3.69 MB/sec execute 298 sec latency 17.679 ms 2023-08-25T09:06:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:53 smithi157 ceph-mon[130879]: pgmap v236: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2023-08-25T09:06:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:53 smithi181 ceph-mon[131296]: pgmap v236: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2023-08-25T09:06:53.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:53 smithi099 ceph-mon[97641]: pgmap v236: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2023-08-25T09:06:53.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 273745 3.71 MB/sec execute 299 sec latency 11.503 ms 2023-08-25T09:06:54.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:54 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:06:54.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:54 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:06:54.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:54 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:06:54.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 274978 3.71 MB/sec execute 300 sec latency 36.441 ms 2023-08-25T09:06:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:55 smithi157 ceph-mon[130879]: pgmap v237: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-08-25T09:06:55.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:55 smithi181 ceph-mon[131296]: pgmap v237: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-08-25T09:06:55.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:55 smithi099 ceph-mon[97641]: pgmap v237: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-08-25T09:06:55.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 277005 3.75 MB/sec execute 301 sec latency 48.757 ms 2023-08-25T09:06:56.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278201 3.75 MB/sec execute 302 sec latency 22.008 ms 2023-08-25T09:06:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:57 smithi157 ceph-mon[130879]: pgmap v238: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2023-08-25T09:06:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:57 smithi181 ceph-mon[131296]: pgmap v238: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2023-08-25T09:06:57.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:57 smithi099 ceph-mon[97641]: pgmap v238: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2023-08-25T09:06:57.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 279901 3.77 MB/sec execute 303 sec latency 15.865 ms 2023-08-25T09:06:58.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 281145 3.78 MB/sec execute 304 sec latency 12.885 ms 2023-08-25T09:06:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:06:59 smithi157 ceph-mon[130879]: pgmap v239: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2023-08-25T09:06:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:06:59 smithi181 ceph-mon[131296]: pgmap v239: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2023-08-25T09:06:59.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:06:59 smithi099 ceph-mon[97641]: pgmap v239: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2023-08-25T09:06:59.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 282632 3.78 MB/sec execute 305 sec latency 18.291 ms 2023-08-25T09:07:00.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 284162 3.81 MB/sec execute 306 sec latency 35.833 ms 2023-08-25T09:07:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:01 smithi157 ceph-mon[130879]: pgmap v240: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2023-08-25T09:07:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:01 smithi181 ceph-mon[131296]: pgmap v240: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2023-08-25T09:07:01.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:01 smithi099 ceph-mon[97641]: pgmap v240: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2023-08-25T09:07:01.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 284162 3.80 MB/sec execute 307 sec latency 1034.151 ms 2023-08-25T09:07:02.925 INFO:tasks.workunit.client.0.smithi099.stdout: 1 284202 3.79 MB/sec execute 308 sec latency 1933.149 ms 2023-08-25T09:07:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.060+0000 7fe02efff700 1 -- 172.21.15.99:0/1983671936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0281068b0 msgr2=0x7fe028106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.060+0000 7fe02efff700 1 --2- 172.21.15.99:0/1983671936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0281068b0 0x7fe028106c90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009990 tx=0x7fe01c009ca0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.060+0000 7fe02efff700 1 -- 172.21.15.99:0/1983671936 shutdown_connections 2023-08-25T09:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.060+0000 7fe02efff700 1 --2- 172.21.15.99:0/1983671936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe028101260 0x7fe02810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.060+0000 7fe02efff700 1 --2- 172.21.15.99:0/1983671936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0281068b0 0x7fe028106c90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.060+0000 7fe02efff700 1 --2- 172.21.15.99:0/1983671936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0281008a0 0x7fe028100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.060+0000 7fe02efff700 1 -- 172.21.15.99:0/1983671936 >> 172.21.15.99:0/1983671936 conn(0x7fe0280fc370 msgr2=0x7fe0280fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.061+0000 7fe02efff700 1 -- 172.21.15.99:0/1983671936 shutdown_connections 2023-08-25T09:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.061+0000 7fe02efff700 1 -- 172.21.15.99:0/1983671936 wait complete. 2023-08-25T09:07:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.062+0000 7fe02efff700 1 Processor -- start 2023-08-25T09:07:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.062+0000 7fe02efff700 1 -- start start 2023-08-25T09:07:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02efff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0281008a0 0x7fe02819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02efff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe028101260 0x7fe02819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02efff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0281068b0 0x7fe0281967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02efff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe028196e10 con 0x7fe028101260 2023-08-25T09:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02efff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe028196f80 con 0x7fe0281008a0 2023-08-25T09:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02efff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe028197140 con 0x7fe0281068b0 2023-08-25T09:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0281008a0 0x7fe02819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0281008a0 0x7fe02819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52474/0 (socket says 172.21.15.99:52474) 2023-08-25T09:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02cd9b700 1 -- 172.21.15.99:0/3906948550 learned_addr learned my addr 172.21.15.99:0/3906948550 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02d59c700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0281068b0 0x7fe0281967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe027fff700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe028101260 0x7fe02819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02cd9b700 1 -- 172.21.15.99:0/3906948550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0281068b0 msgr2=0x7fe0281967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0281068b0 0x7fe0281967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02cd9b700 1 -- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe028101260 msgr2=0x7fe02819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe028101260 0x7fe02819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe02cd9b700 1 -- 172.21.15.99:0/3906948550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe014009520 con 0x7fe0281008a0 2023-08-25T09:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.063+0000 7fe027fff700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe028101260 0x7fe02819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.064+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0281008a0 0x7fe02819c8e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005910 tx=0x7fe01c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.064+0000 7fe025ffb700 1 -- 172.21.15.99:0/3906948550 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c01d070 con 0x7fe0281008a0 2023-08-25T09:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.064+0000 7fe025ffb700 1 -- 172.21.15.99:0/3906948550 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe01c022470 con 0x7fe0281008a0 2023-08-25T09:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.064+0000 7fe02efff700 1 -- 172.21.15.99:0/3906948550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c0095f0 con 0x7fe0281008a0 2023-08-25T09:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.064+0000 7fe025ffb700 1 -- 172.21.15.99:0/3906948550 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c017650 con 0x7fe0281008a0 2023-08-25T09:07:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.064+0000 7fe02efff700 1 -- 172.21.15.99:0/3906948550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0281a36f0 con 0x7fe0281008a0 2023-08-25T09:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.067+0000 7fe025ffb700 1 -- 172.21.15.99:0/3906948550 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fe01c0225e0 con 0x7fe0281008a0 2023-08-25T09:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe0428cdb80 1 Processor -- start 2023-08-25T09:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe0428cdb80 1 -- start start 2023-08-25T09:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe025ffb700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe0180a00c0 0x7fe0180a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe025ffb700 1 -- 172.21.15.99:0/3906948550 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fe01c0ccae0 con 0x7fe0281008a0 2023-08-25T09:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1654730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f164e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f15b2750 con 0x5568f164eda0 2023-08-25T09:07:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f15a3720 con 0x5568f1654350 2023-08-25T09:07:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f15a1040 con 0x5568f1654c70 2023-08-25T09:07:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.069+0000 7fe02d59c700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe0180a00c0 0x7fe0180a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f164e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe02cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1654730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f164e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:36686/0 (socket says 172.21.15.99:36686) 2023-08-25T09:07:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 -- 172.21.15.99:0/2947943931 learned_addr learned my addr 172.21.15.99:0/2947943931 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe02d59c700 1 --2- 172.21.15.99:0/2947943931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 -- 172.21.15.99:0/2947943931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 msgr2=0x5568f1654730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 --2- 172.21.15.99:0/2947943931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1654730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 -- 172.21.15.99:0/2947943931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 msgr2=0x5568f164f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 --2- 172.21.15.99:0/2947943931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 -- 172.21.15.99:0/2947943931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568f141c5d0 con 0x5568f1654c70 2023-08-25T09:07:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe027fff700 1 --2- 172.21.15.99:0/2947943931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f164e860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe014009f10 tx=0x7fe0140116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f56833cd42a886e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe02d59c700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe0180a00c0 0x7fe0180a2580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe02000f870 tx=0x7fe02000fc30 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe012ffd700 1 -- 172.21.15.99:0/2947943931 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014011ee0 con 0x5568f1654c70 2023-08-25T09:07:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe012ffd700 1 -- 172.21.15.99:0/2947943931 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe014012910 con 0x5568f1654c70 2023-08-25T09:07:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.070+0000 7fe012ffd700 1 -- 172.21.15.99:0/2947943931 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400f8d0 con 0x5568f1654c70 2023-08-25T09:07:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 -- 172.21.15.99:0/2947943931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 msgr2=0x5568f164e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/2947943931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f164e860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe014009f10 tx=0x7fe0140116c0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 -- 172.21.15.99:0/2947943931 shutdown_connections 2023-08-25T09:07:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/2947943931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/2947943931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1654730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/2947943931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f164e860 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 -- 172.21.15.99:0/2947943931 >> 172.21.15.99:0/2947943931 conn(0x5568f15bf080 msgr2=0x5568f1644d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 -- 172.21.15.99:0/2947943931 shutdown_connections 2023-08-25T09:07:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.071+0000 7fe0428cdb80 1 -- 172.21.15.99:0/2947943931 wait complete. 2023-08-25T09:07:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.072+0000 7fe0428cdb80 1 -- start start 2023-08-25T09:07:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.072+0000 7fe0428cdb80 1 Processor -- start 2023-08-25T09:07:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f164eda0 0x5568f165aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f1654350 0x5568f1657ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1658420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f159c930 con 0x5568f1654350 2023-08-25T09:07:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f153ab50 con 0x5568f164eda0 2023-08-25T09:07:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f153a850 con 0x5568f1654c70 2023-08-25T09:07:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe0428cdb80 10 client.0 _ll_register_callbacks cb 0x5568f1484a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe027fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f1654350 0x5568f1657ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02d59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1658420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02d59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1658420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:36702/0 (socket says 172.21.15.99:36702) 2023-08-25T09:07:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02d59c700 1 -- 172.21.15.99:0/322228475 learned_addr learned my addr 172.21.15.99:0/322228475 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f164eda0 0x5568f165aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02d59c700 1 -- 172.21.15.99:0/322228475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f164eda0 msgr2=0x5568f165aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02d59c700 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f164eda0 0x5568f165aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02d59c700 1 -- 172.21.15.99:0/322228475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f1654350 msgr2=0x5568f1657ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02d59c700 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f1654350 0x5568f1657ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.073+0000 7fe02d59c700 1 -- 172.21.15.99:0/322228475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe014009b40 con 0x5568f1654c70 2023-08-25T09:07:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe02d59c700 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1658420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe020018b00 tx=0x7fe020004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32d3d2ea16296d13 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe007fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:07:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe007fff700 1 -- 172.21.15.99:0/322228475 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02000e040 con 0x5568f1654c70 2023-08-25T09:07:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe0428cdb80 1 -- 172.21.15.99:0/322228475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568f161a160 con 0x5568f1654c70 2023-08-25T09:07:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe007fff700 1 -- 172.21.15.99:0/322228475 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fe020008ba0 con 0x5568f1654c70 2023-08-25T09:07:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe007fff700 1 -- 172.21.15.99:0/322228475 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe020008d60 con 0x5568f1654c70 2023-08-25T09:07:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe0428cdb80 10 client.25891 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe0428cdb80 1 -- 172.21.15.99:0/322228475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5568f13c8d80 con 0x5568f1654c70 2023-08-25T09:07:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe007fff700 1 -- 172.21.15.99:0/322228475 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568f161a160 con 0x5568f1654c70 2023-08-25T09:07:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.074+0000 7fe007fff700 1 -- 172.21.15.99:0/322228475 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe02001a440 con 0x5568f1654c70 2023-08-25T09:07:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe0428cdb80 10 client.25891 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe0428cdb80 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe0428cdb80 20 client.25891 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5568f1668620 0x5568f166aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe0428cdb80 4 client.25891 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:07:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe0428cdb80 1 -- 172.21.15.99:0/322228475 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568f1542b40 con 0x5568f1668620 2023-08-25T09:07:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5568f1668620 0x5568f166aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe007fff700 10 client.25891 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.075+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5568f1668620 0x5568f166aa40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba18e81289fd3ee server_cookie=8996c6efed66f8f in_seq=0 out_seq=0 2023-08-25T09:07:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe007fff700 1 -- 172.21.15.99:0/322228475 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5568f1542b40 con 0x5568f1668620 2023-08-25T09:07:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe007fff700 10 client.25891 handle_command_reply: tid=0 2023-08-25T09:07:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 client.25891 shutdown 2023-08-25T09:07:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 -- 172.21.15.99:0/322228475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 msgr2=0x5568f1658420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1658420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe020018b00 tx=0x7fe020004b40 comp rx=0 tx=0).stop 2023-08-25T09:07:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 -- 172.21.15.99:0/322228475 shutdown_connections 2023-08-25T09:07:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1658420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5568f1668620 0x5568f166aa40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f164eda0 0x5568f165aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/322228475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f1654350 0x5568f1657ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe007fff700 0 client.25891 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.076+0000 7fe0428cdb80 1 -- 172.21.15.99:0/322228475 >> 172.21.15.99:0/322228475 conn(0x5568f15bf080 msgr2=0x5568f163f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.077+0000 7fe0428cdb80 1 -- 172.21.15.99:0/322228475 shutdown_connections 2023-08-25T09:07:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.077+0000 7fe0428cdb80 1 -- 172.21.15.99:0/322228475 wait complete. 2023-08-25T09:07:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.077+0000 7fe0428cdb80 20 client.25891 trim_cache size 0 max 16384 2023-08-25T09:07:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.092+0000 7fe0428cdb80 1 Processor -- start 2023-08-25T09:07:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.092+0000 7fe0428cdb80 1 -- start start 2023-08-25T09:07:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.092+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.092+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654350 0x5568f165f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.092+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654c70 0x5568f165a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.092+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f141e8c0 con 0x5568f164eda0 2023-08-25T09:07:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.092+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f1654790 con 0x5568f1654c70 2023-08-25T09:07:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.092+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f1664fa0 con 0x5568f1654350 2023-08-25T09:07:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02cd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02d59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654c70 0x5568f165a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02d59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654c70 0x5568f165a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52504/0 (socket says 172.21.15.99:52504) 2023-08-25T09:07:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe027fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654350 0x5568f165f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02d59c700 1 -- 172.21.15.99:0/1969637410 learned_addr learned my addr 172.21.15.99:0/1969637410 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02cd9b700 1 -- 172.21.15.99:0/1969637410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654350 msgr2=0x5568f165f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/1969637410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654350 0x5568f165f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02cd9b700 1 -- 172.21.15.99:0/1969637410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654c70 msgr2=0x5568f165a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/1969637410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654c70 0x5568f165a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02cd9b700 1 -- 172.21.15.99:0/1969637410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568f13b1c00 con 0x5568f164eda0 2023-08-25T09:07:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/1969637410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164e860 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0ca160 tx=0x7fe01c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaadb838bbad0ed server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe0137fe700 1 -- 172.21.15.99:0/1969637410 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c099b10 con 0x5568f164eda0 2023-08-25T09:07:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe0137fe700 1 -- 172.21.15.99:0/1969637410 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe01c09a8b0 con 0x5568f164eda0 2023-08-25T09:07:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.093+0000 7fe0137fe700 1 -- 172.21.15.99:0/1969637410 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0985e0 con 0x5568f164eda0 2023-08-25T09:07:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 -- 172.21.15.99:0/1969637410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 msgr2=0x5568f164e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/1969637410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164e860 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0ca160 tx=0x7fe01c01a040 comp rx=0 tx=0).stop 2023-08-25T09:07:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 -- 172.21.15.99:0/1969637410 shutdown_connections 2023-08-25T09:07:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/1969637410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654c70 0x5568f165a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/1969637410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f164e860 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/1969637410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654350 0x5568f165f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 -- 172.21.15.99:0/1969637410 >> 172.21.15.99:0/1969637410 conn(0x5568f15bf080 msgr2=0x5568f1644bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 -- 172.21.15.99:0/1969637410 shutdown_connections 2023-08-25T09:07:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.094+0000 7fe0428cdb80 1 -- 172.21.15.99:0/1969637410 wait complete. 2023-08-25T09:07:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 1 -- start start 2023-08-25T09:07:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 1 Processor -- start 2023-08-25T09:07:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f1657870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1656240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1656780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f1656cc0 con 0x5568f164eda0 2023-08-25T09:07:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f1656e30 con 0x5568f1654350 2023-08-25T09:07:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f1656ff0 con 0x5568f1654c70 2023-08-25T09:07:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe0428cdb80 10 client.0 _ll_register_callbacks cb 0x7fe020009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe027fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1656240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.095+0000 7fe02cd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f1657870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe027fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1656240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52516/0 (socket says 172.21.15.99:52516) 2023-08-25T09:07:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe027fff700 1 -- 172.21.15.99:0/3175660629 learned_addr learned my addr 172.21.15.99:0/3175660629 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe02d59c700 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1656780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe027fff700 1 -- 172.21.15.99:0/3175660629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 msgr2=0x5568f1656780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe027fff700 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1656780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe027fff700 1 -- 172.21.15.99:0/3175660629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 msgr2=0x5568f1657870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe027fff700 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f1657870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe027fff700 1 -- 172.21.15.99:0/3175660629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0281a36f0 con 0x5568f1654350 2023-08-25T09:07:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe02d59c700 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1656780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe027fff700 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1656240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe014009f10 tx=0x7fe014005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8d03a8bfa3fec5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe007fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:07:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe007fff700 1 -- 172.21.15.99:0/3175660629 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014011e80 con 0x5568f1654350 2023-08-25T09:07:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.096+0000 7fe0428cdb80 1 -- 172.21.15.99:0/3175660629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568f163d600 con 0x5568f1654350 2023-08-25T09:07:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe007fff700 1 -- 172.21.15.99:0/3175660629 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fe014004020 con 0x5568f1654350 2023-08-25T09:07:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe007fff700 1 -- 172.21.15.99:0/3175660629 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe014004200 con 0x5568f1654350 2023-08-25T09:07:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe0428cdb80 10 client.25913 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe0428cdb80 1 -- 172.21.15.99:0/3175660629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5568f13ef1f0 con 0x5568f1654350 2023-08-25T09:07:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe007fff700 1 -- 172.21.15.99:0/3175660629 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568f163d600 con 0x5568f1654350 2023-08-25T09:07:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe007fff700 1 -- 172.21.15.99:0/3175660629 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe0140113e0 con 0x5568f1654350 2023-08-25T09:07:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe0428cdb80 10 client.25913 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe0428cdb80 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe0428cdb80 20 client.25913 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5568f16bd7d0 0x5568f16bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe0428cdb80 4 client.25913 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:07:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.097+0000 7fe0428cdb80 1 -- 172.21.15.99:0/3175660629 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5568f1595910 con 0x5568f16bd7d0 2023-08-25T09:07:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.098+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5568f16bd7d0 0x5568f16bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.098+0000 7fe02cd9b700 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5568f16bd7d0 0x5568f16bfbf0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=235ce82c43aa484a server_cookie=2e591dc88000e84c in_seq=0 out_seq=0 2023-08-25T09:07:03.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.098+0000 7fe007fff700 10 client.25913 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe007fff700 1 -- 172.21.15.99:0/3175660629 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5568f1595910 con 0x5568f16bd7d0 2023-08-25T09:07:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe007fff700 10 client.25913 handle_command_reply: tid=0 2023-08-25T09:07:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 client.25913 shutdown 2023-08-25T09:07:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 -- 172.21.15.99:0/3175660629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 msgr2=0x5568f1656240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1656240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe014009f10 tx=0x7fe014005800 comp rx=0 tx=0).stop 2023-08-25T09:07:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 -- 172.21.15.99:0/3175660629 shutdown_connections 2023-08-25T09:07:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568f1654c70 0x5568f1656780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5568f164eda0 0x5568f1657870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5568f16bd7d0 0x5568f16bfbf0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 --2- 172.21.15.99:0/3175660629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568f1654350 0x5568f1656240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe007fff700 0 client.25913 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 -- 172.21.15.99:0/3175660629 >> 172.21.15.99:0/3175660629 conn(0x5568f15bf080 msgr2=0x5568f1655050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 -- 172.21.15.99:0/3175660629 shutdown_connections 2023-08-25T09:07:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 1 -- 172.21.15.99:0/3175660629 wait complete. 2023-08-25T09:07:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.178+0000 7fe0428cdb80 20 client.25913 trim_cache size 0 max 16384 2023-08-25T09:07:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:07:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:07:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:07:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:07:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 -- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe0180a00c0 msgr2=0x7fe0180a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe0180a00c0 0x7fe0180a2580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe02000f870 tx=0x7fe02000fc30 comp rx=0 tx=0).stop 2023-08-25T09:07:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 -- 172.21.15.99:0/3906948550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0281008a0 msgr2=0x7fe02819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0281008a0 0x7fe02819c8e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005910 tx=0x7fe01c005720 comp rx=0 tx=0).stop 2023-08-25T09:07:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 -- 172.21.15.99:0/3906948550 shutdown_connections 2023-08-25T09:07:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe0180a00c0 0x7fe0180a2580 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0281068b0 0x7fe0281967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0281008a0 0x7fe02819c8e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 --2- 172.21.15.99:0/3906948550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe028101260 0x7fe02819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 -- 172.21.15.99:0/3906948550 >> 172.21.15.99:0/3906948550 conn(0x7fe0280fc370 msgr2=0x7fe0280fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 -- 172.21.15.99:0/3906948550 shutdown_connections 2023-08-25T09:07:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:03.179+0000 7fe02efff700 1 -- 172.21.15.99:0/3906948550 wait complete. 2023-08-25T09:07:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:03 smithi157 ceph-mon[130879]: pgmap v241: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 234 op/s 2023-08-25T09:07:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:03 smithi181 ceph-mon[131296]: pgmap v241: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 234 op/s 2023-08-25T09:07:03.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:03 smithi099 ceph-mon[97641]: pgmap v241: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 234 op/s 2023-08-25T09:07:03.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 285059 3.78 MB/sec execute 309 sec latency 25.579 ms 2023-08-25T09:07:04.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 287312 3.82 MB/sec execute 310 sec latency 21.783 ms 2023-08-25T09:07:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:05 smithi181 ceph-mon[131296]: pgmap v242: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 150 op/s 2023-08-25T09:07:05.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:05 smithi099 ceph-mon[97641]: pgmap v242: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 150 op/s 2023-08-25T09:07:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:05 smithi157 ceph-mon[130879]: pgmap v242: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 150 op/s 2023-08-25T09:07:05.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 288311 3.81 MB/sec execute 311 sec latency 38.467 ms 2023-08-25T09:07:06.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 290342 3.84 MB/sec execute 312 sec latency 31.867 ms 2023-08-25T09:07:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:07 smithi157 ceph-mon[130879]: pgmap v243: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-08-25T09:07:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:07 smithi181 ceph-mon[131296]: pgmap v243: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-08-25T09:07:07.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:07 smithi099 ceph-mon[97641]: pgmap v243: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-08-25T09:07:07.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 291683 3.85 MB/sec execute 313 sec latency 13.871 ms 2023-08-25T09:07:08.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 292721 3.85 MB/sec execute 314 sec latency 36.947 ms 2023-08-25T09:07:09.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:09 smithi157 ceph-mon[130879]: pgmap v244: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2023-08-25T09:07:09.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:09 smithi157 ceph-mon[130879]: osdmap e91: 12 total, 12 up, 12 in 2023-08-25T09:07:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:09 smithi181 ceph-mon[131296]: pgmap v244: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2023-08-25T09:07:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:09 smithi181 ceph-mon[131296]: osdmap e91: 12 total, 12 up, 12 in 2023-08-25T09:07:09.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:09 smithi099 ceph-mon[97641]: pgmap v244: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2023-08-25T09:07:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:09 smithi099 ceph-mon[97641]: osdmap e91: 12 total, 12 up, 12 in 2023-08-25T09:07:09.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 294715 3.88 MB/sec execute 315 sec latency 14.214 ms 2023-08-25T09:07:10.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 295943 3.89 MB/sec execute 316 sec latency 25.307 ms 2023-08-25T09:07:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:11 smithi157 ceph-mon[130879]: pgmap v246: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2023-08-25T09:07:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:11 smithi181 ceph-mon[131296]: pgmap v246: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2023-08-25T09:07:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:11 smithi099 ceph-mon[97641]: pgmap v246: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2023-08-25T09:07:11.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 298127 3.92 MB/sec execute 317 sec latency 23.695 ms 2023-08-25T09:07:12.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 299232 3.91 MB/sec execute 318 sec latency 22.757 ms 2023-08-25T09:07:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:13 smithi157 ceph-mon[130879]: pgmap v247: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2023-08-25T09:07:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:13 smithi181 ceph-mon[131296]: pgmap v247: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2023-08-25T09:07:13.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:13 smithi099 ceph-mon[97641]: pgmap v247: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2023-08-25T09:07:13.926 INFO:tasks.workunit.client.0.smithi099.stdout: 1 300735 3.93 MB/sec execute 319 sec latency 17.121 ms 2023-08-25T09:07:14.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 302096 3.94 MB/sec execute 320 sec latency 11.409 ms 2023-08-25T09:07:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:15 smithi181 ceph-mon[131296]: osdmap e92: 12 total, 12 up, 12 in 2023-08-25T09:07:15.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:15 smithi181 ceph-mon[131296]: pgmap v249: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2023-08-25T09:07:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:15 smithi099 ceph-mon[97641]: osdmap e92: 12 total, 12 up, 12 in 2023-08-25T09:07:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:15 smithi099 ceph-mon[97641]: pgmap v249: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2023-08-25T09:07:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:15 smithi157 ceph-mon[130879]: osdmap e92: 12 total, 12 up, 12 in 2023-08-25T09:07:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:15 smithi157 ceph-mon[130879]: pgmap v249: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2023-08-25T09:07:15.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 303881 3.95 MB/sec execute 321 sec latency 28.323 ms 2023-08-25T09:07:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 -- 172.21.15.99:0/251695603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e9010d0e0 msgr2=0x7f2e9010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 --2- 172.21.15.99:0/251695603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e9010d0e0 0x7f2e9010f550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009f90 tx=0x7f2e8800b330 comp rx=0 tx=0).stop 2023-08-25T09:07:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 -- 172.21.15.99:0/251695603 shutdown_connections 2023-08-25T09:07:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 --2- 172.21.15.99:0/251695603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e9010d0e0 0x7f2e9010f550 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 --2- 172.21.15.99:0/251695603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e90072d50 0x7f2e90073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 --2- 172.21.15.99:0/251695603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e90073670 0x7f2e9010cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 -- 172.21.15.99:0/251695603 >> 172.21.15.99:0/251695603 conn(0x7f2e900fc370 msgr2=0x7f2e900fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 -- 172.21.15.99:0/251695603 shutdown_connections 2023-08-25T09:07:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.336+0000 7f2e961ff700 1 -- 172.21.15.99:0/251695603 wait complete. 2023-08-25T09:07:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.337+0000 7f2e961ff700 1 Processor -- start 2023-08-25T09:07:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.337+0000 7f2e961ff700 1 -- start start 2023-08-25T09:07:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e961ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e90072d50 0x7f2e90198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e961ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e90073670 0x7f2e90199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e961ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9010d0e0 0x7f2e9019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e961ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9019cde0 con 0x7f2e9010d0e0 2023-08-25T09:07:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e961ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9019cf50 con 0x7f2e90072d50 2023-08-25T09:07:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e961ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9019d110 con 0x7f2e90073670 2023-08-25T09:07:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e90072d50 0x7f2e90198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e90072d50 0x7f2e90198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51782/0 (socket says 172.21.15.99:51782) 2023-08-25T09:07:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8f7fe700 1 -- 172.21.15.99:0/3101573815 learned_addr learned my addr 172.21.15.99:0/3101573815 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8effd700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e90073670 0x7f2e90199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9010d0e0 0x7f2e9019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8f7fe700 1 -- 172.21.15.99:0/3101573815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e90073670 msgr2=0x7f2e90199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e90073670 0x7f2e90199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8f7fe700 1 -- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9010d0e0 msgr2=0x7f2e9019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9010d0e0 0x7f2e9019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.338+0000 7f2e8f7fe700 1 -- 172.21.15.99:0/3101573815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88009c40 con 0x7f2e90072d50 2023-08-25T09:07:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.339+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e90072d50 0x7f2e90198ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000e9c0 tx=0x7f2e8000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.339+0000 7f2e8cff9700 1 -- 172.21.15.99:0/3101573815 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80004a30 con 0x7f2e90072d50 2023-08-25T09:07:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.339+0000 7f2e961ff700 1 -- 172.21.15.99:0/3101573815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9019d3f0 con 0x7f2e90072d50 2023-08-25T09:07:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.339+0000 7f2e8cff9700 1 -- 172.21.15.99:0/3101573815 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e8000cc80 con 0x7f2e90072d50 2023-08-25T09:07:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.339+0000 7f2e8cff9700 1 -- 172.21.15.99:0/3101573815 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e800054e0 con 0x7f2e90072d50 2023-08-25T09:07:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.339+0000 7f2e961ff700 1 -- 172.21.15.99:0/3101573815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e901a5d50 con 0x7f2e90072d50 2023-08-25T09:07:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.342+0000 7f2e8cff9700 1 -- 172.21.15.99:0/3101573815 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f2e80004b90 con 0x7f2e90072d50 2023-08-25T09:07:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.344+0000 7f2ea9bacb80 1 Processor -- start 2023-08-25T09:07:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.344+0000 7f2e8cff9700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e780a00c0 0x7f2e780a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.344+0000 7f2ea9bacb80 1 -- start start 2023-08-25T09:07:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.344+0000 7f2e8cff9700 1 -- 172.21.15.99:0/3101573815 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2e80014070 con 0x7f2e90072d50 2023-08-25T09:07:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.344+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e780a00c0 0x7f2e780a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.344+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb7f40 0x558a1fbb8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fb1a750 con 0x558a1fbb7f40 2023-08-25T09:07:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fb0b720 con 0x558a1fbb8860 2023-08-25T09:07:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fb09040 con 0x558a1fbb2990 2023-08-25T09:07:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb7f40 0x558a1fbb8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51784/0 (socket says 172.21.15.99:51784) 2023-08-25T09:07:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8effd700 1 -- 172.21.15.99:0/1956076579 learned_addr learned my addr 172.21.15.99:0/1956076579 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/1956076579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8effd700 1 -- 172.21.15.99:0/1956076579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 msgr2=0x558a1fbb2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8effd700 1 --2- 172.21.15.99:0/1956076579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8effd700 1 -- 172.21.15.99:0/1956076579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb7f40 msgr2=0x558a1fbb8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8effd700 1 --2- 172.21.15.99:0/1956076579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb7f40 0x558a1fbb8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.345+0000 7f2e8effd700 1 -- 172.21.15.99:0/1956076579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a1f9845d0 con 0x558a1fbb8860 2023-08-25T09:07:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.346+0000 7f2e8effd700 1 --2- 172.21.15.99:0/1956076579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb2450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009cb0 tx=0x7f2e84011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c327532d08f16aad server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.346+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e780a00c0 0x7f2e780a2580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009790 tx=0x7f2e88007e20 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.346+0000 7f2e75ffb700 1 -- 172.21.15.99:0/1956076579 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84011d20 con 0x558a1fbb8860 2023-08-25T09:07:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.346+0000 7f2e75ffb700 1 -- 172.21.15.99:0/1956076579 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e84011e80 con 0x558a1fbb8860 2023-08-25T09:07:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.346+0000 7f2e75ffb700 1 -- 172.21.15.99:0/1956076579 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84005280 con 0x558a1fbb8860 2023-08-25T09:07:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/1956076579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 msgr2=0x558a1fbb2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/1956076579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb2450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009cb0 tx=0x7f2e84011440 comp rx=0 tx=0).stop 2023-08-25T09:07:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/1956076579 shutdown_connections 2023-08-25T09:07:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/1956076579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/1956076579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb7f40 0x558a1fbb8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/1956076579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb2450 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/1956076579 >> 172.21.15.99:0/1956076579 conn(0x558a1fb27080 msgr2=0x558a1fbacd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/1956076579 shutdown_connections 2023-08-25T09:07:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/1956076579 wait complete. 2023-08-25T09:07:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.347+0000 7f2ea9bacb80 1 -- start start 2023-08-25T09:07:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2ea9bacb80 1 Processor -- start 2023-08-25T09:07:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbbe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb7f40 0x558a1fbbb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbbc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fb04930 con 0x558a1fbb8860 2023-08-25T09:07:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1faa2b50 con 0x558a1fbb7f40 2023-08-25T09:07:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1faa2850 con 0x558a1fbb2990 2023-08-25T09:07:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2ea9bacb80 10 client.0 _ll_register_callbacks cb 0x558a1f9eca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.348+0000 7f2e8effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb7f40 0x558a1fbbb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb7f40 0x558a1fbbb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51794/0 (socket says 172.21.15.99:51794) 2023-08-25T09:07:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8effd700 1 -- 172.21.15.99:0/3058480810 learned_addr learned my addr 172.21.15.99:0/3058480810 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbbe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbbc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8effd700 1 -- 172.21.15.99:0/3058480810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 msgr2=0x558a1fbbe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8effd700 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbbe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8effd700 1 -- 172.21.15.99:0/3058480810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 msgr2=0x558a1fbbc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8effd700 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbbc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8effd700 1 -- 172.21.15.99:0/3058480810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e840098e0 con 0x558a1fbb7f40 2023-08-25T09:07:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbbc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e8effd700 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb7f40 0x558a1fbbb8e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009230 tx=0x7f2e84013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4978a0a87715fc2b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.349+0000 7f2e6effd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:07:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.350+0000 7f2e6effd700 1 -- 172.21.15.99:0/3058480810 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84011b20 con 0x558a1fbb7f40 2023-08-25T09:07:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.350+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3058480810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a1fb82160 con 0x558a1fbb7f40 2023-08-25T09:07:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.350+0000 7f2e6effd700 1 -- 172.21.15.99:0/3058480810 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2e84011c80 con 0x558a1fbb7f40 2023-08-25T09:07:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.350+0000 7f2e6effd700 1 -- 172.21.15.99:0/3058480810 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e84011e40 con 0x558a1fbb7f40 2023-08-25T09:07:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.350+0000 7f2ea9bacb80 10 client.25925 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.350+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3058480810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558a1f930d80 con 0x558a1fbb7f40 2023-08-25T09:07:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.350+0000 7f2e6effd700 1 -- 172.21.15.99:0/3058480810 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a1fb82160 con 0x558a1fbb7f40 2023-08-25T09:07:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.350+0000 7f2e6effd700 1 -- 172.21.15.99:0/3058480810 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2e84003cb0 con 0x558a1fbb7f40 2023-08-25T09:07:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2ea9bacb80 10 client.25925 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2ea9bacb80 10 client.25925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2ea9bacb80 20 client.25925 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558a1fbcc040 0x558a1fbce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2ea9bacb80 4 client.25925 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:07:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3058480810 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a1faaab40 con 0x558a1fbcc040 2023-08-25T09:07:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558a1fbcc040 0x558a1fbce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2e6effd700 10 client.25925 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.351+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558a1fbcc040 0x558a1fbce460 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93432287ec4a87e server_cookie=679d7a208905a16c in_seq=0 out_seq=0 2023-08-25T09:07:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2e6effd700 1 -- 172.21.15.99:0/3058480810 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558a1faaab40 con 0x558a1fbcc040 2023-08-25T09:07:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2e6effd700 10 client.25925 handle_command_reply: tid=0 2023-08-25T09:07:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 client.25925 shutdown 2023-08-25T09:07:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3058480810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb7f40 msgr2=0x558a1fbbb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb7f40 0x558a1fbbb8e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009230 tx=0x7f2e84013bd0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3058480810 shutdown_connections 2023-08-25T09:07:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbbc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbbe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558a1fbcc040 0x558a1fbce460 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3058480810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb7f40 0x558a1fbbb8e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3058480810 >> 172.21.15.99:0/3058480810 conn(0x558a1fb27080 msgr2=0x558a1fba7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2e6effd700 0 client.25925 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3058480810 shutdown_connections 2023-08-25T09:07:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3058480810 wait complete. 2023-08-25T09:07:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.353+0000 7f2ea9bacb80 20 client.25925 trim_cache size 0 max 16384 2023-08-25T09:07:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.374+0000 7f2ea9bacb80 1 Processor -- start 2023-08-25T09:07:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.374+0000 7f2ea9bacb80 1 -- start start 2023-08-25T09:07:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.374+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.374+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbc3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.374+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fba6870 0x558a1fbbe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.374+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f9868c0 con 0x558a1fbb8860 2023-08-25T09:07:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.374+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fbbf030 con 0x558a1fba6870 2023-08-25T09:07:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.374+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fbb8320 con 0x558a1fbb2990 2023-08-25T09:07:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:51688/0 (socket says 172.21.15.99:51688) 2023-08-25T09:07:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 -- 172.21.15.99:0/3437176949 learned_addr learned my addr 172.21.15.99:0/3437176949 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8effd700 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbc3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fba6870 0x558a1fbbe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 -- 172.21.15.99:0/3437176949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fba6870 msgr2=0x558a1fbbe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fba6870 0x558a1fbbe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 -- 172.21.15.99:0/3437176949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 msgr2=0x558a1fbc3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbc3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 -- 172.21.15.99:0/3437176949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a1f919c00 con 0x558a1fbb2990 2023-08-25T09:07:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8effd700 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbc3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:07:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e800c9470 tx=0x7f2e800b8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56b49cb07c64fc0e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e767fc700 1 -- 172.21.15.99:0/3437176949 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e800a0c40 con 0x558a1fbb2990 2023-08-25T09:07:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e767fc700 1 -- 172.21.15.99:0/3437176949 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e80096f70 con 0x558a1fbb2990 2023-08-25T09:07:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.375+0000 7f2e767fc700 1 -- 172.21.15.99:0/3437176949 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80095540 con 0x558a1fbb2990 2023-08-25T09:07:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3437176949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 msgr2=0x558a1fbb2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e800c9470 tx=0x7f2e800b8040 comp rx=0 tx=0).stop 2023-08-25T09:07:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3437176949 shutdown_connections 2023-08-25T09:07:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fba6870 0x558a1fbbe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb2450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/3437176949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fbb8860 0x558a1fbc3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3437176949 >> 172.21.15.99:0/3437176949 conn(0x558a1fb27080 msgr2=0x558a1fbacc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3437176949 shutdown_connections 2023-08-25T09:07:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.376+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/3437176949 wait complete. 2023-08-25T09:07:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.377+0000 7f2ea9bacb80 1 -- start start 2023-08-25T09:07:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.377+0000 7f2ea9bacb80 1 Processor -- start 2023-08-25T09:07:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.377+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fba6870 0x558a1fbb4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.377+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.377+0000 7f2ea9bacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.377+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fbb5540 con 0x558a1fba6870 2023-08-25T09:07:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.377+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fbb4300 con 0x558a1fbb8860 2023-08-25T09:07:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.377+0000 7f2ea9bacb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1fbb44c0 con 0x558a1fbb2990 2023-08-25T09:07:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2ea9bacb80 10 client.0 _ll_register_callbacks cb 0x7f2e80004f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51802/0 (socket says 172.21.15.99:51802) 2023-08-25T09:07:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8ffff700 1 -- 172.21.15.99:0/4081939370 learned_addr learned my addr 172.21.15.99:0/4081939370 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8effd700 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fba6870 0x558a1fbb4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8ffff700 1 -- 172.21.15.99:0/4081939370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 msgr2=0x558a1fbb3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8ffff700 1 -- 172.21.15.99:0/4081939370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fba6870 msgr2=0x558a1fbb4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fba6870 0x558a1fbb4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8ffff700 1 -- 172.21.15.99:0/4081939370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e901a5d50 con 0x558a1fbb8860 2023-08-25T09:07:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.378+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fba6870 0x558a1fbb4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.379+0000 7f2e8ffff700 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb3dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800bc70 tx=0x7f2e8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d97e38f075a2f69a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.379+0000 7f2e6effd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:07:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.379+0000 7f2e6effd700 1 -- 172.21.15.99:0/4081939370 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800e040 con 0x558a1fbb8860 2023-08-25T09:07:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.379+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/4081939370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a1fba5600 con 0x558a1fbb8860 2023-08-25T09:07:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.379+0000 7f2e6effd700 1 -- 172.21.15.99:0/4081939370 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2e880077b0 con 0x558a1fbb8860 2023-08-25T09:07:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.379+0000 7f2e6effd700 1 -- 172.21.15.99:0/4081939370 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e88005860 con 0x558a1fbb8860 2023-08-25T09:07:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.379+0000 7f2ea9bacb80 10 client.25931 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.379+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/4081939370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558a1f9571f0 con 0x558a1fbb8860 2023-08-25T09:07:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2e6effd700 1 -- 172.21.15.99:0/4081939370 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e88037020 con 0x558a1fbb8860 2023-08-25T09:07:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2e6effd700 1 -- 172.21.15.99:0/4081939370 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2e8801f440 con 0x558a1fbb8860 2023-08-25T09:07:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2ea9bacb80 10 client.25931 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2ea9bacb80 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2ea9bacb80 20 client.25931 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558a1fc21210 0x558a1fc23630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2ea9bacb80 4 client.25931 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:07:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/4081939370 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558a1fafd910 con 0x558a1fc21210 2023-08-25T09:07:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.380+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558a1fc21210 0x558a1fc23630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.381+0000 7f2e8f7fe700 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558a1fc21210 0x558a1fc23630 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3ebec299c3bdb4 server_cookie=5fdbd652b439d5eb in_seq=0 out_seq=0 2023-08-25T09:07:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.381+0000 7f2e6effd700 10 client.25931 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2e6effd700 1 -- 172.21.15.99:0/4081939370 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558a1fafd910 con 0x558a1fc21210 2023-08-25T09:07:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2e6effd700 10 client.25931 handle_command_reply: tid=0 2023-08-25T09:07:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 client.25931 shutdown 2023-08-25T09:07:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/4081939370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 msgr2=0x558a1fbb3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb3dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800bc70 tx=0x7f2e8800b330 comp rx=0 tx=0).stop 2023-08-25T09:07:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/4081939370 shutdown_connections 2023-08-25T09:07:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558a1fc21210 0x558a1fc23630 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a1fbb8860 0x558a1fbb3dc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a1fba6870 0x558a1fbb4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2e6effd700 0 client.25931 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 --2- 172.21.15.99:0/4081939370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1fbb2990 0x558a1fbb3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/4081939370 >> 172.21.15.99:0/4081939370 conn(0x558a1fb27080 msgr2=0x558a1fbb59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/4081939370 shutdown_connections 2023-08-25T09:07:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 1 -- 172.21.15.99:0/4081939370 wait complete. 2023-08-25T09:07:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.443+0000 7f2ea9bacb80 20 client.25931 trim_cache size 0 max 16384 2023-08-25T09:07:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:07:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:07:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:07:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:07:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 -- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e780a00c0 msgr2=0x7f2e780a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e780a00c0 0x7f2e780a2580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009790 tx=0x7f2e88007e20 comp rx=0 tx=0).stop 2023-08-25T09:07:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 -- 172.21.15.99:0/3101573815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e90072d50 msgr2=0x7f2e90198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e90072d50 0x7f2e90198ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000e9c0 tx=0x7f2e8000ecd0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 -- 172.21.15.99:0/3101573815 shutdown_connections 2023-08-25T09:07:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e780a00c0 0x7f2e780a2580 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9010d0e0 0x7f2e9019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e90072d50 0x7f2e90198ce0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 --2- 172.21.15.99:0/3101573815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e90073670 0x7f2e90199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 -- 172.21.15.99:0/3101573815 >> 172.21.15.99:0/3101573815 conn(0x7f2e900fc370 msgr2=0x7f2e90104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 -- 172.21.15.99:0/3101573815 shutdown_connections 2023-08-25T09:07:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:16.444+0000 7f2e961ff700 1 -- 172.21.15.99:0/3101573815 wait complete. 2023-08-25T09:07:16.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 305405 3.98 MB/sec execute 322 sec latency 34.712 ms 2023-08-25T09:07:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:17 smithi157 ceph-mon[130879]: pgmap v250: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 236 op/s 2023-08-25T09:07:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:17 smithi181 ceph-mon[131296]: pgmap v250: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 236 op/s 2023-08-25T09:07:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:17 smithi099 ceph-mon[97641]: pgmap v250: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 236 op/s 2023-08-25T09:07:17.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306677 3.98 MB/sec execute 323 sec latency 45.381 ms 2023-08-25T09:07:18.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.97 MB/sec execute 324 sec latency 999.887 ms 2023-08-25T09:07:19.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:19 smithi157 ceph-mon[130879]: pgmap v251: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2023-08-25T09:07:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:19 smithi181 ceph-mon[131296]: pgmap v251: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2023-08-25T09:07:19.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:19 smithi099 ceph-mon[97641]: pgmap v251: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2023-08-25T09:07:19.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.96 MB/sec execute 325 sec latency 2000.038 ms 2023-08-25T09:07:20.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.95 MB/sec execute 326 sec latency 3000.183 ms 2023-08-25T09:07:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:21 smithi157 ceph-mon[130879]: pgmap v252: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2023-08-25T09:07:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:21 smithi181 ceph-mon[131296]: pgmap v252: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2023-08-25T09:07:21.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:21 smithi099 ceph-mon[97641]: pgmap v252: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2023-08-25T09:07:21.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.93 MB/sec execute 327 sec latency 4000.333 ms 2023-08-25T09:07:22.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.92 MB/sec execute 328 sec latency 5000.491 ms 2023-08-25T09:07:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:23 smithi157 ceph-mon[130879]: pgmap v253: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2023-08-25T09:07:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:23 smithi181 ceph-mon[131296]: pgmap v253: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2023-08-25T09:07:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:23 smithi099 ceph-mon[97641]: pgmap v253: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2023-08-25T09:07:23.927 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.91 MB/sec execute 329 sec latency 6000.602 ms 2023-08-25T09:07:24.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.90 MB/sec execute 330 sec latency 7000.714 ms 2023-08-25T09:07:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:25 smithi181 ceph-mon[131296]: pgmap v254: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 188 op/s 2023-08-25T09:07:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:25 smithi099 ceph-mon[97641]: pgmap v254: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 188 op/s 2023-08-25T09:07:25.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:25 smithi157 ceph-mon[130879]: pgmap v254: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 188 op/s 2023-08-25T09:07:25.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.89 MB/sec execute 331 sec latency 8000.858 ms 2023-08-25T09:07:26.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:26 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:07:26.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:26 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:07:26.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:26 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:07:26.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.87 MB/sec execute 332 sec latency 9000.923 ms 2023-08-25T09:07:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:27 smithi157 ceph-mon[130879]: pgmap v255: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2023-08-25T09:07:27.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:27 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:07:27.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:27 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:07:27.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:27 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:07:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:27 smithi181 ceph-mon[131296]: pgmap v255: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2023-08-25T09:07:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:27 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:07:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:27 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:07:27.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:27 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:07:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:27 smithi099 ceph-mon[97641]: pgmap v255: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2023-08-25T09:07:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:27 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:07:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:27 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:07:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:27 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:07:27.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.86 MB/sec execute 333 sec latency 10001.067 ms 2023-08-25T09:07:28.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306684 3.85 MB/sec execute 334 sec latency 11001.204 ms 2023-08-25T09:07:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.611+0000 7f9cc3bff700 1 -- 172.21.15.99:0/379688601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 msgr2=0x7f9cbc106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.611+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/379688601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 0x7f9cbc106c90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009990 tx=0x7f9cb8009ca0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.612+0000 7f9cc3bff700 1 -- 172.21.15.99:0/379688601 shutdown_connections 2023-08-25T09:07:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.612+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/379688601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cbc101260 0x7f9cbc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.612+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/379688601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 0x7f9cbc106c90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.612+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/379688601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cbc1008a0 0x7f9cbc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.612+0000 7f9cc3bff700 1 -- 172.21.15.99:0/379688601 >> 172.21.15.99:0/379688601 conn(0x7f9cbc0fc370 msgr2=0x7f9cbc0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.612+0000 7f9cc3bff700 1 -- 172.21.15.99:0/379688601 shutdown_connections 2023-08-25T09:07:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.612+0000 7f9cc3bff700 1 -- 172.21.15.99:0/379688601 wait complete. 2023-08-25T09:07:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc3bff700 1 Processor -- start 2023-08-25T09:07:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc3bff700 1 -- start start 2023-08-25T09:07:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc3bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cbc1008a0 0x7f9cbc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc3bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cbc101260 0x7f9cbc19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc3bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 0x7f9cbc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc3bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc199180 con 0x7f9cbc1068b0 2023-08-25T09:07:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc3bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc1992f0 con 0x7f9cbc1008a0 2023-08-25T09:07:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc3bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc1994b0 con 0x7f9cbc101260 2023-08-25T09:07:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cbc1008a0 0x7f9cbc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cbc1008a0 0x7f9cbc19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:45054/0 (socket says 172.21.15.99:45054) 2023-08-25T09:07:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.613+0000 7f9cc199b700 1 -- 172.21.15.99:0/4159205732 learned_addr learned my addr 172.21.15.99:0/4159205732 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc219c700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 0x7f9cbc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc119a700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cbc101260 0x7f9cbc19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc199b700 1 -- 172.21.15.99:0/4159205732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cbc101260 msgr2=0x7f9cbc19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc199b700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cbc101260 0x7f9cbc19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc199b700 1 -- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 msgr2=0x7f9cbc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc199b700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 0x7f9cbc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc199b700 1 -- 172.21.15.99:0/4159205732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb80095f0 con 0x7f9cbc1008a0 2023-08-25T09:07:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc119a700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cbc101260 0x7f9cbc19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc219c700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 0x7f9cbc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc199b700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cbc1008a0 0x7f9cbc19ebe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005910 tx=0x7f9cb8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cb2ffd700 1 -- 172.21.15.99:0/4159205732 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb801d070 con 0x7f9cbc1008a0 2023-08-25T09:07:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.614+0000 7f9cc3bff700 1 -- 172.21.15.99:0/4159205732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc199730 con 0x7f9cbc1008a0 2023-08-25T09:07:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.615+0000 7f9cb2ffd700 1 -- 172.21.15.99:0/4159205732 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb8022470 con 0x7f9cbc1008a0 2023-08-25T09:07:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.615+0000 7f9cb2ffd700 1 -- 172.21.15.99:0/4159205732 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8017650 con 0x7f9cbc1008a0 2023-08-25T09:07:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.615+0000 7f9cc3bff700 1 -- 172.21.15.99:0/4159205732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc1a5b60 con 0x7f9cbc1008a0 2023-08-25T09:07:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.618+0000 7f9cb2ffd700 1 -- 172.21.15.99:0/4159205732 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9cb800fad0 con 0x7f9cbc1008a0 2023-08-25T09:07:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.620+0000 7f9cb2ffd700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ca80a00c0 0x7f9ca80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.620+0000 7f9cd75a1b80 1 Processor -- start 2023-08-25T09:07:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.620+0000 7f9cb2ffd700 1 -- 172.21.15.99:0/4159205732 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9cb80ccb60 con 0x7f9cbc1008a0 2023-08-25T09:07:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.620+0000 7f9cc119a700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ca80a00c0 0x7f9ca80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.620+0000 7f9cd75a1b80 1 -- start start 2023-08-25T09:07:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.620+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0bf40 0x5584adb0c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.620+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 0x5584adb06450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ada6e750 con 0x5584adb06990 2023-08-25T09:07:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ada5f720 con 0x5584adb0c860 2023-08-25T09:07:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ada5d040 con 0x5584adb0bf40 2023-08-25T09:07:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc119a700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ca80a00c0 0x7f9ca80a2580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc19a3f0 tx=0x7f9cac00b410 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0bf40 0x5584adb0c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0bf40 0x5584adb0c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:41666/0 (socket says 172.21.15.99:41666) 2023-08-25T09:07:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 -- 172.21.15.99:0/3416091061 learned_addr learned my addr 172.21.15.99:0/3416091061 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc219c700 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc119a700 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 0x5584adb06450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 -- 172.21.15.99:0/3416091061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 msgr2=0x5584adb06450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 0x5584adb06450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 -- 172.21.15.99:0/3416091061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 msgr2=0x5584adb06dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 -- 172.21.15.99:0/3416091061 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584ad8d85d0 con 0x5584adb0bf40 2023-08-25T09:07:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.621+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0bf40 0x5584adb0c320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8096730 tx=0x7f9cb809a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=541102c001546854 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9ca7fff700 1 -- 172.21.15.99:0/3416091061 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb809a230 con 0x5584adb0bf40 2023-08-25T09:07:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9ca7fff700 1 -- 172.21.15.99:0/3416091061 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb809e060 con 0x5584adb0bf40 2023-08-25T09:07:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9ca7fff700 1 -- 172.21.15.99:0/3416091061 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80986b0 con 0x5584adb0bf40 2023-08-25T09:07:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3416091061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0bf40 msgr2=0x5584adb0c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0bf40 0x5584adb0c320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8096730 tx=0x7f9cb809a100 comp rx=0 tx=0).stop 2023-08-25T09:07:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3416091061 shutdown_connections 2023-08-25T09:07:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0bf40 0x5584adb0c320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3416091061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 0x5584adb06450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.622+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3416091061 >> 172.21.15.99:0/3416091061 conn(0x5584ada7b080 msgr2=0x5584adb00d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.623+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3416091061 shutdown_connections 2023-08-25T09:07:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.623+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3416091061 wait complete. 2023-08-25T09:07:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.623+0000 7f9cd75a1b80 1 -- start start 2023-08-25T09:07:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.623+0000 7f9cd75a1b80 1 Processor -- start 2023-08-25T09:07:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0bf40 0x5584adb0f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 0x5584adb0ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ada58930 con 0x5584adb0bf40 2023-08-25T09:07:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ad9f6b50 con 0x5584adb0c860 2023-08-25T09:07:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ad9f6850 con 0x5584adb06990 2023-08-25T09:07:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cd75a1b80 10 client.0 _ll_register_callbacks cb 0x5584ad940a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb125e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:41682/0 (socket says 172.21.15.99:41682) 2023-08-25T09:07:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc199b700 1 -- 172.21.15.99:0/3136180995 learned_addr learned my addr 172.21.15.99:0/3136180995 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc119a700 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0bf40 0x5584adb0f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc219c700 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 0x5584adb0ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc199b700 1 -- 172.21.15.99:0/3136180995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 msgr2=0x5584adb0ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 0x5584adb0ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc199b700 1 -- 172.21.15.99:0/3136180995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0bf40 msgr2=0x5584adb0f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0bf40 0x5584adb0f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.624+0000 7f9cc199b700 1 -- 172.21.15.99:0/3136180995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc1a5b60 con 0x5584adb06990 2023-08-25T09:07:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb125e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80ca760 tx=0x7f9cb809e110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e046584cd3ac4aa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9ca57fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:07:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9ca57fa700 1 -- 172.21.15.99:0/3136180995 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80b6070 con 0x5584adb06990 2023-08-25T09:07:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3136180995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584adad6160 con 0x5584adb06990 2023-08-25T09:07:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9ca57fa700 1 -- 172.21.15.99:0/3136180995 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9cb80b9040 con 0x5584adb06990 2023-08-25T09:07:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9ca57fa700 1 -- 172.21.15.99:0/3136180995 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb809a2b0 con 0x5584adb06990 2023-08-25T09:07:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9cd75a1b80 10 client.25927 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3136180995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5584ad884d80 con 0x5584adb06990 2023-08-25T09:07:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9ca57fa700 1 -- 172.21.15.99:0/3136180995 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584adad6160 con 0x5584adb06990 2023-08-25T09:07:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.625+0000 7f9ca57fa700 1 -- 172.21.15.99:0/3136180995 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9cb809a490 con 0x5584adb06990 2023-08-25T09:07:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9cd75a1b80 10 client.25927 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9cd75a1b80 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9cd75a1b80 20 client.25927 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5584adb1f9c0 0x5584adb21de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9cd75a1b80 4 client.25927 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:07:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3136180995 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584ad9feb40 con 0x5584adb1f9c0 2023-08-25T09:07:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9cc119a700 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5584adb1f9c0 0x5584adb21de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9cc119a700 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5584adb1f9c0 0x5584adb21de0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a1a9c3b89b6e1c server_cookie=548b3888ae2dcfec in_seq=0 out_seq=0 2023-08-25T09:07:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.626+0000 7f9ca57fa700 10 client.25927 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.627+0000 7f9ca57fa700 1 -- 172.21.15.99:0/3136180995 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5584ad9feb40 con 0x5584adb1f9c0 2023-08-25T09:07:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.627+0000 7f9ca57fa700 10 client.25927 handle_command_reply: tid=0 2023-08-25T09:07:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.627+0000 7f9cd75a1b80 1 client.25927 shutdown 2023-08-25T09:07:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3136180995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 msgr2=0x5584adb125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb125e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80ca760 tx=0x7f9cb809e110 comp rx=0 tx=0).stop 2023-08-25T09:07:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3136180995 shutdown_connections 2023-08-25T09:07:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5584adb1f9c0 0x5584adb21de0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0c860 0x5584adb0ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb125e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9ca57fa700 0 client.25927 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3136180995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0bf40 0x5584adb0f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3136180995 >> 172.21.15.99:0/3136180995 conn(0x5584ada7b080 msgr2=0x5584adafb820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3136180995 shutdown_connections 2023-08-25T09:07:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3136180995 wait complete. 2023-08-25T09:07:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.628+0000 7f9cd75a1b80 20 client.25927 trim_cache size 0 max 16384 2023-08-25T09:07:29.641 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:29 smithi157 ceph-mon[130879]: pgmap v256: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2023-08-25T09:07:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.644+0000 7f9cd75a1b80 1 Processor -- start 2023-08-25T09:07:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.644+0000 7f9cd75a1b80 1 -- start start 2023-08-25T09:07:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.644+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.644+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb16fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.644+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0c860 0x5584adb0f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.644+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ad8da8c0 con 0x5584adb06990 2023-08-25T09:07:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.644+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584adafd9d0 con 0x5584adb0bf40 2023-08-25T09:07:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.644+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584adb17500 con 0x5584adb0c860 2023-08-25T09:07:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc219c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0c860 0x5584adb0f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60306/0 (socket says 172.21.15.99:60306) 2023-08-25T09:07:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 -- 172.21.15.99:0/3734881083 learned_addr learned my addr 172.21.15.99:0/3734881083 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc119a700 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb16fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 -- 172.21.15.99:0/3734881083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0c860 msgr2=0x5584adb0f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0c860 0x5584adb0f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 -- 172.21.15.99:0/3734881083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 msgr2=0x5584adb16fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb16fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 -- 172.21.15.99:0/3734881083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584adb07040 con 0x5584adb06990 2023-08-25T09:07:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc219c700 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0c860 0x5584adb0f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cc199b700 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8098580 tx=0x7f9cb8085020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c98bcd9cccb3889 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.645+0000 7f9cb0bff700 1 -- 172.21.15.99:0/3734881083 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80b6070 con 0x5584adb06990 2023-08-25T09:07:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cb0bff700 1 -- 172.21.15.99:0/3734881083 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb8099d70 con 0x5584adb06990 2023-08-25T09:07:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cb0bff700 1 -- 172.21.15.99:0/3734881083 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80b21e0 con 0x5584adb06990 2023-08-25T09:07:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3734881083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 msgr2=0x5584adb06450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8098580 tx=0x7f9cb8085020 comp rx=0 tx=0).stop 2023-08-25T09:07:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3734881083 shutdown_connections 2023-08-25T09:07:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb0c860 0x5584adb0f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb06990 0x5584adb06450 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/3734881083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb16fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3734881083 >> 172.21.15.99:0/3734881083 conn(0x5584ada7b080 msgr2=0x5584adb00bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3734881083 shutdown_connections 2023-08-25T09:07:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.646+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/3734881083 wait complete. 2023-08-25T09:07:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.647+0000 7f9cd75a1b80 1 -- start start 2023-08-25T09:07:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.647+0000 7f9cd75a1b80 1 Processor -- start 2023-08-25T09:07:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.647+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb08ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb09000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cd75a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0c860 0x5584adb07880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584adb07dc0 con 0x5584adb0c860 2023-08-25T09:07:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584adb07f30 con 0x5584adb0bf40 2023-08-25T09:07:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cd75a1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584adb080f0 con 0x5584adb06990 2023-08-25T09:07:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cd75a1b80 10 client.0 _ll_register_callbacks cb 0x7f9cb4009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb09000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc219c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0c860 0x5584adb07880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb08ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb09000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:45098/0 (socket says 172.21.15.99:45098) 2023-08-25T09:07:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 -- 172.21.15.99:0/2223734491 learned_addr learned my addr 172.21.15.99:0/2223734491 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb08ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:41696/0 (socket says 172.21.15.99:41696) 2023-08-25T09:07:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc219c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0c860 0x5584adb07880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60316/0 (socket says 172.21.15.99:60316) 2023-08-25T09:07:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 -- 172.21.15.99:0/2223734491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 msgr2=0x5584adb08ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb08ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 -- 172.21.15.99:0/2223734491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0c860 msgr2=0x5584adb07880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0c860 0x5584adb07880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 -- 172.21.15.99:0/2223734491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb40091c0 con 0x5584adb0bf40 2023-08-25T09:07:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc219c700 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0c860 0x5584adb07880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9cc119a700 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb09000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc19a480 tx=0x7f9cac0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e149ccf46339d3c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.648+0000 7f9ca57fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:07:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.649+0000 7f9ca57fa700 1 -- 172.21.15.99:0/2223734491 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac024040 con 0x5584adb0bf40 2023-08-25T09:07:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.649+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/2223734491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584adaf9600 con 0x5584adb0bf40 2023-08-25T09:07:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.649+0000 7f9ca57fa700 1 -- 172.21.15.99:0/2223734491 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9cac017070 con 0x5584adb0bf40 2023-08-25T09:07:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.649+0000 7f9ca57fa700 1 -- 172.21.15.99:0/2223734491 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cac004220 con 0x5584adb0bf40 2023-08-25T09:07:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.649+0000 7f9cd75a1b80 10 client.25949 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.649+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/2223734491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5584ad884d80 con 0x5584adb0bf40 2023-08-25T09:07:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.649+0000 7f9ca57fa700 1 -- 172.21.15.99:0/2223734491 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584adaf9600 con 0x5584adb0bf40 2023-08-25T09:07:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9ca57fa700 1 -- 172.21.15.99:0/2223734491 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9cac003710 con 0x5584adb0bf40 2023-08-25T09:07:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9cd75a1b80 10 client.25949 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9cd75a1b80 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9cd75a1b80 20 client.25949 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5584adb74760 0x5584adb76b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9cd75a1b80 4 client.25949 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:07:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/2223734491 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584ada51910 con 0x5584adb74760 2023-08-25T09:07:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9cc199b700 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5584adb74760 0x5584adb76b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9ca57fa700 10 client.25949 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.650+0000 7f9cc199b700 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5584adb74760 0x5584adb76b80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e810f9d9c4c6458c server_cookie=bf3f0d6b0c1ab4f0 in_seq=0 out_seq=0 2023-08-25T09:07:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.686+0000 7f9ca57fa700 1 -- 172.21.15.99:0/2223734491 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584ada51910 con 0x5584adb74760 2023-08-25T09:07:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9ca57fa700 10 client.25949 handle_command_reply: tid=0 2023-08-25T09:07:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 client.25949 shutdown 2023-08-25T09:07:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/2223734491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 msgr2=0x5584adb09000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb09000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc19a480 tx=0x7f9cac0047c0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/2223734491 shutdown_connections 2023-08-25T09:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584adb0c860 0x5584adb07880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584adb06990 0x5584adb08ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5584adb74760 0x5584adb76b80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 --2- 172.21.15.99:0/2223734491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584adb0bf40 0x5584adb09000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/2223734491 >> 172.21.15.99:0/2223734491 conn(0x5584ada7b080 msgr2=0x5584adb09ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9ca57fa700 0 client.25949 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/2223734491 shutdown_connections 2023-08-25T09:07:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.687+0000 7f9cd75a1b80 1 -- 172.21.15.99:0/2223734491 wait complete. 2023-08-25T09:07:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.688+0000 7f9cd75a1b80 20 client.25949 trim_cache size 0 max 16384 2023-08-25T09:07:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:07:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:07:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:07:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:07:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.688+0000 7f9cc3bff700 1 -- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ca80a00c0 msgr2=0x7f9ca80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.688+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ca80a00c0 0x7f9ca80a2580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc19a3f0 tx=0x7f9cac00b410 comp rx=0 tx=0).stop 2023-08-25T09:07:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.688+0000 7f9cc3bff700 1 -- 172.21.15.99:0/4159205732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cbc1008a0 msgr2=0x7f9cbc19ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.688+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cbc1008a0 0x7f9cbc19ebe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005910 tx=0x7f9cb8005740 comp rx=0 tx=0).stop 2023-08-25T09:07:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.689+0000 7f9cc3bff700 1 -- 172.21.15.99:0/4159205732 shutdown_connections 2023-08-25T09:07:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.689+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ca80a00c0 0x7f9ca80a2580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.689+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cbc1068b0 0x7f9cbc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.689+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cbc1008a0 0x7f9cbc19ebe0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.689+0000 7f9cc3bff700 1 --2- 172.21.15.99:0/4159205732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cbc101260 0x7f9cbc19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.689+0000 7f9cc3bff700 1 -- 172.21.15.99:0/4159205732 >> 172.21.15.99:0/4159205732 conn(0x7f9cbc0fc370 msgr2=0x7f9cbc10bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.689+0000 7f9cc3bff700 1 -- 172.21.15.99:0/4159205732 shutdown_connections 2023-08-25T09:07:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:29.689+0000 7f9cc3bff700 1 -- 172.21.15.99:0/4159205732 wait complete. 2023-08-25T09:07:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:29 smithi181 ceph-mon[131296]: pgmap v256: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2023-08-25T09:07:29.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:29 smithi099 ceph-mon[97641]: pgmap v256: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2023-08-25T09:07:29.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 307544 3.85 MB/sec execute 335 sec latency 11539.329 ms 2023-08-25T09:07:30.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 309170 3.87 MB/sec execute 336 sec latency 30.347 ms 2023-08-25T09:07:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:31 smithi157 ceph-mon[130879]: pgmap v257: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2023-08-25T09:07:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:31 smithi181 ceph-mon[131296]: pgmap v257: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2023-08-25T09:07:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:31 smithi099 ceph-mon[97641]: pgmap v257: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2023-08-25T09:07:31.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 311062 3.88 MB/sec execute 337 sec latency 22.603 ms 2023-08-25T09:07:32.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 312633 3.90 MB/sec execute 338 sec latency 11.894 ms 2023-08-25T09:07:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:33 smithi157 ceph-mon[130879]: pgmap v258: 97 pgs: 97 active+clean; 109 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-08-25T09:07:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:33 smithi181 ceph-mon[131296]: pgmap v258: 97 pgs: 97 active+clean; 109 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-08-25T09:07:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:33 smithi099 ceph-mon[97641]: pgmap v258: 97 pgs: 97 active+clean; 109 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-08-25T09:07:33.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 314491 3.91 MB/sec execute 339 sec latency 19.269 ms 2023-08-25T09:07:34.928 INFO:tasks.workunit.client.0.smithi099.stdout: 1 316135 3.94 MB/sec execute 340 sec latency 13.344 ms 2023-08-25T09:07:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:35 smithi181 ceph-mon[131296]: pgmap v259: 97 pgs: 97 active+clean; 109 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2023-08-25T09:07:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:35 smithi099 ceph-mon[97641]: pgmap v259: 97 pgs: 97 active+clean; 109 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2023-08-25T09:07:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:35 smithi157 ceph-mon[130879]: pgmap v259: 97 pgs: 97 active+clean; 109 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2023-08-25T09:07:35.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 317372 3.94 MB/sec execute 341 sec latency 24.527 ms 2023-08-25T09:07:36.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 319517 3.97 MB/sec execute 342 sec latency 15.318 ms 2023-08-25T09:07:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:37 smithi157 ceph-mon[130879]: pgmap v260: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-08-25T09:07:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:37 smithi181 ceph-mon[131296]: pgmap v260: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-08-25T09:07:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:37 smithi099 ceph-mon[97641]: pgmap v260: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-08-25T09:07:37.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 320739 3.97 MB/sec execute 343 sec latency 24.398 ms 2023-08-25T09:07:38.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 322433 3.98 MB/sec execute 344 sec latency 15.986 ms 2023-08-25T09:07:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:39 smithi157 ceph-mon[130879]: pgmap v261: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2023-08-25T09:07:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:39 smithi181 ceph-mon[131296]: pgmap v261: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2023-08-25T09:07:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:39 smithi099 ceph-mon[97641]: pgmap v261: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2023-08-25T09:07:39.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 323516 3.99 MB/sec execute 345 sec latency 52.551 ms 2023-08-25T09:07:40.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 325577 4.01 MB/sec execute 346 sec latency 17.381 ms 2023-08-25T09:07:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:41 smithi157 ceph-mon[130879]: pgmap v262: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2023-08-25T09:07:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:41 smithi181 ceph-mon[131296]: pgmap v262: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2023-08-25T09:07:41.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:41 smithi099 ceph-mon[97641]: pgmap v262: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2023-08-25T09:07:41.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 326878 4.02 MB/sec execute 347 sec latency 24.633 ms 2023-08-25T09:07:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/636247290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b141068b0 msgr2=0x7f9b14106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/636247290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b141068b0 0x7f9b14106c90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009990 tx=0x7f9b08009ca0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/636247290 shutdown_connections 2023-08-25T09:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/636247290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b14101260 0x7f9b1410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/636247290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b141068b0 0x7f9b14106c90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/636247290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b141008a0 0x7f9b14100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/636247290 >> 172.21.15.99:0/636247290 conn(0x7f9b140fc370 msgr2=0x7f9b140fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/636247290 shutdown_connections 2023-08-25T09:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.844+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/636247290 wait complete. 2023-08-25T09:07:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.845+0000 7f9b1b1ff700 1 Processor -- start 2023-08-25T09:07:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.845+0000 7f9b1b1ff700 1 -- start start 2023-08-25T09:07:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b1b1ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b141008a0 0x7f9b1419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b1b1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b14101260 0x7f9b1419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b1b1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b141068b0 0x7f9b141989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b1b1ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b14199010 con 0x7f9b141008a0 2023-08-25T09:07:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b1b1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b14199180 con 0x7f9b14101260 2023-08-25T09:07:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b1b1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b14199340 con 0x7f9b141068b0 2023-08-25T09:07:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b18f9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b141008a0 0x7f9b1419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b1979c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b141068b0 0x7f9b141989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b14101260 0x7f9b1419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b14101260 0x7f9b1419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:43112/0 (socket says 172.21.15.99:43112) 2023-08-25T09:07:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b1979c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b141068b0 0x7f9b141989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45080/0 (socket says 172.21.15.99:45080) 2023-08-25T09:07:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 -- 172.21.15.99:0/2195876796 learned_addr learned my addr 172.21.15.99:0/2195876796 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 -- 172.21.15.99:0/2195876796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b141068b0 msgr2=0x7f9b141989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b141068b0 0x7f9b141989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 -- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b141008a0 msgr2=0x7f9b1419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b141008a0 0x7f9b1419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 -- 172.21.15.99:0/2195876796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b00009520 con 0x7f9b14101260 2023-08-25T09:07:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.846+0000 7f9b13fff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b14101260 0x7f9b1419f080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000eb30 tx=0x7f9b0000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.847+0000 7f9b11ffb700 1 -- 172.21.15.99:0/2195876796 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00009880 con 0x7f9b14101260 2023-08-25T09:07:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.847+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/2195876796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b080095f0 con 0x7f9b14101260 2023-08-25T09:07:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.847+0000 7f9b11ffb700 1 -- 172.21.15.99:0/2195876796 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b000044e0 con 0x7f9b14101260 2023-08-25T09:07:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.847+0000 7f9b11ffb700 1 -- 172.21.15.99:0/2195876796 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00003d80 con 0x7f9b14101260 2023-08-25T09:07:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.847+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/2195876796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b141a5950 con 0x7f9b14101260 2023-08-25T09:07:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.850+0000 7f9b11ffb700 1 -- 172.21.15.99:0/2195876796 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9b00003f20 con 0x7f9b14101260 2023-08-25T09:07:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.851+0000 7f9b2eb90b80 1 Processor -- start 2023-08-25T09:07:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.851+0000 7f9b11ffb700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9b040a00c0 0x7f9b040a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b2eb90b80 1 -- start start 2023-08-25T09:07:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b11ffb700 1 -- 172.21.15.99:0/2195876796 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9b00014070 con 0x7f9b14101260 2023-08-25T09:07:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b1979c700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9b040a00c0 0x7f9b040a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ff1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494feb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494febdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494f53750 con 0x561494ff0f40 2023-08-25T09:07:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494f44720 con 0x561494feb990 2023-08-25T09:07:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494f42040 con 0x561494ff1860 2023-08-25T09:07:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b13fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494feb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b18f9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ff1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b13fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494feb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45094/0 (socket says 172.21.15.99:45094) 2023-08-25T09:07:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b13fff700 1 -- 172.21.15.99:0/1467046780 learned_addr learned my addr 172.21.15.99:0/1467046780 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.852+0000 7f9b1979c700 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494febdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9b13fff700 1 -- 172.21.15.99:0/1467046780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 msgr2=0x561494febdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9b13fff700 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494febdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9b13fff700 1 -- 172.21.15.99:0/1467046780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 msgr2=0x561494ff1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9b13fff700 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ff1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9b13fff700 1 -- 172.21.15.99:0/1467046780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561494dbd5d0 con 0x561494ff1860 2023-08-25T09:07:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9b1979c700 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494febdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:07:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9b1979c700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9b040a00c0 0x7f9b040a2580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b1419f840 tx=0x7f9b0c00fc00 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9b13fff700 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494feb450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9b000a0b80 tx=0x7f9b000991a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=742452eaa8efb6a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9afeffd700 1 -- 172.21.15.99:0/1467046780 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b000992a0 con 0x561494ff1860 2023-08-25T09:07:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9afeffd700 1 -- 172.21.15.99:0/1467046780 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b0009d0d0 con 0x561494ff1860 2023-08-25T09:07:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.853+0000 7f9afeffd700 1 -- 172.21.15.99:0/1467046780 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b000b1220 con 0x561494ff1860 2023-08-25T09:07:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/1467046780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 msgr2=0x561494feb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494feb450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9b000a0b80 tx=0x7f9b000991a0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/1467046780 shutdown_connections 2023-08-25T09:07:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494febdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ff1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/1467046780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494feb450 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/1467046780 >> 172.21.15.99:0/1467046780 conn(0x561494f60080 msgr2=0x561494fe5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/1467046780 shutdown_connections 2023-08-25T09:07:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.854+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/1467046780 wait complete. 2023-08-25T09:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 1 -- start start 2023-08-25T09:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 1 Processor -- start 2023-08-25T09:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494feb990 0x561494ff7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ff4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494f3d930 con 0x561494feb990 2023-08-25T09:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494edbb50 con 0x561494ff0f40 2023-08-25T09:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494edb850 con 0x561494ff1860 2023-08-25T09:07:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.855+0000 7f9b2eb90b80 10 client.0 _ll_register_callbacks cb 0x561494e25a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ff4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b13fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ff4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45098/0 (socket says 172.21.15.99:45098) 2023-08-25T09:07:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 -- 172.21.15.99:0/641263355 learned_addr learned my addr 172.21.15.99:0/641263355 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494feb990 0x561494ff7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 -- 172.21.15.99:0/641263355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 msgr2=0x561494ff4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 -- 172.21.15.99:0/641263355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494feb990 msgr2=0x561494ff7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494feb990 0x561494ff7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 -- 172.21.15.99:0/641263355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b141a5950 con 0x561494ff1860 2023-08-25T09:07:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b13fff700 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9b1979c700 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ff4fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c018b80 tx=0x7f9b0c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28a332bac222c664 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9af3fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:07:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.856+0000 7f9af3fff700 1 -- 172.21.15.99:0/641263355 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c00e040 con 0x561494ff1860 2023-08-25T09:07:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/641263355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561494fbb160 con 0x561494ff1860 2023-08-25T09:07:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9af3fff700 1 -- 172.21.15.99:0/641263355 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9b0c0093f0 con 0x561494ff1860 2023-08-25T09:07:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9af3fff700 1 -- 172.21.15.99:0/641263355 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b0c008920 con 0x561494ff1860 2023-08-25T09:07:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 10 client.25945 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/641263355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561494d69d80 con 0x561494ff1860 2023-08-25T09:07:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9af3fff700 1 -- 172.21.15.99:0/641263355 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561494fbb160 con 0x561494ff1860 2023-08-25T09:07:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9af3fff700 1 -- 172.21.15.99:0/641263355 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9b0c024bb0 con 0x561494ff1860 2023-08-25T09:07:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 10 client.25945 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 10 client.25945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 20 client.25945 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5614950051c0 0x5614950075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 4 client.25945 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:07:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.857+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/641263355 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561494ee3b40 con 0x5614950051c0 2023-08-25T09:07:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.858+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5614950051c0 0x5614950075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.858+0000 7f9af3fff700 10 client.25945 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.858+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5614950051c0 0x5614950075e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3439905b0d83042c server_cookie=b4af85f90c4deb81 in_seq=0 out_seq=0 2023-08-25T09:07:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9af3fff700 1 -- 172.21.15.99:0/641263355 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561494ee3b40 con 0x5614950051c0 2023-08-25T09:07:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9af3fff700 10 client.25945 handle_command_reply: tid=0 2023-08-25T09:07:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 client.25945 shutdown 2023-08-25T09:07:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/641263355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 msgr2=0x561494ff4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ff4fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c018b80 tx=0x7f9b0c004b40 comp rx=0 tx=0).stop 2023-08-25T09:07:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/641263355 shutdown_connections 2023-08-25T09:07:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ff4fc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494feb990 0x561494ff7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5614950051c0 0x5614950075e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/641263355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/641263355 >> 172.21.15.99:0/641263355 conn(0x561494f60080 msgr2=0x561494fe0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9af3fff700 0 client.25945 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.859+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/641263355 shutdown_connections 2023-08-25T09:07:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.860+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/641263355 wait complete. 2023-08-25T09:07:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.860+0000 7f9b2eb90b80 20 client.25945 trim_cache size 0 max 16384 2023-08-25T09:07:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.875+0000 7f9b2eb90b80 1 Processor -- start 2023-08-25T09:07:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.875+0000 7f9b2eb90b80 1 -- start start 2023-08-25T09:07:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.875+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494feb990 0x561494feb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.875+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.875+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff1860 0x561494ff2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.875+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494dbf8c0 con 0x561494ff1860 2023-08-25T09:07:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.875+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494ff5380 con 0x561494ff0f40 2023-08-25T09:07:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.875+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494fde4c0 con 0x561494feb990 2023-08-25T09:07:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494feb990 0x561494feb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b1979c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff1860 0x561494ff2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494feb990 0x561494feb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45100/0 (socket says 172.21.15.99:45100) 2023-08-25T09:07:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 -- 172.21.15.99:0/2552534211 learned_addr learned my addr 172.21.15.99:0/2552534211 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b13fff700 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 -- 172.21.15.99:0/2552534211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 msgr2=0x561494ff1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 -- 172.21.15.99:0/2552534211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff1860 msgr2=0x561494ff2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff1860 0x561494ff2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 -- 172.21.15.99:0/2552534211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561494d52c00 con 0x561494feb990 2023-08-25T09:07:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b1979c700 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff1860 0x561494ff2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.876+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494feb990 0x561494feb450 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b6f0 tx=0x7f9b08005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7e836837d18c33 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9aff7fe700 1 -- 172.21.15.99:0/2552534211 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0801d070 con 0x561494feb990 2023-08-25T09:07:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9aff7fe700 1 -- 172.21.15.99:0/2552534211 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b0801e040 con 0x561494feb990 2023-08-25T09:07:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9aff7fe700 1 -- 172.21.15.99:0/2552534211 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08005670 con 0x561494feb990 2023-08-25T09:07:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2552534211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494feb990 msgr2=0x561494feb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494feb990 0x561494feb450 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b6f0 tx=0x7f9b08005fb0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2552534211 shutdown_connections 2023-08-25T09:07:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff1860 0x561494ff2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494feb990 0x561494feb450 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2552534211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494ff0f40 0x561494ff1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2552534211 >> 172.21.15.99:0/2552534211 conn(0x561494f60080 msgr2=0x561494feeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2552534211 shutdown_connections 2023-08-25T09:07:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.877+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2552534211 wait complete. 2023-08-25T09:07:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.878+0000 7f9b2eb90b80 1 -- start start 2023-08-25T09:07:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.878+0000 7f9b2eb90b80 1 Processor -- start 2023-08-25T09:07:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494ffa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ffaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b2eb90b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ffb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494ffbf20 con 0x561494ff0f40 2023-08-25T09:07:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494ffc090 con 0x561494feb990 2023-08-25T09:07:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b2eb90b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561494ffc250 con 0x561494ff1860 2023-08-25T09:07:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b2eb90b80 10 client.0 _ll_register_callbacks cb 0x7f9b0c009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b18f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494ffa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b1979c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ffb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b13fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ffaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b18f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494ffa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:43132/0 (socket says 172.21.15.99:43132) 2023-08-25T09:07:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b18f9b700 1 -- 172.21.15.99:0/2873387710 learned_addr learned my addr 172.21.15.99:0/2873387710 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b1979c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ffb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45104/0 (socket says 172.21.15.99:45104) 2023-08-25T09:07:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b18f9b700 1 -- 172.21.15.99:0/2873387710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 msgr2=0x561494ffb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ffb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b18f9b700 1 -- 172.21.15.99:0/2873387710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 msgr2=0x561494ffaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ffaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b18f9b700 1 -- 172.21.15.99:0/2873387710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561494d69d80 con 0x561494feb990 2023-08-25T09:07:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.879+0000 7f9b13fff700 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ffaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9b18f9b700 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494ffa500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b08000c00 tx=0x7f9b0802ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14a2dd5c4e5aa90f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9af3fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:07:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9af3fff700 1 -- 172.21.15.99:0/2873387710 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0801d040 con 0x561494feb990 2023-08-25T09:07:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2873387710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561494fde600 con 0x561494feb990 2023-08-25T09:07:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9af3fff700 1 -- 172.21.15.99:0/2873387710 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9b08029150 con 0x561494feb990 2023-08-25T09:07:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9af3fff700 1 -- 172.21.15.99:0/2873387710 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b08029310 con 0x561494feb990 2023-08-25T09:07:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9b2eb90b80 10 client.25967 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2873387710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f9b1410e070 con 0x561494feb990 2023-08-25T09:07:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9af3fff700 1 -- 172.21.15.99:0/2873387710 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561494fde600 con 0x561494feb990 2023-08-25T09:07:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.880+0000 7f9af3fff700 1 -- 172.21.15.99:0/2873387710 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9b0800f850 con 0x561494feb990 2023-08-25T09:07:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9b2eb90b80 10 client.25967 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9b2eb90b80 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9b2eb90b80 20 client.25967 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x561495059e10 0x56149505c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9b2eb90b80 4 client.25967 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:07:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2873387710 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561494f36910 con 0x561495059e10 2023-08-25T09:07:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9b13fff700 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x561495059e10 0x56149505c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9b13fff700 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x561495059e10 0x56149505c230 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49e895b9620f188 server_cookie=7307ae362e5f7a4 in_seq=0 out_seq=0 2023-08-25T09:07:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.881+0000 7f9af3fff700 10 client.25967 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:42.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 328479 4.03 MB/sec execute 348 sec latency 44.904 ms 2023-08-25T09:07:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.937+0000 7f9af3fff700 1 -- 172.21.15.99:0/2873387710 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561494f36910 con 0x561495059e10 2023-08-25T09:07:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.937+0000 7f9af3fff700 10 client.25967 handle_command_reply: tid=0 2023-08-25T09:07:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.937+0000 7f9b2eb90b80 1 client.25967 shutdown 2023-08-25T09:07:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2873387710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 msgr2=0x561494ffa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494ffa500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b08000c00 tx=0x7f9b0802ac40 comp rx=0 tx=0).stop 2023-08-25T09:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2873387710 shutdown_connections 2023-08-25T09:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561494ff1860 0x561494ffb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x561495059e10 0x56149505c230 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561494feb990 0x561494ffa500 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 --2- 172.21.15.99:0/2873387710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561494ff0f40 0x561494ffaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2873387710 >> 172.21.15.99:0/2873387710 conn(0x561494f60080 msgr2=0x561494ff1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9af3fff700 0 client.25967 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2873387710 shutdown_connections 2023-08-25T09:07:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.938+0000 7f9b2eb90b80 1 -- 172.21.15.99:0/2873387710 wait complete. 2023-08-25T09:07:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.939+0000 7f9b2eb90b80 20 client.25967 trim_cache size 0 max 16384 2023-08-25T09:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9b040a00c0 msgr2=0x7f9b040a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9b040a00c0 0x7f9b040a2580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b1419f840 tx=0x7f9b0c00fc00 comp rx=0 tx=0).stop 2023-08-25T09:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/2195876796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b14101260 msgr2=0x7f9b1419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b14101260 0x7f9b1419f080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000eb30 tx=0x7f9b0000ee40 comp rx=0 tx=0).stop 2023-08-25T09:07:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/2195876796 shutdown_connections 2023-08-25T09:07:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9b040a00c0 0x7f9b040a2580 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b141068b0 0x7f9b141989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b141008a0 0x7f9b1419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 --2- 172.21.15.99:0/2195876796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b14101260 0x7f9b1419f080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/2195876796 >> 172.21.15.99:0/2195876796 conn(0x7f9b140fc370 msgr2=0x7f9b1410bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/2195876796 shutdown_connections 2023-08-25T09:07:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:42.945+0000 7f9b1b1ff700 1 -- 172.21.15.99:0/2195876796 wait complete. 2023-08-25T09:07:43.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:43 smithi157 ceph-mon[130879]: pgmap v263: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2023-08-25T09:07:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:43 smithi181 ceph-mon[131296]: pgmap v263: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2023-08-25T09:07:43.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:43 smithi099 ceph-mon[97641]: pgmap v263: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2023-08-25T09:07:43.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 330249 4.05 MB/sec execute 349 sec latency 59.873 ms 2023-08-25T09:07:44.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331534 4.06 MB/sec execute 350 sec latency 29.822 ms 2023-08-25T09:07:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:45 smithi181 ceph-mon[131296]: pgmap v264: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2023-08-25T09:07:45.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:45 smithi099 ceph-mon[97641]: pgmap v264: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2023-08-25T09:07:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:45 smithi157 ceph-mon[130879]: pgmap v264: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2023-08-25T09:07:45.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 333696 4.08 MB/sec execute 351 sec latency 15.068 ms 2023-08-25T09:07:46.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 334881 4.09 MB/sec execute 352 sec latency 43.894 ms 2023-08-25T09:07:47.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:47 smithi157 ceph-mon[130879]: pgmap v265: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 300 op/s 2023-08-25T09:07:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:47 smithi181 ceph-mon[131296]: pgmap v265: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 300 op/s 2023-08-25T09:07:47.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:47 smithi099 ceph-mon[97641]: pgmap v265: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 300 op/s 2023-08-25T09:07:47.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 337105 4.11 MB/sec execute 353 sec latency 17.209 ms 2023-08-25T09:07:48.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 338184 4.11 MB/sec execute 354 sec latency 22.005 ms 2023-08-25T09:07:49.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:49 smithi157 ceph-mon[130879]: pgmap v266: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-08-25T09:07:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:49 smithi181 ceph-mon[131296]: pgmap v266: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-08-25T09:07:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:49 smithi099 ceph-mon[97641]: pgmap v266: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-08-25T09:07:49.929 INFO:tasks.workunit.client.0.smithi099.stdout: 1 339535 4.12 MB/sec execute 355 sec latency 19.848 ms 2023-08-25T09:07:50.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 340990 4.13 MB/sec execute 356 sec latency 31.516 ms 2023-08-25T09:07:51.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:51 smithi157 ceph-mon[130879]: pgmap v267: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-08-25T09:07:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:51 smithi181 ceph-mon[131296]: pgmap v267: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-08-25T09:07:51.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:51 smithi099 ceph-mon[97641]: pgmap v267: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-08-25T09:07:51.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 342718 4.14 MB/sec execute 357 sec latency 72.192 ms 2023-08-25T09:07:52.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 344479 4.17 MB/sec execute 358 sec latency 12.010 ms 2023-08-25T09:07:53.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:53 smithi157 ceph-mon[130879]: pgmap v268: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2023-08-25T09:07:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:53 smithi181 ceph-mon[131296]: pgmap v268: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2023-08-25T09:07:53.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:53 smithi099 ceph-mon[97641]: pgmap v268: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2023-08-25T09:07:53.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 345687 4.17 MB/sec execute 359 sec latency 16.094 ms 2023-08-25T09:07:54.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 347698 4.19 MB/sec execute 360 sec latency 15.285 ms 2023-08-25T09:07:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:55 smithi181 ceph-mon[131296]: pgmap v269: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2023-08-25T09:07:55.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:55 smithi099 ceph-mon[97641]: pgmap v269: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2023-08-25T09:07:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:55 smithi157 ceph-mon[130879]: pgmap v269: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2023-08-25T09:07:55.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 348736 4.19 MB/sec execute 361 sec latency 36.038 ms 2023-08-25T09:07:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 -- 172.21.15.99:0/2935216490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac8073610 msgr2=0x7fbac810cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 --2- 172.21.15.99:0/2935216490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac8073610 0x7fbac810cc00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0098f0 tx=0x7fbabc009c00 comp rx=0 tx=0).stop 2023-08-25T09:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 -- 172.21.15.99:0/2935216490 shutdown_connections 2023-08-25T09:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 --2- 172.21.15.99:0/2935216490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbac810d140 0x7fbac810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 --2- 172.21.15.99:0/2935216490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac8072cf0 0x7fbac80730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 --2- 172.21.15.99:0/2935216490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac8073610 0x7fbac810cc00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 -- 172.21.15.99:0/2935216490 >> 172.21.15.99:0/2935216490 conn(0x7fbac80fc370 msgr2=0x7fbac80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 -- 172.21.15.99:0/2935216490 shutdown_connections 2023-08-25T09:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.100+0000 7fbacdfff700 1 -- 172.21.15.99:0/2935216490 wait complete. 2023-08-25T09:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.101+0000 7fbacdfff700 1 Processor -- start 2023-08-25T09:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.102+0000 7fbacdfff700 1 -- start start 2023-08-25T09:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.102+0000 7fbacdfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac8072cf0 0x7fbac819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.102+0000 7fbacdfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbac8073610 0x7fbac819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.102+0000 7fbacdfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac810d140 0x7fbac8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.102+0000 7fbacdfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac81991c0 con 0x7fbac8072cf0 2023-08-25T09:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.102+0000 7fbacdfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac8199330 con 0x7fbac8073610 2023-08-25T09:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.102+0000 7fbacdfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac81994f0 con 0x7fbac810d140 2023-08-25T09:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac8072cf0 0x7fbac819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac8072cf0 0x7fbac819ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54720/0 (socket says 172.21.15.99:54720) 2023-08-25T09:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 -- 172.21.15.99:0/750303458 learned_addr learned my addr 172.21.15.99:0/750303458 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac7fff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac810d140 0x7fbac8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbac8073610 0x7fbac819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 -- 172.21.15.99:0/750303458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac810d140 msgr2=0x7fbac8198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac810d140 0x7fbac8198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 -- 172.21.15.99:0/750303458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbac8073610 msgr2=0x7fbac819f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbac8073610 0x7fbac819f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 -- 172.21.15.99:0/750303458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc009520 con 0x7fbac8072cf0 2023-08-25T09:07:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.103+0000 7fbac77fe700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac8072cf0 0x7fbac819ecf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbab800e9c0 tx=0x7fbab800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.104+0000 7fbac4ff9700 1 -- 172.21.15.99:0/750303458 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8004a30 con 0x7fbac8072cf0 2023-08-25T09:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.104+0000 7fbacdfff700 1 -- 172.21.15.99:0/750303458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac81997d0 con 0x7fbac8072cf0 2023-08-25T09:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.104+0000 7fbac4ff9700 1 -- 172.21.15.99:0/750303458 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbab800cc80 con 0x7fbac8072cf0 2023-08-25T09:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.104+0000 7fbac4ff9700 1 -- 172.21.15.99:0/750303458 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80054e0 con 0x7fbac8072cf0 2023-08-25T09:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.104+0000 7fbacdfff700 1 -- 172.21.15.99:0/750303458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac81a5ce0 con 0x7fbac8072cf0 2023-08-25T09:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.107+0000 7fbac4ff9700 1 -- 172.21.15.99:0/750303458 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fbab8004b90 con 0x7fbac8072cf0 2023-08-25T09:07:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.108+0000 7fbae19e6b80 1 Processor -- start 2023-08-25T09:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.108+0000 7fbae19e6b80 1 -- start start 2023-08-25T09:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d16320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d16860 0x5609e4d10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d10dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4c78750 con 0x5609e4d10990 2023-08-25T09:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4c69720 con 0x5609e4d16860 2023-08-25T09:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4c67040 con 0x5609e4d15f40 2023-08-25T09:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d16320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d16860 0x5609e4d10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d16320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34460/0 (socket says 172.21.15.99:34460) 2023-08-25T09:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac7fff700 1 -- 172.21.15.99:0/879006447 learned_addr learned my addr 172.21.15.99:0/879006447 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac4ff9700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbab00a00c0 0x7fbab00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d10dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac7fff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbab00a00c0 0x7fbab00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac77fe700 1 -- 172.21.15.99:0/879006447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 msgr2=0x5609e4d16320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac4ff9700 1 -- 172.21.15.99:0/750303458 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbab8014070 con 0x7fbac8072cf0 2023-08-25T09:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac77fe700 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d16320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac77fe700 1 -- 172.21.15.99:0/879006447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 msgr2=0x5609e4d10dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac77fe700 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d10dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac77fe700 1 -- 172.21.15.99:0/879006447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e4ae25d0 con 0x5609e4d16860 2023-08-25T09:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.109+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d10dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.110+0000 7fbac77fe700 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d16860 0x5609e4d10450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbab80c8e60 tx=0x7fbab8009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b10f847b1f49ff5d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.110+0000 7fbaadffb700 1 -- 172.21.15.99:0/879006447 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab805f0e0 con 0x5609e4d16860 2023-08-25T09:07:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.110+0000 7fbaadffb700 1 -- 172.21.15.99:0/879006447 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbab8098960 con 0x5609e4d16860 2023-08-25T09:07:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.110+0000 7fbaadffb700 1 -- 172.21.15.99:0/879006447 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80a0c30 con 0x5609e4d16860 2023-08-25T09:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 -- 172.21.15.99:0/879006447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d16860 msgr2=0x5609e4d10450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbac7fff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbab00a00c0 0x7fbab00a2580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbac00060b0 tx=0x7fbac0004b40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d16860 0x5609e4d10450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbab80c8e60 tx=0x7fbab8009550 comp rx=0 tx=0).stop 2023-08-25T09:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 -- 172.21.15.99:0/879006447 shutdown_connections 2023-08-25T09:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d10dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d16320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/879006447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d16860 0x5609e4d10450 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 -- 172.21.15.99:0/879006447 >> 172.21.15.99:0/879006447 conn(0x5609e4c85080 msgr2=0x5609e4d0ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 -- 172.21.15.99:0/879006447 shutdown_connections 2023-08-25T09:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.111+0000 7fbae19e6b80 1 -- 172.21.15.99:0/879006447 wait complete. 2023-08-25T09:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.112+0000 7fbae19e6b80 1 -- start start 2023-08-25T09:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.112+0000 7fbae19e6b80 1 Processor -- start 2023-08-25T09:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.112+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d10990 0x5609e4d1c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.112+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d19890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.112+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d19ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.112+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4c62930 con 0x5609e4d16860 2023-08-25T09:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.112+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4c00b50 con 0x5609e4d10990 2023-08-25T09:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.112+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4c00850 con 0x5609e4d15f40 2023-08-25T09:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbae19e6b80 10 client.0 _ll_register_callbacks cb 0x5609e4b4aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d10990 0x5609e4d1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d19890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d10990 0x5609e4d1c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40432/0 (socket says 172.21.15.99:40432) 2023-08-25T09:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 -- 172.21.15.99:0/708949713 learned_addr learned my addr 172.21.15.99:0/708949713 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac7fff700 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d19ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 -- 172.21.15.99:0/708949713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 msgr2=0x5609e4d19890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d19890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 -- 172.21.15.99:0/708949713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 msgr2=0x5609e4d19ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d19ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 -- 172.21.15.99:0/708949713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac81a5ce0 con 0x5609e4d10990 2023-08-25T09:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac7fff700 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d19ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbac77fe700 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d10990 0x5609e4d1c680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009fa0 tx=0x7fbab8095160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60dc2d9aaafce9c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.113+0000 7fbaa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/708949713 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8099520 con 0x5609e4d10990 2023-08-25T09:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 1 -- 172.21.15.99:0/708949713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609e4ce0160 con 0x5609e4d10990 2023-08-25T09:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/708949713 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbab80a0c30 con 0x5609e4d10990 2023-08-25T09:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/708949713 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbab8098e20 con 0x5609e4d10990 2023-08-25T09:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 10 client.25979 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 1 -- 172.21.15.99:0/708949713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5609e4a8ed80 con 0x5609e4d10990 2023-08-25T09:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/708949713 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609e4ce0160 con 0x5609e4d10990 2023-08-25T09:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/708949713 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbab80b8160 con 0x5609e4d10990 2023-08-25T09:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 10 client.25979 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 20 client.25979 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5609e4d29dd0 0x5609e4d2c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 4 client.25979 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:07:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.114+0000 7fbae19e6b80 1 -- 172.21.15.99:0/708949713 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609e4c08b40 con 0x5609e4d29dd0 2023-08-25T09:07:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.115+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5609e4d29dd0 0x5609e4d2c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.115+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5609e4d29dd0 0x5609e4d2c1c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6785323795349cf9 server_cookie=85d714293f7f23ae in_seq=0 out_seq=0 2023-08-25T09:07:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.115+0000 7fbaa6ffd700 10 client.25979 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/708949713 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5609e4c08b40 con 0x5609e4d29dd0 2023-08-25T09:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbaa6ffd700 10 client.25979 handle_command_reply: tid=0 2023-08-25T09:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 client.25979 shutdown 2023-08-25T09:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 -- 172.21.15.99:0/708949713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d10990 msgr2=0x5609e4d1c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d10990 0x5609e4d1c680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009fa0 tx=0x7fbab8095160 comp rx=0 tx=0).stop 2023-08-25T09:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 -- 172.21.15.99:0/708949713 shutdown_connections 2023-08-25T09:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d19ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5609e4d29dd0 0x5609e4d2c1c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d10990 0x5609e4d1c680 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/708949713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d15f40 0x5609e4d19890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 -- 172.21.15.99:0/708949713 >> 172.21.15.99:0/708949713 conn(0x5609e4c85080 msgr2=0x5609e4d05820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbaa6ffd700 0 client.25979 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 -- 172.21.15.99:0/708949713 shutdown_connections 2023-08-25T09:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 1 -- 172.21.15.99:0/708949713 wait complete. 2023-08-25T09:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.116+0000 7fbae19e6b80 20 client.25979 trim_cache size 0 max 16384 2023-08-25T09:07:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.132+0000 7fbae19e6b80 1 Processor -- start 2023-08-25T09:07:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbae19e6b80 1 -- start start 2023-08-25T09:07:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d10990 0x5609e4d19850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d19130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4ae48c0 con 0x5609e4d16860 2023-08-25T09:07:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4d10850 con 0x5609e4d15f40 2023-08-25T09:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4d0b990 con 0x5609e4d10990 2023-08-25T09:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d10990 0x5609e4d19850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d10990 0x5609e4d19850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34474/0 (socket says 172.21.15.99:34474) 2023-08-25T09:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac77fe700 1 -- 172.21.15.99:0/1622183634 learned_addr learned my addr 172.21.15.99:0/1622183634 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d19130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac7fff700 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac77fe700 1 -- 172.21.15.99:0/1622183634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 msgr2=0x5609e4d19130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac77fe700 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d19130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac77fe700 1 -- 172.21.15.99:0/1622183634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 msgr2=0x5609e4d211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac77fe700 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac77fe700 1 -- 172.21.15.99:0/1622183634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e4a77c00 con 0x5609e4d10990 2023-08-25T09:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac7fff700 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.133+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d19130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.134+0000 7fbac77fe700 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d10990 0x5609e4d19850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbab800f010 tx=0x7fbab80973d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54206de0e9d5378 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.134+0000 7fbaae7fc700 1 -- 172.21.15.99:0/1622183634 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8099520 con 0x5609e4d10990 2023-08-25T09:07:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.134+0000 7fbaae7fc700 1 -- 172.21.15.99:0/1622183634 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbab80a0c30 con 0x5609e4d10990 2023-08-25T09:07:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.134+0000 7fbaae7fc700 1 -- 172.21.15.99:0/1622183634 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab809cc50 con 0x5609e4d10990 2023-08-25T09:07:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 -- 172.21.15.99:0/1622183634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d10990 msgr2=0x5609e4d19850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d10990 0x5609e4d19850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbab800f010 tx=0x7fbab80973d0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 -- 172.21.15.99:0/1622183634 shutdown_connections 2023-08-25T09:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d16860 0x5609e4d211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d10990 0x5609e4d19850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/1622183634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d19130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 -- 172.21.15.99:0/1622183634 >> 172.21.15.99:0/1622183634 conn(0x5609e4c85080 msgr2=0x5609e4d30ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 -- 172.21.15.99:0/1622183634 shutdown_connections 2023-08-25T09:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 -- 172.21.15.99:0/1622183634 wait complete. 2023-08-25T09:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 -- start start 2023-08-25T09:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.135+0000 7fbae19e6b80 1 Processor -- start 2023-08-25T09:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d12a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d12f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbae19e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d16860 0x5609e4d11950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4d13670 con 0x5609e4d10990 2023-08-25T09:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4d11e90 con 0x5609e4d15f40 2023-08-25T09:07:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbae19e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4d12050 con 0x5609e4d16860 2023-08-25T09:07:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbae19e6b80 10 client.0 _ll_register_callbacks cb 0x7fbac0009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:07:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbac7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d16860 0x5609e4d11950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d12a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbac7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d16860 0x5609e4d11950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34484/0 (socket says 172.21.15.99:34484) 2023-08-25T09:07:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbac77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d12a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54752/0 (socket says 172.21.15.99:54752) 2023-08-25T09:07:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbac7fff700 1 -- 172.21.15.99:0/250012867 learned_addr learned my addr 172.21.15.99:0/250012867 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:07:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d12f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbac6ffd700 1 -- 172.21.15.99:0/250012867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d16860 msgr2=0x5609e4d11950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.136+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d16860 0x5609e4d11950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbac6ffd700 1 -- 172.21.15.99:0/250012867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 msgr2=0x5609e4d12a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d12a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbac6ffd700 1 -- 172.21.15.99:0/250012867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac8110280 con 0x5609e4d15f40 2023-08-25T09:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbac6ffd700 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d12f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009230 tx=0x7fbabc00f8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b74f60089b32486 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbaa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/250012867 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc01c040 con 0x5609e4d15f40 2023-08-25T09:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbae19e6b80 1 -- 172.21.15.99:0/250012867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609e4d03600 con 0x5609e4d15f40 2023-08-25T09:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/250012867 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbabc0260c0 con 0x5609e4d15f40 2023-08-25T09:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/250012867 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbabc026e80 con 0x5609e4d15f40 2023-08-25T09:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbae19e6b80 10 client.25985 fetch_fsmap learned FSMap version 32 2023-08-25T09:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.137+0000 7fbae19e6b80 1 -- 172.21.15.99:0/250012867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5609e4a8ed80 con 0x5609e4d15f40 2023-08-25T09:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/250012867 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609e4d03600 con 0x5609e4d15f40 2023-08-25T09:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/250012867 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbabc015020 con 0x5609e4d15f40 2023-08-25T09:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbae19e6b80 10 client.25985 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbae19e6b80 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbae19e6b80 20 client.25985 populate_metadata read hostname 'smithi099' 2023-08-25T09:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5609e4d7e9b0 0x5609e4d80dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbae19e6b80 4 client.25985 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbae19e6b80 1 -- 172.21.15.99:0/250012867 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609e4c5b910 con 0x5609e4d7e9b0 2023-08-25T09:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbac77fe700 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5609e4d7e9b0 0x5609e4d80dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:07:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbaa6ffd700 10 client.25985 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.138+0000 7fbac77fe700 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5609e4d7e9b0 0x5609e4d80dd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52fad52e5799c6a8 server_cookie=f5cedd0de8edde52 in_seq=0 out_seq=0 2023-08-25T09:07:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbaa6ffd700 1 -- 172.21.15.99:0/250012867 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609e4c5b910 con 0x5609e4d7e9b0 2023-08-25T09:07:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbaa6ffd700 10 client.25985 handle_command_reply: tid=0 2023-08-25T09:07:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 client.25985 shutdown 2023-08-25T09:07:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 -- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 msgr2=0x5609e4d12f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d12f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009230 tx=0x7fbabc00f8e0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 -- 172.21.15.99:0/250012867 shutdown_connections 2023-08-25T09:07:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609e4d16860 0x5609e4d11950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609e4d10990 0x5609e4d12a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5609e4d7e9b0 0x5609e4d80dd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 --2- 172.21.15.99:0/250012867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609e4d15f40 0x5609e4d12f80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbae19e6b80 1 -- 172.21.15.99:0/250012867 >> 172.21.15.99:0/250012867 conn(0x5609e4c85080 msgr2=0x5609e4d0abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.201+0000 7fbaa6ffd700 0 client.25985 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:07:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.202+0000 7fbae19e6b80 1 -- 172.21.15.99:0/250012867 shutdown_connections 2023-08-25T09:07:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.202+0000 7fbae19e6b80 1 -- 172.21.15.99:0/250012867 wait complete. 2023-08-25T09:07:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.202+0000 7fbae19e6b80 20 client.25985 trim_cache size 0 max 16384 2023-08-25T09:07:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:07:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:07:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:07:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:07:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 -- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbab00a00c0 msgr2=0x7fbab00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbab00a00c0 0x7fbab00a2580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbac00060b0 tx=0x7fbac0004b40 comp rx=0 tx=0).stop 2023-08-25T09:07:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 -- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac8072cf0 msgr2=0x7fbac819ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:07:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac8072cf0 0x7fbac819ecf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbab800e9c0 tx=0x7fbab800ecd0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 -- 172.21.15.99:0/750303458 shutdown_connections 2023-08-25T09:07:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbab00a00c0 0x7fbab00a2580 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac810d140 0x7fbac8198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac8072cf0 0x7fbac819ecf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 --2- 172.21.15.99:0/750303458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbac8073610 0x7fbac819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:07:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 -- 172.21.15.99:0/750303458 >> 172.21.15.99:0/750303458 conn(0x7fbac80fc370 msgr2=0x7fbac810f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:07:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 -- 172.21.15.99:0/750303458 shutdown_connections 2023-08-25T09:07:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:07:56.203+0000 7fbacdfff700 1 -- 172.21.15.99:0/750303458 wait complete. 2023-08-25T09:07:56.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 350006 4.19 MB/sec execute 362 sec latency 305.711 ms 2023-08-25T09:07:57.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:57 smithi157 ceph-mon[130879]: pgmap v270: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2023-08-25T09:07:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:57 smithi181 ceph-mon[131296]: pgmap v270: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2023-08-25T09:07:57.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:57 smithi099 ceph-mon[97641]: pgmap v270: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2023-08-25T09:07:57.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 351565 4.21 MB/sec execute 363 sec latency 11.319 ms 2023-08-25T09:07:58.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 352717 4.22 MB/sec execute 364 sec latency 88.785 ms 2023-08-25T09:07:59.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:07:59 smithi157 ceph-mon[130879]: pgmap v271: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-08-25T09:07:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:07:59 smithi181 ceph-mon[131296]: pgmap v271: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-08-25T09:07:59.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:07:59 smithi099 ceph-mon[97641]: pgmap v271: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-08-25T09:07:59.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 354529 4.23 MB/sec execute 365 sec latency 16.225 ms 2023-08-25T09:08:00.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 355527 4.23 MB/sec execute 366 sec latency 15.748 ms 2023-08-25T09:08:01.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:01 smithi157 ceph-mon[130879]: pgmap v272: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-08-25T09:08:01.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:01 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:01 smithi181 ceph-mon[131296]: pgmap v272: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-08-25T09:08:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:01 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:01 smithi099 ceph-mon[97641]: pgmap v272: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-08-25T09:08:01.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:01 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:01.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 355574 4.22 MB/sec execute 367 sec latency 966.846 ms 2023-08-25T09:08:02.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 355702 4.21 MB/sec execute 368 sec latency 1869.016 ms 2023-08-25T09:08:03.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:03 smithi157 ceph-mon[130879]: pgmap v273: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-08-25T09:08:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:03 smithi181 ceph-mon[131296]: pgmap v273: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-08-25T09:08:03.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:03 smithi099 ceph-mon[97641]: pgmap v273: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-08-25T09:08:03.932 INFO:tasks.workunit.client.0.smithi099.stdout: 1 357715 4.23 MB/sec execute 369 sec latency 26.341 ms 2023-08-25T09:08:04.930 INFO:tasks.workunit.client.0.smithi099.stdout: 1 358926 4.24 MB/sec execute 370 sec latency 37.750 ms 2023-08-25T09:08:05.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:05 smithi181 ceph-mon[131296]: pgmap v274: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2023-08-25T09:08:05.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:05 smithi099 ceph-mon[97641]: pgmap v274: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2023-08-25T09:08:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:05 smithi157 ceph-mon[130879]: pgmap v274: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2023-08-25T09:08:05.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 360165 4.24 MB/sec execute 371 sec latency 18.404 ms 2023-08-25T09:08:06.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 362036 4.26 MB/sec execute 372 sec latency 32.146 ms 2023-08-25T09:08:07.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:07 smithi157 ceph-mon[130879]: pgmap v275: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2023-08-25T09:08:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:07 smithi181 ceph-mon[131296]: pgmap v275: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2023-08-25T09:08:07.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:07 smithi099 ceph-mon[97641]: pgmap v275: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2023-08-25T09:08:07.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 363201 4.27 MB/sec execute 373 sec latency 25.521 ms 2023-08-25T09:08:08.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 364958 4.28 MB/sec execute 374 sec latency 29.800 ms 2023-08-25T09:08:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 -- 172.21.15.99:0/3263573655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f642c1068b0 msgr2=0x7f642c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 --2- 172.21.15.99:0/3263573655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f642c1068b0 0x7f642c106c90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6428009990 tx=0x7f6428009ca0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 -- 172.21.15.99:0/3263573655 shutdown_connections 2023-08-25T09:08:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 --2- 172.21.15.99:0/3263573655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c101260 0x7f642c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 --2- 172.21.15.99:0/3263573655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f642c1068b0 0x7f642c106c90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 --2- 172.21.15.99:0/3263573655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642c1008a0 0x7f642c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 -- 172.21.15.99:0/3263573655 >> 172.21.15.99:0/3263573655 conn(0x7f642c0fc370 msgr2=0x7f642c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 -- 172.21.15.99:0/3263573655 shutdown_connections 2023-08-25T09:08:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.359+0000 7f64333ff700 1 -- 172.21.15.99:0/3263573655 wait complete. 2023-08-25T09:08:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.361+0000 7f64333ff700 1 Processor -- start 2023-08-25T09:08:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.361+0000 7f64333ff700 1 -- start start 2023-08-25T09:08:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.361+0000 7f64333ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f642c1008a0 0x7f642c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.361+0000 7f64333ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642c101260 0x7f642c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.361+0000 7f64333ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c1068b0 0x7f642c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.361+0000 7f64333ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c199160 con 0x7f642c1008a0 2023-08-25T09:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.361+0000 7f64333ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c1992d0 con 0x7f642c101260 2023-08-25T09:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.361+0000 7f64333ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c199490 con 0x7f642c1068b0 2023-08-25T09:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c1068b0 0x7f642c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643119b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f642c1008a0 0x7f642c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c1068b0 0x7f642c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52098/0 (socket says 172.21.15.99:52098) 2023-08-25T09:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 -- 172.21.15.99:0/2155669991 learned_addr learned my addr 172.21.15.99:0/2155669991 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643099a700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642c101260 0x7f642c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 -- 172.21.15.99:0/2155669991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642c101260 msgr2=0x7f642c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642c101260 0x7f642c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 -- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f642c1008a0 msgr2=0x7f642c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f642c1008a0 0x7f642c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 -- 172.21.15.99:0/2155669991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280095f0 con 0x7f642c1068b0 2023-08-25T09:08:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643099a700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642c101260 0x7f642c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:08:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f643199c700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c1068b0 0x7f642c198990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f642400ef30 tx=0x7f642400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f64227fc700 1 -- 172.21.15.99:0/2155669991 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424018070 con 0x7f642c1068b0 2023-08-25T09:08:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f64333ff700 1 -- 172.21.15.99:0/2155669991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c199710 con 0x7f642c1068b0 2023-08-25T09:08:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.362+0000 7f64227fc700 1 -- 172.21.15.99:0/2155669991 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f642400f040 con 0x7f642c1068b0 2023-08-25T09:08:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.363+0000 7f64227fc700 1 -- 172.21.15.99:0/2155669991 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642400cdb0 con 0x7f642c1068b0 2023-08-25T09:08:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.363+0000 7f64333ff700 1 -- 172.21.15.99:0/2155669991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c1a5b40 con 0x7f642c1068b0 2023-08-25T09:08:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.366+0000 7f64227fc700 1 -- 172.21.15.99:0/2155669991 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f6424003a00 con 0x7f642c1068b0 2023-08-25T09:08:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.367+0000 7f6446d77b80 1 Processor -- start 2023-08-25T09:08:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.367+0000 7f6446d77b80 1 -- start start 2023-08-25T09:08:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6d860 0x55e3b4a67450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a67dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f6446d77b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b49cf750 con 0x55e3b4a67990 2023-08-25T09:08:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f6446d77b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b49c0720 con 0x55e3b4a6cf40 2023-08-25T09:08:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f6446d77b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b49be040 con 0x55e3b4a6d860 2023-08-25T09:08:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f64227fc700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64180a00c0 0x7f64180a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f643119b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6d860 0x55e3b4a67450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f643119b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6d860 0x55e3b4a67450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52110/0 (socket says 172.21.15.99:52110) 2023-08-25T09:08:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f643119b700 1 -- 172.21.15.99:0/3186187169 learned_addr learned my addr 172.21.15.99:0/3186187169 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f64227fc700 1 -- 172.21.15.99:0/2155669991 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f64240cb320 con 0x7f642c1068b0 2023-08-25T09:08:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f643199c700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64180a00c0 0x7f64180a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f643099a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.368+0000 7f643099a700 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a67dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f643119b700 1 -- 172.21.15.99:0/3186187169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6cf40 msgr2=0x55e3b4a6d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f643119b700 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f643119b700 1 -- 172.21.15.99:0/3186187169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 msgr2=0x55e3b4a67dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f643119b700 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a67dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f643119b700 1 -- 172.21.15.99:0/3186187169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b48395d0 con 0x55e3b4a6d860 2023-08-25T09:08:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f643099a700 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a67dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:08:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f643119b700 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6d860 0x55e3b4a67450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6428005ea0 tx=0x7f6428005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f70ab3b5e31a5a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f643199c700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64180a00c0 0x7f64180a2580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f64240c89d0 tx=0x7f6424004dc0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f64177fe700 1 -- 172.21.15.99:0/3186187169 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642801d070 con 0x55e3b4a6d860 2023-08-25T09:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f64177fe700 1 -- 172.21.15.99:0/3186187169 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f642801e040 con 0x55e3b4a6d860 2023-08-25T09:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.369+0000 7f64177fe700 1 -- 172.21.15.99:0/3186187169 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428004a30 con 0x55e3b4a6d860 2023-08-25T09:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 -- 172.21.15.99:0/3186187169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6d860 msgr2=0x55e3b4a67450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6d860 0x55e3b4a67450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6428005ea0 tx=0x7f6428005ed0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 -- 172.21.15.99:0/3186187169 shutdown_connections 2023-08-25T09:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a67dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3186187169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6d860 0x55e3b4a67450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 -- 172.21.15.99:0/3186187169 >> 172.21.15.99:0/3186187169 conn(0x55e3b49dc080 msgr2=0x55e3b4a61d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 -- 172.21.15.99:0/3186187169 shutdown_connections 2023-08-25T09:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.370+0000 7f6446d77b80 1 -- 172.21.15.99:0/3186187169 wait complete. 2023-08-25T09:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.371+0000 7f6446d77b80 1 -- start start 2023-08-25T09:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.371+0000 7f6446d77b80 1 Processor -- start 2023-08-25T09:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a73630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a70920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6d860 0x55e3b4a70f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f6446d77b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b49b9930 con 0x55e3b4a67990 2023-08-25T09:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f6446d77b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b4957b50 con 0x55e3b4a6d860 2023-08-25T09:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f6446d77b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b4957850 con 0x55e3b4a6cf40 2023-08-25T09:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f6446d77b80 10 client.0 _ll_register_callbacks cb 0x55e3b48a1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f643119b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a70920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f643099a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a73630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f643119b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a70920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52124/0 (socket says 172.21.15.99:52124) 2023-08-25T09:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f643119b700 1 -- 172.21.15.99:0/3952864270 learned_addr learned my addr 172.21.15.99:0/3952864270 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.372+0000 7f643099a700 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6d860 0x55e3b4a70f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f643119b700 1 -- 172.21.15.99:0/3952864270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6d860 msgr2=0x55e3b4a70f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f643119b700 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6d860 0x55e3b4a70f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f643119b700 1 -- 172.21.15.99:0/3952864270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 msgr2=0x55e3b4a73630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f643119b700 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a73630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f643119b700 1 -- 172.21.15.99:0/3952864270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c10e070 con 0x55e3b4a6cf40 2023-08-25T09:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f643099a700 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:08:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f643119b700 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a70920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f642800b2e0 tx=0x7f642800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da57c13e055d27f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f6414ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:08:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f6414ff9700 1 -- 172.21.15.99:0/3952864270 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428031070 con 0x55e3b4a6cf40 2023-08-25T09:08:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f6446d77b80 1 -- 172.21.15.99:0/3952864270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3b4a37160 con 0x55e3b4a6cf40 2023-08-25T09:08:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f6414ff9700 1 -- 172.21.15.99:0/3952864270 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f642800fb40 con 0x55e3b4a6cf40 2023-08-25T09:08:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f6414ff9700 1 -- 172.21.15.99:0/3952864270 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f642801d070 con 0x55e3b4a6cf40 2023-08-25T09:08:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f6446d77b80 10 client.25981 fetch_fsmap learned FSMap version 33 2023-08-25T09:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.373+0000 7f6446d77b80 1 -- 172.21.15.99:0/3952864270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e3b47e5d80 con 0x55e3b4a6cf40 2023-08-25T09:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f6414ff9700 1 -- 172.21.15.99:0/3952864270 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3b4a37160 con 0x55e3b4a6cf40 2023-08-25T09:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f6414ff9700 1 -- 172.21.15.99:0/3952864270 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6428034020 con 0x55e3b4a6cf40 2023-08-25T09:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f6446d77b80 10 client.25981 fetch_fsmap finished waiting for FSMap version 33 2023-08-25T09:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f6446d77b80 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f6446d77b80 20 client.25981 populate_metadata read hostname 'smithi099' 2023-08-25T09:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e3b4a80b60 0x55e3b4a82f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f6446d77b80 4 client.25981 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:08:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f6446d77b80 1 -- 172.21.15.99:0/3952864270 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3b495fb40 con 0x55e3b4a80b60 2023-08-25T09:08:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.374+0000 7f643099a700 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e3b4a80b60 0x55e3b4a82f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6414ff9700 10 client.25981 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f643099a700 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e3b4a80b60 0x55e3b4a82f80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b44418f6f491841 server_cookie=eb8edeff336cb678 in_seq=0 out_seq=0 2023-08-25T09:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6414ff9700 1 -- 172.21.15.99:0/3952864270 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e3b495fb40 con 0x55e3b4a80b60 2023-08-25T09:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6414ff9700 10 client.25981 handle_command_reply: tid=0 2023-08-25T09:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 client.25981 shutdown 2023-08-25T09:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 -- 172.21.15.99:0/3952864270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 msgr2=0x55e3b4a70920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a70920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f642800b2e0 tx=0x7f642800f690 comp rx=0 tx=0).stop 2023-08-25T09:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 -- 172.21.15.99:0/3952864270 shutdown_connections 2023-08-25T09:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e3b4a80b60 0x55e3b4a82f80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a6d860 0x55e3b4a70f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a67990 0x55e3b4a73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3952864270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a70920 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6414ff9700 0 client.25981 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.375+0000 7f6446d77b80 1 -- 172.21.15.99:0/3952864270 >> 172.21.15.99:0/3952864270 conn(0x55e3b49dc080 msgr2=0x55e3b4a5c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.376+0000 7f6446d77b80 1 -- 172.21.15.99:0/3952864270 shutdown_connections 2023-08-25T09:08:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.376+0000 7f6446d77b80 1 -- 172.21.15.99:0/3952864270 wait complete. 2023-08-25T09:08:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.376+0000 7f6446d77b80 20 client.25981 trim_cache size 0 max 16384 2023-08-25T09:08:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.391+0000 7f6446d77b80 1 Processor -- start 2023-08-25T09:08:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.391+0000 7f6446d77b80 1 -- start start 2023-08-25T09:08:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a67450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a78210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 0x55e3b4a6ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f6446d77b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b483b8c0 con 0x55e3b4a6d860 2023-08-25T09:08:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f6446d77b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b4a78750 con 0x55e3b4a67990 2023-08-25T09:08:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f6446d77b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b4a6eea0 con 0x55e3b4a6cf40 2023-08-25T09:08:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643119b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a78210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643099a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a67450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643119b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a78210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52126/0 (socket says 172.21.15.99:52126) 2023-08-25T09:08:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643119b700 1 -- 172.21.15.99:0/32053925 learned_addr learned my addr 172.21.15.99:0/32053925 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643099a700 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 0x55e3b4a6ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643119b700 1 -- 172.21.15.99:0/32053925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 msgr2=0x55e3b4a67450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643119b700 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a67450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643119b700 1 -- 172.21.15.99:0/32053925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 msgr2=0x55e3b4a6ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643119b700 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 0x55e3b4a6ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643119b700 1 -- 172.21.15.99:0/32053925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b4a63d00 con 0x55e3b4a6cf40 2023-08-25T09:08:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.392+0000 7f643099a700 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a67450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:08:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.393+0000 7f643119b700 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a78210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f642802bd80 tx=0x7f6428005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb7d2cdd6027b1bf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.393+0000 7f6417fff700 1 -- 172.21.15.99:0/32053925 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428031070 con 0x55e3b4a6cf40 2023-08-25T09:08:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.393+0000 7f6417fff700 1 -- 172.21.15.99:0/32053925 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f642801e040 con 0x55e3b4a6cf40 2023-08-25T09:08:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.393+0000 7f6417fff700 1 -- 172.21.15.99:0/32053925 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642801d040 con 0x55e3b4a6cf40 2023-08-25T09:08:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 -- 172.21.15.99:0/32053925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 msgr2=0x55e3b4a78210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a78210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f642802bd80 tx=0x7f6428005cc0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 -- 172.21.15.99:0/32053925 shutdown_connections 2023-08-25T09:08:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 0x55e3b4a6ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a67450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 --2- 172.21.15.99:0/32053925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a78210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 -- 172.21.15.99:0/32053925 >> 172.21.15.99:0/32053925 conn(0x55e3b49dc080 msgr2=0x55e3b4a7c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 -- 172.21.15.99:0/32053925 shutdown_connections 2023-08-25T09:08:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 -- 172.21.15.99:0/32053925 wait complete. 2023-08-25T09:08:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 -- start start 2023-08-25T09:08:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.394+0000 7f6446d77b80 1 Processor -- start 2023-08-25T09:08:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a69b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f6446d77b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 0x55e3b4a68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f6446d77b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b4a68f90 con 0x55e3b4a6d860 2023-08-25T09:08:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f6446d77b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b4a69100 con 0x55e3b4a67990 2023-08-25T09:08:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f6446d77b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b4a692c0 con 0x55e3b4a6cf40 2023-08-25T09:08:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f6446d77b80 10 client.0 _ll_register_callbacks cb 0x7f6428009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:08:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643119b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643099a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a69b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643119b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52138/0 (socket says 172.21.15.99:52138) 2023-08-25T09:08:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643119b700 1 -- 172.21.15.99:0/3899894271 learned_addr learned my addr 172.21.15.99:0/3899894271 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643099a700 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 0x55e3b4a68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643119b700 1 -- 172.21.15.99:0/3899894271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 msgr2=0x55e3b4a69b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643119b700 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a69b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643119b700 1 -- 172.21.15.99:0/3899894271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 msgr2=0x55e3b4a68880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643119b700 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 0x55e3b4a68880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.395+0000 7f643119b700 1 -- 172.21.15.99:0/3899894271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c019040 con 0x55e3b4a6cf40 2023-08-25T09:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.396+0000 7f643119b700 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6a060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64280057d0 tx=0x7f642800ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54873736a058368e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.396+0000 7f6414ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.396+0000 7f6414ff9700 1 -- 172.21.15.99:0/3899894271 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642801d040 con 0x55e3b4a6cf40 2023-08-25T09:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.396+0000 7f6414ff9700 1 -- 172.21.15.99:0/3899894271 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f6428000e40 con 0x55e3b4a6cf40 2023-08-25T09:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.396+0000 7f6446d77b80 1 -- 172.21.15.99:0/3899894271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3b4a5a600 con 0x55e3b4a6cf40 2023-08-25T09:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.396+0000 7f6414ff9700 1 -- 172.21.15.99:0/3899894271 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6428031040 con 0x55e3b4a6cf40 2023-08-25T09:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.396+0000 7f6446d77b80 10 client.25987 fetch_fsmap learned FSMap version 33 2023-08-25T09:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.396+0000 7f6446d77b80 1 -- 172.21.15.99:0/3899894271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e3b47e5d80 con 0x55e3b4a6cf40 2023-08-25T09:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f6414ff9700 1 -- 172.21.15.99:0/3899894271 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3b4a5a600 con 0x55e3b4a6cf40 2023-08-25T09:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f6414ff9700 1 -- 172.21.15.99:0/3899894271 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f642800b850 con 0x55e3b4a6cf40 2023-08-25T09:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f6446d77b80 10 client.25987 fetch_fsmap finished waiting for FSMap version 33 2023-08-25T09:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f6446d77b80 10 client.25987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f6446d77b80 20 client.25987 populate_metadata read hostname 'smithi099' 2023-08-25T09:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e3b4ad5530 0x55e3b4ad7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f6446d77b80 4 client.25987 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f6446d77b80 1 -- 172.21.15.99:0/3899894271 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3b49b2910 con 0x55e3b4ad5530 2023-08-25T09:08:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.397+0000 7f643099a700 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e3b4ad5530 0x55e3b4ad7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.398+0000 7f643099a700 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e3b4ad5530 0x55e3b4ad7950 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba724c9662512f2 server_cookie=816c17b0b9607ad1 in_seq=0 out_seq=0 2023-08-25T09:08:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.398+0000 7f6414ff9700 10 client.25987 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.452+0000 7f6414ff9700 1 -- 172.21.15.99:0/3899894271 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3b49b2910 con 0x55e3b4ad5530 2023-08-25T09:08:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.452+0000 7f6414ff9700 10 client.25987 handle_command_reply: tid=0 2023-08-25T09:08:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.452+0000 7f6446d77b80 1 client.25987 shutdown 2023-08-25T09:08:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 -- 172.21.15.99:0/3899894271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 msgr2=0x55e3b4a6a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6a060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64280057d0 tx=0x7f642800ff80 comp rx=0 tx=0).stop 2023-08-25T09:08:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 -- 172.21.15.99:0/3899894271 shutdown_connections 2023-08-25T09:08:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b4a6d860 0x55e3b4a68880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e3b4ad5530 0x55e3b4ad7950 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e3b4a67990 0x55e3b4a69b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 --2- 172.21.15.99:0/3899894271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e3b4a6cf40 0x55e3b4a6a060 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 -- 172.21.15.99:0/3899894271 >> 172.21.15.99:0/3899894271 conn(0x55e3b49dc080 msgr2=0x55e3b4a6ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6414ff9700 0 client.25987 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.453+0000 7f6446d77b80 1 -- 172.21.15.99:0/3899894271 shutdown_connections 2023-08-25T09:08:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.454+0000 7f6446d77b80 1 -- 172.21.15.99:0/3899894271 wait complete. 2023-08-25T09:08:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.454+0000 7f6446d77b80 20 client.25987 trim_cache size 0 max 16384 2023-08-25T09:08:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:08:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:08:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:08:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:08:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.456+0000 7f64333ff700 1 -- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64180a00c0 msgr2=0x7f64180a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.456+0000 7f64333ff700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64180a00c0 0x7f64180a2580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f64240c89d0 tx=0x7f6424004dc0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.456+0000 7f64333ff700 1 -- 172.21.15.99:0/2155669991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c1068b0 msgr2=0x7f642c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.456+0000 7f64333ff700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c1068b0 0x7f642c198990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f642400ef30 tx=0x7f642400c3a0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.457+0000 7f64333ff700 1 -- 172.21.15.99:0/2155669991 shutdown_connections 2023-08-25T09:08:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.457+0000 7f64333ff700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f64180a00c0 0x7f64180a2580 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.457+0000 7f64333ff700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c1068b0 0x7f642c198990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.457+0000 7f64333ff700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f642c1008a0 0x7f642c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.457+0000 7f64333ff700 1 --2- 172.21.15.99:0/2155669991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642c101260 0x7f642c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.457+0000 7f64333ff700 1 -- 172.21.15.99:0/2155669991 >> 172.21.15.99:0/2155669991 conn(0x7f642c0fc370 msgr2=0x7f642c10bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.457+0000 7f64333ff700 1 -- 172.21.15.99:0/2155669991 shutdown_connections 2023-08-25T09:08:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:09.457+0000 7f64333ff700 1 -- 172.21.15.99:0/2155669991 wait complete. 2023-08-25T09:08:09.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:09 smithi157 ceph-mon[130879]: pgmap v276: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-08-25T09:08:09.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:09 smithi157 ceph-mon[130879]: osdmap e93: 12 total, 12 up, 12 in 2023-08-25T09:08:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:09 smithi181 ceph-mon[131296]: pgmap v276: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-08-25T09:08:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:09 smithi181 ceph-mon[131296]: osdmap e93: 12 total, 12 up, 12 in 2023-08-25T09:08:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:09 smithi099 ceph-mon[97641]: pgmap v276: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-08-25T09:08:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:09 smithi099 ceph-mon[97641]: osdmap e93: 12 total, 12 up, 12 in 2023-08-25T09:08:09.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365933 4.28 MB/sec execute 375 sec latency 31.133 ms 2023-08-25T09:08:10.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 367214 4.29 MB/sec execute 376 sec latency 16.916 ms 2023-08-25T09:08:11.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:11 smithi157 ceph-mon[130879]: pgmap v278: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2023-08-25T09:08:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:11 smithi181 ceph-mon[131296]: pgmap v278: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2023-08-25T09:08:11.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:11 smithi099 ceph-mon[97641]: pgmap v278: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2023-08-25T09:08:11.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 369138 4.31 MB/sec execute 377 sec latency 28.991 ms 2023-08-25T09:08:12.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 370145 4.30 MB/sec execute 378 sec latency 30.194 ms 2023-08-25T09:08:13.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:13 smithi157 ceph-mon[130879]: pgmap v279: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2023-08-25T09:08:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:13 smithi181 ceph-mon[131296]: pgmap v279: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2023-08-25T09:08:13.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:13 smithi099 ceph-mon[97641]: pgmap v279: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2023-08-25T09:08:13.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 371643 4.32 MB/sec execute 379 sec latency 16.548 ms 2023-08-25T09:08:14.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 373020 4.33 MB/sec execute 380 sec latency 12.175 ms 2023-08-25T09:08:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:15 smithi181 ceph-mon[131296]: osdmap e94: 12 total, 12 up, 12 in 2023-08-25T09:08:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:15 smithi181 ceph-mon[131296]: pgmap v281: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-08-25T09:08:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:15 smithi099 ceph-mon[97641]: osdmap e94: 12 total, 12 up, 12 in 2023-08-25T09:08:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:15 smithi099 ceph-mon[97641]: pgmap v281: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-08-25T09:08:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:15 smithi157 ceph-mon[130879]: osdmap e94: 12 total, 12 up, 12 in 2023-08-25T09:08:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:15 smithi157 ceph-mon[130879]: pgmap v281: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-08-25T09:08:15.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 374276 4.33 MB/sec execute 381 sec latency 19.824 ms 2023-08-25T09:08:16.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 376134 4.35 MB/sec execute 382 sec latency 73.409 ms 2023-08-25T09:08:17.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:17 smithi157 ceph-mon[130879]: pgmap v282: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2023-08-25T09:08:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:17 smithi181 ceph-mon[131296]: pgmap v282: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2023-08-25T09:08:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:17 smithi099 ceph-mon[97641]: pgmap v282: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2023-08-25T09:08:17.931 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377162 4.35 MB/sec execute 383 sec latency 17.668 ms 2023-08-25T09:08:18.932 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.35 MB/sec execute 384 sec latency 637.551 ms 2023-08-25T09:08:19.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:19 smithi157 ceph-mon[130879]: pgmap v283: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-08-25T09:08:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:19 smithi181 ceph-mon[131296]: pgmap v283: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-08-25T09:08:19.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:19 smithi099 ceph-mon[97641]: pgmap v283: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-08-25T09:08:19.932 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.33 MB/sec execute 385 sec latency 1637.707 ms 2023-08-25T09:08:20.932 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.32 MB/sec execute 386 sec latency 2637.876 ms 2023-08-25T09:08:21.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:21 smithi157 ceph-mon[130879]: pgmap v284: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 245 op/s 2023-08-25T09:08:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:21 smithi181 ceph-mon[131296]: pgmap v284: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 245 op/s 2023-08-25T09:08:21.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:21 smithi099 ceph-mon[97641]: pgmap v284: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 245 op/s 2023-08-25T09:08:21.932 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.31 MB/sec execute 387 sec latency 3638.080 ms 2023-08-25T09:08:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.620+0000 7f5fedbff700 1 -- 172.21.15.99:0/2374611941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105ce0 msgr2=0x7f5fe810d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.620+0000 7f5fedbff700 1 --2- 172.21.15.99:0/2374611941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105ce0 0x7f5fe810d6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000b0d0 tx=0x7f5fe000b3e0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.621+0000 7f5fedbff700 1 -- 172.21.15.99:0/2374611941 shutdown_connections 2023-08-25T09:08:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.621+0000 7f5fedbff700 1 --2- 172.21.15.99:0/2374611941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105ce0 0x7f5fe810d6f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.621+0000 7f5fedbff700 1 --2- 172.21.15.99:0/2374611941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8069150 0x7f5fe8069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.621+0000 7f5fedbff700 1 --2- 172.21.15.99:0/2374611941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe8105320 0x7f5fe81057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.621+0000 7f5fedbff700 1 -- 172.21.15.99:0/2374611941 >> 172.21.15.99:0/2374611941 conn(0x7f5fe8076920 msgr2=0x7f5fe8076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.621+0000 7f5fedbff700 1 -- 172.21.15.99:0/2374611941 shutdown_connections 2023-08-25T09:08:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.621+0000 7f5fedbff700 1 -- 172.21.15.99:0/2374611941 wait complete. 2023-08-25T09:08:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fedbff700 1 Processor -- start 2023-08-25T09:08:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fedbff700 1 -- start start 2023-08-25T09:08:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fedbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe8069150 0x7f5fe80ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fedbff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105320 0x7f5fe80fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fedbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8105ce0 0x7f5fe81a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fedbff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a51f0 con 0x7f5fe8105320 2023-08-25T09:08:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fedbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a5330 con 0x7f5fe8105ce0 2023-08-25T09:08:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fedbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a54a0 con 0x7f5fe8069150 2023-08-25T09:08:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.622+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105320 0x7f5fe80fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105320 0x7f5fe80fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46412/0 (socket says 172.21.15.99:46412) 2023-08-25T09:08:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe6ffd700 1 -- 172.21.15.99:0/454031761 learned_addr learned my addr 172.21.15.99:0/454031761 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe8069150 0x7f5fe80ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8105ce0 0x7f5fe81a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe6ffd700 1 -- 172.21.15.99:0/454031761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe8069150 msgr2=0x7f5fe80ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe8069150 0x7f5fe80ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe6ffd700 1 -- 172.21.15.99:0/454031761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8105ce0 msgr2=0x7f5fe81a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8105ce0 0x7f5fe81a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe6ffd700 1 -- 172.21.15.99:0/454031761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0009d00 con 0x7f5fe8105320 2023-08-25T09:08:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8105ce0 0x7f5fe81a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:08:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105320 0x7f5fe80fff80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9a0 tx=0x7f5fdc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe4ff9700 1 -- 172.21.15.99:0/454031761 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc009790 con 0x7f5fe8105320 2023-08-25T09:08:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fedbff700 1 -- 172.21.15.99:0/454031761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe81a5780 con 0x7f5fe8105320 2023-08-25T09:08:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe4ff9700 1 -- 172.21.15.99:0/454031761 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fdc00cbd0 con 0x7f5fe8105320 2023-08-25T09:08:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.623+0000 7f5fe4ff9700 1 -- 172.21.15.99:0/454031761 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc018680 con 0x7f5fe8105320 2023-08-25T09:08:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.624+0000 7f5fedbff700 1 -- 172.21.15.99:0/454031761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81a5c90 con 0x7f5fe8105320 2023-08-25T09:08:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.627+0000 7f5fe4ff9700 1 -- 172.21.15.99:0/454031761 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f5fdc00cd40 con 0x7f5fe8105320 2023-08-25T09:08:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.628+0000 7f600159db80 1 Processor -- start 2023-08-25T09:08:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.628+0000 7f600159db80 1 -- start start 2023-08-25T09:08:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.628+0000 7f5fe4ff9700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5fd00a00c0 0x7f5fd00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.628+0000 7f600159db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ed83320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.628+0000 7f600159db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed7d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.628+0000 7f600159db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ed7ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f600159db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ece5750 con 0x558b9ed83860 2023-08-25T09:08:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f600159db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ecd6720 con 0x558b9ed7d990 2023-08-25T09:08:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f600159db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ecd4040 con 0x558b9ed82f40 2023-08-25T09:08:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe4ff9700 1 -- 172.21.15.99:0/454031761 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5fdc014070 con 0x7f5fe8105320 2023-08-25T09:08:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5fd00a00c0 0x7f5fd00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed7d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed7d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46418/0 (socket says 172.21.15.99:46418) 2023-08-25T09:08:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe6ffd700 1 -- 172.21.15.99:0/250424941 learned_addr learned my addr 172.21.15.99:0/250424941 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ed83320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ed7ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe6ffd700 1 -- 172.21.15.99:0/250424941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 msgr2=0x558b9ed83320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ed83320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe6ffd700 1 -- 172.21.15.99:0/250424941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 msgr2=0x558b9ed7ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ed7ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.629+0000 7f5fe6ffd700 1 -- 172.21.15.99:0/250424941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b9eb4f5d0 con 0x558b9ed83860 2023-08-25T09:08:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.630+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed7d450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0a0890 tx=0x7f5fdc098b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531548abe859683 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.630+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5fd00a00c0 0x7f5fd00a2580 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000b770 tx=0x7f5fe00094f0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.630+0000 7f5fcdffb700 1 -- 172.21.15.99:0/250424941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0996b0 con 0x558b9ed83860 2023-08-25T09:08:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.630+0000 7f5fcdffb700 1 -- 172.21.15.99:0/250424941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fdc098fb0 con 0x558b9ed83860 2023-08-25T09:08:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.630+0000 7f5fcdffb700 1 -- 172.21.15.99:0/250424941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc099210 con 0x558b9ed83860 2023-08-25T09:08:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.630+0000 7f600159db80 1 -- 172.21.15.99:0/250424941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 msgr2=0x558b9ed7d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.630+0000 7f600159db80 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed7d450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0a0890 tx=0x7f5fdc098b70 comp rx=0 tx=0).stop 2023-08-25T09:08:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 -- 172.21.15.99:0/250424941 shutdown_connections 2023-08-25T09:08:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ed7ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ed83320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 --2- 172.21.15.99:0/250424941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed7d450 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 -- 172.21.15.99:0/250424941 >> 172.21.15.99:0/250424941 conn(0x558b9ecf2080 msgr2=0x558b9ed77d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 -- 172.21.15.99:0/250424941 shutdown_connections 2023-08-25T09:08:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 -- 172.21.15.99:0/250424941 wait complete. 2023-08-25T09:08:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 -- start start 2023-08-25T09:08:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.631+0000 7f600159db80 1 Processor -- start 2023-08-25T09:08:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f600159db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed7d990 0x558b9ed89650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f600159db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed82f40 0x558b9ed868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f600159db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed86ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f600159db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9eccf930 con 0x558b9ed83860 2023-08-25T09:08:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f600159db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ec6db50 con 0x558b9ed82f40 2023-08-25T09:08:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f600159db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ec6d850 con 0x558b9ed7d990 2023-08-25T09:08:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f600159db80 10 client.0 _ll_register_callbacks cb 0x558b9ebb7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:08:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f5fe7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed86ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f5fe7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed86ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46434/0 (socket says 172.21.15.99:46434) 2023-08-25T09:08:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f5fe7fff700 1 -- 172.21.15.99:0/3021741422 learned_addr learned my addr 172.21.15.99:0/3021741422 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed7d990 0x558b9ed89650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.632+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed82f40 0x558b9ed868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fe7fff700 1 -- 172.21.15.99:0/3021741422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed7d990 msgr2=0x558b9ed89650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed7d990 0x558b9ed89650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fe7fff700 1 -- 172.21.15.99:0/3021741422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed82f40 msgr2=0x558b9ed868d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed82f40 0x558b9ed868d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fe7fff700 1 -- 172.21.15.99:0/3021741422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81a5c90 con 0x558b9ed83860 2023-08-25T09:08:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed86ee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000b010 tx=0x7f5fe000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f2f92383bbe526 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:08:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/3021741422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0025070 con 0x558b9ed83860 2023-08-25T09:08:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f600159db80 1 -- 172.21.15.99:0/3021741422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b9ed4d160 con 0x558b9ed83860 2023-08-25T09:08:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/3021741422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5fe00076f0 con 0x558b9ed83860 2023-08-25T09:08:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/3021741422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fe000e040 con 0x558b9ed83860 2023-08-25T09:08:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f600159db80 10 client.15957 fetch_fsmap learned FSMap version 33 2023-08-25T09:08:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.633+0000 7f600159db80 1 -- 172.21.15.99:0/3021741422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558b9eafbd80 con 0x558b9ed83860 2023-08-25T09:08:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/3021741422 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b9ed4d160 con 0x558b9ed83860 2023-08-25T09:08:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/3021741422 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 33) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5fe0019030 con 0x558b9ed83860 2023-08-25T09:08:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f600159db80 10 client.15957 fetch_fsmap finished waiting for FSMap version 33 2023-08-25T09:08:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f600159db80 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:08:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f600159db80 20 client.15957 populate_metadata read hostname 'smithi099' 2023-08-25T09:08:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f600159db80 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558b9ed970f0 0x558b9ed99510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f600159db80 4 client.15957 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:08:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f600159db80 1 -- 172.21.15.99:0/3021741422 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b9ec75b40 con 0x558b9ed970f0 2023-08-25T09:08:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.634+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558b9ed970f0 0x558b9ed99510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.635+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558b9ed970f0 0x558b9ed99510 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d86085cbf3f1e9 server_cookie=5c3d66ea156b2480 in_seq=0 out_seq=0 2023-08-25T09:08:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.635+0000 7f5fc6ffd700 10 client.15957 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/3021741422 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558b9ec75b40 con 0x558b9ed970f0 2023-08-25T09:08:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f5fc6ffd700 10 client.15957 handle_command_reply: tid=0 2023-08-25T09:08:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 client.15957 shutdown 2023-08-25T09:08:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 -- 172.21.15.99:0/3021741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 msgr2=0x558b9ed86ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed86ee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000b010 tx=0x7f5fe000b3e0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 -- 172.21.15.99:0/3021741422 shutdown_connections 2023-08-25T09:08:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed86ee0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed7d990 0x558b9ed89650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558b9ed970f0 0x558b9ed99510 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 --2- 172.21.15.99:0/3021741422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed82f40 0x558b9ed868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f5fc6ffd700 0 client.15957 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 -- 172.21.15.99:0/3021741422 >> 172.21.15.99:0/3021741422 conn(0x558b9ecf2080 msgr2=0x558b9ed72820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 -- 172.21.15.99:0/3021741422 shutdown_connections 2023-08-25T09:08:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 1 -- 172.21.15.99:0/3021741422 wait complete. 2023-08-25T09:08:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.636+0000 7f600159db80 20 client.15957 trim_cache size 0 max 16384 2023-08-25T09:08:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f600159db80 1 Processor -- start 2023-08-25T09:08:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f600159db80 1 -- start start 2023-08-25T09:08:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f600159db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ed7d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f600159db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ed809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f600159db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f600159db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9eb518c0 con 0x558b9ed83860 2023-08-25T09:08:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f600159db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ed83320 con 0x558b9ed7d990 2023-08-25T09:08:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f600159db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ed84e30 con 0x558b9ed82f40 2023-08-25T09:08:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f5fe7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f5fe7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed80f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46448/0 (socket says 172.21.15.99:46448) 2023-08-25T09:08:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.652+0000 7f5fe7fff700 1 -- 172.21.15.99:0/3159271880 learned_addr learned my addr 172.21.15.99:0/3159271880 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ed7d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ed809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fe7fff700 1 -- 172.21.15.99:0/3159271880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 msgr2=0x558b9ed809f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ed809f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fe7fff700 1 -- 172.21.15.99:0/3159271880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 msgr2=0x558b9ed7d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ed7d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fe7fff700 1 -- 172.21.15.99:0/3159271880 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b9eae4c00 con 0x558b9ed83860 2023-08-25T09:08:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed80f30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000afe0 tx=0x7f5fe00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9257faa552b302c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fce7fc700 1 -- 172.21.15.99:0/3159271880 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00076f0 con 0x558b9ed83860 2023-08-25T09:08:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fce7fc700 1 -- 172.21.15.99:0/3159271880 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fe002be30 con 0x558b9ed83860 2023-08-25T09:08:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.653+0000 7f5fce7fc700 1 -- 172.21.15.99:0/3159271880 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0033c10 con 0x558b9ed83860 2023-08-25T09:08:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 -- 172.21.15.99:0/3159271880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 msgr2=0x558b9ed80f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed80f30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000afe0 tx=0x7f5fe00095e0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 -- 172.21.15.99:0/3159271880 shutdown_connections 2023-08-25T09:08:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed80f30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ed7d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 --2- 172.21.15.99:0/3159271880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ed809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 -- 172.21.15.99:0/3159271880 >> 172.21.15.99:0/3159271880 conn(0x558b9ecf2080 msgr2=0x558b9ed929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 -- 172.21.15.99:0/3159271880 shutdown_connections 2023-08-25T09:08:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 -- 172.21.15.99:0/3159271880 wait complete. 2023-08-25T09:08:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 -- start start 2023-08-25T09:08:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.654+0000 7f600159db80 1 Processor -- start 2023-08-25T09:08:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f600159db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ede4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f600159db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ede4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f600159db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed8c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f600159db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ed8c940 con 0x558b9ed83860 2023-08-25T09:08:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f600159db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ed8cab0 con 0x558b9ed7d990 2023-08-25T09:08:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f600159db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9ed8cc70 con 0x558b9ed82f40 2023-08-25T09:08:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f600159db80 10 client.0 _ll_register_callbacks cb 0x7f5fe0008d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:08:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f5fe77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ede4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f5fe77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ede4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:39368/0 (socket says 172.21.15.99:39368) 2023-08-25T09:08:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f5fe77fe700 1 -- 172.21.15.99:0/1121614359 learned_addr learned my addr 172.21.15.99:0/1121614359 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f5fe7fff700 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed8c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.655+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ede4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fe77fe700 1 -- 172.21.15.99:0/1121614359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 msgr2=0x558b9ede4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ede4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fe77fe700 1 -- 172.21.15.99:0/1121614359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 msgr2=0x558b9ed8c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed8c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fe77fe700 1 -- 172.21.15.99:0/1121614359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b9eafbd80 con 0x558b9ed7d990 2023-08-25T09:08:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ede4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:08:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fe77fe700 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ede4300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x558b9ed99dc0 tx=0x7f5fd8004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bcbb15904dd48e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:08:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/1121614359 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8017040 con 0x558b9ed7d990 2023-08-25T09:08:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f600159db80 1 -- 172.21.15.99:0/1121614359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b9ed70600 con 0x558b9ed7d990 2023-08-25T09:08:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/1121614359 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5fd801eb80 con 0x558b9ed7d990 2023-08-25T09:08:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/1121614359 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd801ed40 con 0x558b9ed7d990 2023-08-25T09:08:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f600159db80 10 client.26021 fetch_fsmap learned FSMap version 33 2023-08-25T09:08:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.656+0000 7f600159db80 1 -- 172.21.15.99:0/1121614359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558b9eb221f0 con 0x558b9ed7d990 2023-08-25T09:08:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/1121614359 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b9ed70600 con 0x558b9ed7d990 2023-08-25T09:08:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/1121614359 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5fd80109c0 con 0x558b9ed7d990 2023-08-25T09:08:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f600159db80 10 client.26021 fetch_fsmap finished waiting for FSMap version 33 2023-08-25T09:08:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f600159db80 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:08:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f600159db80 20 client.26021 populate_metadata read hostname 'smithi099' 2023-08-25T09:08:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f600159db80 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558b9edec1b0 0x558b9edee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f600159db80 4 client.26021 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:08:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f600159db80 1 -- 172.21.15.99:0/1121614359 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b9ecc8910 con 0x558b9edec1b0 2023-08-25T09:08:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.657+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558b9edec1b0 0x558b9edee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.658+0000 7f5fc6ffd700 10 client.26021 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.658+0000 7f5fe6ffd700 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558b9edec1b0 0x558b9edee5d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806c878c28be21be server_cookie=b9f50dfca7014e12 in_seq=0 out_seq=0 2023-08-25T09:08:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f5fc6ffd700 1 -- 172.21.15.99:0/1121614359 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b9ecc8910 con 0x558b9edec1b0 2023-08-25T09:08:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f5fc6ffd700 10 client.26021 handle_command_reply: tid=0 2023-08-25T09:08:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 client.26021 shutdown 2023-08-25T09:08:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 -- 172.21.15.99:0/1121614359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 msgr2=0x558b9ede4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ede4300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x558b9ed99dc0 tx=0x7f5fd8004320 comp rx=0 tx=0).stop 2023-08-25T09:08:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 -- 172.21.15.99:0/1121614359 shutdown_connections 2023-08-25T09:08:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b9ed83860 0x558b9ed8c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558b9edec1b0 0x558b9edee5d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b9ed7d990 0x558b9ede4300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 --2- 172.21.15.99:0/1121614359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9ed82f40 0x558b9ede4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f5fc6ffd700 0 client.26021 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 -- 172.21.15.99:0/1121614359 >> 172.21.15.99:0/1121614359 conn(0x558b9ecf2080 msgr2=0x558b9ed809f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.719+0000 7f600159db80 1 -- 172.21.15.99:0/1121614359 shutdown_connections 2023-08-25T09:08:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.720+0000 7f600159db80 1 -- 172.21.15.99:0/1121614359 wait complete. 2023-08-25T09:08:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.720+0000 7f600159db80 20 client.26021 trim_cache size 0 max 16384 2023-08-25T09:08:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:08:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:08:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:08:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:08:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.720+0000 7f5fedbff700 1 -- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5fd00a00c0 msgr2=0x7f5fd00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.720+0000 7f5fedbff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5fd00a00c0 0x7f5fd00a2580 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000b770 tx=0x7f5fe00094f0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.720+0000 7f5fedbff700 1 -- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105320 msgr2=0x7f5fe80fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.720+0000 7f5fedbff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105320 0x7f5fe80fff80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9a0 tx=0x7f5fdc00ecb0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.720+0000 7f5fedbff700 1 -- 172.21.15.99:0/454031761 shutdown_connections 2023-08-25T09:08:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.720+0000 7f5fedbff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5fd00a00c0 0x7f5fd00a2580 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.721+0000 7f5fedbff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8105ce0 0x7f5fe81a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.721+0000 7f5fedbff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe8069150 0x7f5fe80ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.721+0000 7f5fedbff700 1 --2- 172.21.15.99:0/454031761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fe8105320 0x7f5fe80fff80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.721+0000 7f5fedbff700 1 -- 172.21.15.99:0/454031761 >> 172.21.15.99:0/454031761 conn(0x7f5fe8076920 msgr2=0x7f5fe80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.721+0000 7f5fedbff700 1 -- 172.21.15.99:0/454031761 shutdown_connections 2023-08-25T09:08:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:22.721+0000 7f5fedbff700 1 -- 172.21.15.99:0/454031761 wait complete. 2023-08-25T09:08:22.932 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.30 MB/sec execute 388 sec latency 4638.208 ms 2023-08-25T09:08:23.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:23 smithi157 ceph-mon[130879]: pgmap v285: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-08-25T09:08:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:23 smithi181 ceph-mon[131296]: pgmap v285: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-08-25T09:08:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:23 smithi099 ceph-mon[97641]: pgmap v285: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-08-25T09:08:23.932 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.29 MB/sec execute 389 sec latency 5638.323 ms 2023-08-25T09:08:24.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.28 MB/sec execute 390 sec latency 6638.461 ms 2023-08-25T09:08:25.494 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:25 smithi181 ceph-mon[131296]: pgmap v286: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-08-25T09:08:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:25 smithi099 ceph-mon[97641]: pgmap v286: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-08-25T09:08:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:25 smithi157 ceph-mon[130879]: pgmap v286: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-08-25T09:08:25.933 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.27 MB/sec execute 391 sec latency 7638.602 ms 2023-08-25T09:08:26.933 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.26 MB/sec execute 392 sec latency 8638.739 ms 2023-08-25T09:08:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:27 smithi157 ceph-mon[130879]: pgmap v287: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2023-08-25T09:08:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:27 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:08:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:27 smithi181 ceph-mon[131296]: pgmap v287: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2023-08-25T09:08:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:27 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:08:27.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:27 smithi099 ceph-mon[97641]: pgmap v287: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2023-08-25T09:08:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:27 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:08:27.933 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.25 MB/sec execute 393 sec latency 9638.836 ms 2023-08-25T09:08:28.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:28 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:08:28.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:28 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:08:28.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:28 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:08:28.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:28 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:08:28.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:28 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:08:28.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:28 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:08:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:28 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:08:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:28 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:08:28.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:28 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:08:28.933 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.24 MB/sec execute 394 sec latency 10638.945 ms 2023-08-25T09:08:29.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:29 smithi157 ceph-mon[130879]: pgmap v288: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-08-25T09:08:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:29 smithi181 ceph-mon[131296]: pgmap v288: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-08-25T09:08:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:29 smithi099 ceph-mon[97641]: pgmap v288: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-08-25T09:08:29.933 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.22 MB/sec execute 395 sec latency 11639.078 ms 2023-08-25T09:08:30.933 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.21 MB/sec execute 396 sec latency 12639.179 ms 2023-08-25T09:08:31.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:31 smithi157 ceph-mon[130879]: pgmap v289: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-08-25T09:08:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:31 smithi181 ceph-mon[131296]: pgmap v289: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-08-25T09:08:31.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:31 smithi099 ceph-mon[97641]: pgmap v289: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-08-25T09:08:31.933 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.20 MB/sec execute 397 sec latency 13639.302 ms 2023-08-25T09:08:32.933 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.19 MB/sec execute 398 sec latency 14639.443 ms 2023-08-25T09:08:33.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:33 smithi157 ceph-mon[130879]: pgmap v290: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-08-25T09:08:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:33 smithi181 ceph-mon[131296]: pgmap v290: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-08-25T09:08:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:33 smithi099 ceph-mon[97641]: pgmap v290: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-08-25T09:08:33.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377834 4.18 MB/sec execute 399 sec latency 15639.620 ms 2023-08-25T09:08:34.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 378887 4.19 MB/sec execute 400 sec latency 16199.717 ms 2023-08-25T09:08:35.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:35 smithi181 ceph-mon[131296]: pgmap v291: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2023-08-25T09:08:35.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:35 smithi099 ceph-mon[97641]: pgmap v291: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2023-08-25T09:08:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:35 smithi157 ceph-mon[130879]: pgmap v291: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2023-08-25T09:08:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.874+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3384473259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8073670 msgr2=0x7f41e810cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.874+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3384473259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8073670 0x7f41e810cbb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009990 tx=0x7f41dc009ca0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.875+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3384473259 shutdown_connections 2023-08-25T09:08:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.875+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3384473259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41e810d0f0 0x7f41e810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.875+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3384473259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e8072d50 0x7f41e8073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.875+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3384473259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8073670 0x7f41e810cbb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.875+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3384473259 >> 172.21.15.99:0/3384473259 conn(0x7f41e80fc380 msgr2=0x7f41e80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.875+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3384473259 shutdown_connections 2023-08-25T09:08:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.875+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3384473259 wait complete. 2023-08-25T09:08:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.876+0000 7f41ed9ff700 1 Processor -- start 2023-08-25T09:08:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.876+0000 7f41ed9ff700 1 -- start start 2023-08-25T09:08:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.876+0000 7f41ed9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8072d50 0x7f41e819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.876+0000 7f41ed9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e8073670 0x7f41e819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41ed9ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41e810d0f0 0x7f41e8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41ed9ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81990f0 con 0x7f41e810d0f0 2023-08-25T09:08:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41ed9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8199260 con 0x7f41e8073670 2023-08-25T09:08:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41ed9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8199420 con 0x7f41e8072d50 2023-08-25T09:08:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8072d50 0x7f41e819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41e810d0f0 0x7f41e8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e8073670 0x7f41e819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8072d50 0x7f41e819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:38812/0 (socket says 172.21.15.99:38812) 2023-08-25T09:08:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e6ffd700 1 -- 172.21.15.99:0/3899741189 learned_addr learned my addr 172.21.15.99:0/3899741189 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e77fe700 1 -- 172.21.15.99:0/3899741189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8072d50 msgr2=0x7f41e819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e77fe700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8072d50 0x7f41e819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e77fe700 1 -- 172.21.15.99:0/3899741189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e8073670 msgr2=0x7f41e819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e77fe700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e8073670 0x7f41e819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e77fe700 1 -- 172.21.15.99:0/3899741189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0095f0 con 0x7f41e810d0f0 2023-08-25T09:08:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e67fc700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e8073670 0x7f41e819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:08:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.877+0000 7f41e77fe700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41e810d0f0 0x7f41e8198a80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f41e000efc0 tx=0x7f41e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.878+0000 7f41ec9fd700 1 -- 172.21.15.99:0/3899741189 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0018070 con 0x7f41e810d0f0 2023-08-25T09:08:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.878+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3899741189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e81996a0 con 0x7f41e810d0f0 2023-08-25T09:08:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.878+0000 7f41ec9fd700 1 -- 172.21.15.99:0/3899741189 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e000f040 con 0x7f41e810d0f0 2023-08-25T09:08:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.878+0000 7f41ec9fd700 1 -- 172.21.15.99:0/3899741189 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e000cd40 con 0x7f41e810d0f0 2023-08-25T09:08:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.878+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3899741189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e81a5c10 con 0x7f41e810d0f0 2023-08-25T09:08:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.881+0000 7f41ec9fd700 1 -- 172.21.15.99:0/3899741189 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f41e0003a00 con 0x7f41e810d0f0 2023-08-25T09:08:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f4201334b80 1 Processor -- start 2023-08-25T09:08:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f4201334b80 1 -- start start 2023-08-25T09:08:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f4201334b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b68320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f4201334b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b62450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f4201334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b62990 0x5561a6b62dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f4201334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6aca750 con 0x5561a6b62990 2023-08-25T09:08:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f4201334b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6abb720 con 0x5561a6b68860 2023-08-25T09:08:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f4201334b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6ab9040 con 0x5561a6b67f40 2023-08-25T09:08:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.883+0000 7f41ec9fd700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f41d00a0140 0x7f41d00a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b62450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b62450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:45690/0 (socket says 172.21.15.99:45690) 2023-08-25T09:08:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 -- 172.21.15.99:0/884660507 learned_addr learned my addr 172.21.15.99:0/884660507 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e67fc700 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b68320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e67fc700 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b62990 0x5561a6b62dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41ec9fd700 1 -- 172.21.15.99:0/3899741189 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f41e00cb200 con 0x7f41e810d0f0 2023-08-25T09:08:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e77fe700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f41d00a0140 0x7f41d00a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 -- 172.21.15.99:0/884660507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 msgr2=0x5561a6b68320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b68320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 -- 172.21.15.99:0/884660507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b62990 msgr2=0x5561a6b62dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b62990 0x5561a6b62dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 -- 172.21.15.99:0/884660507 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a69345d0 con 0x5561a6b68860 2023-08-25T09:08:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e67fc700 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b62990 0x5561a6b62dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:08:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b62450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f41d8007da0 tx=0x7f41d80106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849ddd5f1debee09 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.884+0000 7f41e77fe700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f41d00a0140 0x7f41d00a2600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41e0007fd0 tx=0x7f41e0009040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f41cdffb700 1 -- 172.21.15.99:0/884660507 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8007870 con 0x5561a6b68860 2023-08-25T09:08:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f41cdffb700 1 -- 172.21.15.99:0/884660507 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41d8015070 con 0x5561a6b68860 2023-08-25T09:08:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f41cdffb700 1 -- 172.21.15.99:0/884660507 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8019740 con 0x5561a6b68860 2023-08-25T09:08:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 -- 172.21.15.99:0/884660507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 msgr2=0x5561a6b62450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b62450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f41d8007da0 tx=0x7f41d80106b0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 -- 172.21.15.99:0/884660507 shutdown_connections 2023-08-25T09:08:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b62990 0x5561a6b62dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b68320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 --2- 172.21.15.99:0/884660507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b62450 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 -- 172.21.15.99:0/884660507 >> 172.21.15.99:0/884660507 conn(0x5561a6ad7080 msgr2=0x5561a6b5cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 -- 172.21.15.99:0/884660507 shutdown_connections 2023-08-25T09:08:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.885+0000 7f4201334b80 1 -- 172.21.15.99:0/884660507 wait complete. 2023-08-25T09:08:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.886+0000 7f4201334b80 1 -- start start 2023-08-25T09:08:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.886+0000 7f4201334b80 1 Processor -- start 2023-08-25T09:08:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f4201334b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b62990 0x5561a6b6e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f4201334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b6b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f4201334b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b6c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f4201334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6ab4930 con 0x5561a6b67f40 2023-08-25T09:08:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f4201334b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6a52b50 con 0x5561a6b68860 2023-08-25T09:08:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f4201334b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6a52850 con 0x5561a6b62990 2023-08-25T09:08:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f4201334b80 10 client.0 _ll_register_callbacks cb 0x5561a699ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:08:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b62990 0x5561a6b6e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b6b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b62990 0x5561a6b6e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:38830/0 (socket says 172.21.15.99:38830) 2023-08-25T09:08:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e67fc700 1 -- 172.21.15.99:0/3584904258 learned_addr learned my addr 172.21.15.99:0/3584904258 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e67fc700 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e6ffd700 1 -- 172.21.15.99:0/3584904258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b62990 msgr2=0x5561a6b6e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b62990 0x5561a6b6e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e6ffd700 1 -- 172.21.15.99:0/3584904258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 msgr2=0x5561a6b6c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b6c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.887+0000 7f41e6ffd700 1 -- 172.21.15.99:0/3584904258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d80079d0 con 0x5561a6b67f40 2023-08-25T09:08:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.888+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b6b910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f41d8007d70 tx=0x7f41d8010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d33787347844b7e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.888+0000 7f41c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:08:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.888+0000 7f41c6ffd700 1 -- 172.21.15.99:0/3584904258 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8011ea0 con 0x5561a6b67f40 2023-08-25T09:08:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.888+0000 7f4201334b80 1 -- 172.21.15.99:0/3584904258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561a6b32160 con 0x5561a6b67f40 2023-08-25T09:08:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.888+0000 7f41c6ffd700 1 -- 172.21.15.99:0/3584904258 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f41d8004260 con 0x5561a6b67f40 2023-08-25T09:08:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.888+0000 7f41c6ffd700 1 -- 172.21.15.99:0/3584904258 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41d8004420 con 0x5561a6b67f40 2023-08-25T09:08:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.888+0000 7f4201334b80 10 client.15975 fetch_fsmap learned FSMap version 33 2023-08-25T09:08:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.888+0000 7f4201334b80 1 -- 172.21.15.99:0/3584904258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5561a68e0d80 con 0x5561a6b67f40 2023-08-25T09:08:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f41c6ffd700 1 -- 172.21.15.99:0/3584904258 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561a6b32160 con 0x5561a6b67f40 2023-08-25T09:08:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f41c6ffd700 1 -- 172.21.15.99:0/3584904258 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 33) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f41d8005450 con 0x5561a6b67f40 2023-08-25T09:08:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f4201334b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 33 2023-08-25T09:08:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f4201334b80 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:08:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f4201334b80 20 client.15975 populate_metadata read hostname 'smithi099' 2023-08-25T09:08:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f4201334b80 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5561a6b7c040 0x5561a6b7e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f4201334b80 4 client.15975 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:08:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f4201334b80 1 -- 172.21.15.99:0/3584904258 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561a6a5ab40 con 0x5561a6b7c040 2023-08-25T09:08:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.889+0000 7f41e67fc700 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5561a6b7c040 0x5561a6b7e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.890+0000 7f41c6ffd700 10 client.15975 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.890+0000 7f41e67fc700 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5561a6b7c040 0x5561a6b7e460 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e164fb75d63605 server_cookie=f55b2cca144a1e33 in_seq=0 out_seq=0 2023-08-25T09:08:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f41c6ffd700 1 -- 172.21.15.99:0/3584904258 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5561a6a5ab40 con 0x5561a6b7c040 2023-08-25T09:08:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f41c6ffd700 10 client.15975 handle_command_reply: tid=0 2023-08-25T09:08:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 client.15975 shutdown 2023-08-25T09:08:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 -- 172.21.15.99:0/3584904258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 msgr2=0x5561a6b6b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b6b910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f41d8007d70 tx=0x7f41d8010cc0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 -- 172.21.15.99:0/3584904258 shutdown_connections 2023-08-25T09:08:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5561a6b7c040 0x5561a6b7e460 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b68860 0x5561a6b6c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b62990 0x5561a6b6e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 --2- 172.21.15.99:0/3584904258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b6b910 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 -- 172.21.15.99:0/3584904258 >> 172.21.15.99:0/3584904258 conn(0x5561a6ad7080 msgr2=0x5561a6b57820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f41c6ffd700 0 client.15975 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 -- 172.21.15.99:0/3584904258 shutdown_connections 2023-08-25T09:08:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 1 -- 172.21.15.99:0/3584904258 wait complete. 2023-08-25T09:08:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.891+0000 7f4201334b80 20 client.15975 trim_cache size 0 max 16384 2023-08-25T09:08:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.907+0000 7f4201334b80 1 Processor -- start 2023-08-25T09:08:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.907+0000 7f4201334b80 1 -- start start 2023-08-25T09:08:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f4201334b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b62450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f4201334b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b6aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f4201334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b68860 0x5561a6b73240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f4201334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a69368c0 con 0x5561a6b68860 2023-08-25T09:08:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f4201334b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6b6ed40 con 0x5561a6b62990 2023-08-25T09:08:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f4201334b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6b553e0 con 0x5561a6b67f40 2023-08-25T09:08:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b62450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b6aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b6aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:38842/0 (socket says 172.21.15.99:38842) 2023-08-25T09:08:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e6ffd700 1 -- 172.21.15.99:0/3372315313 learned_addr learned my addr 172.21.15.99:0/3372315313 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e67fc700 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b68860 0x5561a6b73240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e67fc700 1 -- 172.21.15.99:0/3372315313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 msgr2=0x5561a6b6aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e67fc700 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b6aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e67fc700 1 -- 172.21.15.99:0/3372315313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b68860 msgr2=0x5561a6b73240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e67fc700 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b68860 0x5561a6b73240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e67fc700 1 -- 172.21.15.99:0/3372315313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a6b56b40 con 0x5561a6b62990 2023-08-25T09:08:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.908+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b6aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:08:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.909+0000 7f41e67fc700 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b62450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5561a6b7ecd0 tx=0x7f41dc01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6f26b43aefa3aa1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.909+0000 7f41ce7fc700 1 -- 172.21.15.99:0/3372315313 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc02d210 con 0x5561a6b62990 2023-08-25T09:08:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.909+0000 7f41ce7fc700 1 -- 172.21.15.99:0/3372315313 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41dc01e070 con 0x5561a6b62990 2023-08-25T09:08:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.909+0000 7f41ce7fc700 1 -- 172.21.15.99:0/3372315313 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc004920 con 0x5561a6b62990 2023-08-25T09:08:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 -- 172.21.15.99:0/3372315313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 msgr2=0x5561a6b62450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b62450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5561a6b7ecd0 tx=0x7f41dc01a040 comp rx=0 tx=0).stop 2023-08-25T09:08:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 -- 172.21.15.99:0/3372315313 shutdown_connections 2023-08-25T09:08:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b68860 0x5561a6b73240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b62450 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 --2- 172.21.15.99:0/3372315313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b67f40 0x5561a6b6aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 -- 172.21.15.99:0/3372315313 >> 172.21.15.99:0/3372315313 conn(0x5561a6ad7080 msgr2=0x5561a6b65af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 -- 172.21.15.99:0/3372315313 shutdown_connections 2023-08-25T09:08:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 -- 172.21.15.99:0/3372315313 wait complete. 2023-08-25T09:08:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 -- start start 2023-08-25T09:08:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.910+0000 7f4201334b80 1 Processor -- start 2023-08-25T09:08:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f4201334b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b64c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f4201334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f4201334b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b68860 0x5561a6b638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f4201334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6b63fb0 con 0x5561a6b67f40 2023-08-25T09:08:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f4201334b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6b64120 con 0x5561a6b62990 2023-08-25T09:08:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f4201334b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6b642e0 con 0x5561a6b68860 2023-08-25T09:08:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f4201334b80 10 client.0 _ll_register_callbacks cb 0x7f41d8007bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:08:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f41e67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b64c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f41e67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b64c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:45716/0 (socket says 172.21.15.99:45716) 2023-08-25T09:08:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f41e67fc700 1 -- 172.21.15.99:0/239226853 learned_addr learned my addr 172.21.15.99:0/239226853 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f41e67fc700 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b68860 0x5561a6b638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f41e67fc700 1 -- 172.21.15.99:0/239226853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b68860 msgr2=0x5561a6b638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f41e67fc700 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b68860 0x5561a6b638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.911+0000 7f41e67fc700 1 -- 172.21.15.99:0/239226853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 msgr2=0x5561a6b651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f41e67fc700 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f41e67fc700 1 -- 172.21.15.99:0/239226853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a68e0d80 con 0x5561a6b62990 2023-08-25T09:08:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f41e67fc700 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b64c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f41dc005cd0 tx=0x7f41dc004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b172bac100bd81ff server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f41c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:08:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f41c6ffd700 1 -- 172.21.15.99:0/239226853 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc02d510 con 0x5561a6b62990 2023-08-25T09:08:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f4201334b80 1 -- 172.21.15.99:0/239226853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561a6b55600 con 0x5561a6b62990 2023-08-25T09:08:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f41c6ffd700 1 -- 172.21.15.99:0/239226853 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f41dc005250 con 0x5561a6b62990 2023-08-25T09:08:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f41c6ffd700 1 -- 172.21.15.99:0/239226853 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41dc005410 con 0x5561a6b62990 2023-08-25T09:08:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f4201334b80 10 client.26039 fetch_fsmap learned FSMap version 33 2023-08-25T09:08:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f4201334b80 1 -- 172.21.15.99:0/239226853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f41dc020040 con 0x5561a6b62990 2023-08-25T09:08:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.912+0000 7f41c6ffd700 1 -- 172.21.15.99:0/239226853 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561a6b55600 con 0x5561a6b62990 2023-08-25T09:08:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f41c6ffd700 1 -- 172.21.15.99:0/239226853 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f41dc0099c0 con 0x5561a6b62990 2023-08-25T09:08:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f4201334b80 10 client.26039 fetch_fsmap finished waiting for FSMap version 33 2023-08-25T09:08:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f4201334b80 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:08:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f4201334b80 20 client.26039 populate_metadata read hostname 'smithi099' 2023-08-25T09:08:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f4201334b80 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5561a6bd0b60 0x5561a6bd2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f4201334b80 4 client.26039 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:08:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f4201334b80 1 -- 172.21.15.99:0/239226853 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5561a6aad910 con 0x5561a6bd0b60 2023-08-25T09:08:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5561a6bd0b60 0x5561a6bd2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f41e6ffd700 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5561a6bd0b60 0x5561a6bd2f80 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7987e1c529d1187f server_cookie=ce1d5b269a24708a in_seq=0 out_seq=0 2023-08-25T09:08:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.913+0000 7f41c6ffd700 10 client.26039 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:35.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 380044 4.20 MB/sec execute 401 sec latency 15.708 ms 2023-08-25T09:08:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f41c6ffd700 1 -- 172.21.15.99:0/239226853 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5561a6aad910 con 0x5561a6bd0b60 2023-08-25T09:08:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f41c6ffd700 10 client.26039 handle_command_reply: tid=0 2023-08-25T09:08:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 client.26039 shutdown 2023-08-25T09:08:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 -- 172.21.15.99:0/239226853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 msgr2=0x5561a6b64c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b64c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f41dc005cd0 tx=0x7f41dc004500 comp rx=0 tx=0).stop 2023-08-25T09:08:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 -- 172.21.15.99:0/239226853 shutdown_connections 2023-08-25T09:08:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561a6b68860 0x5561a6b638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5561a6bd0b60 0x5561a6bd2f80 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561a6b62990 0x5561a6b64c60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f41c6ffd700 0 client.26039 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 --2- 172.21.15.99:0/239226853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561a6b67f40 0x5561a6b651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 -- 172.21.15.99:0/239226853 >> 172.21.15.99:0/239226853 conn(0x5561a6ad7080 msgr2=0x5561a6b6b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 -- 172.21.15.99:0/239226853 shutdown_connections 2023-08-25T09:08:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 1 -- 172.21.15.99:0/239226853 wait complete. 2023-08-25T09:08:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.950+0000 7f4201334b80 20 client.26039 trim_cache size 0 max 16384 2023-08-25T09:08:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:08:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:08:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:08:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:08:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f41d00a0140 msgr2=0x7f41d00a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f41d00a0140 0x7f41d00a2600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41e0007fd0 tx=0x7f41e0009040 comp rx=0 tx=0).stop 2023-08-25T09:08:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41e810d0f0 msgr2=0x7f41e8198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41e810d0f0 0x7f41e8198a80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f41e000efc0 tx=0x7f41e000c3a0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3899741189 shutdown_connections 2023-08-25T09:08:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f41d00a0140 0x7f41d00a2600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41e810d0f0 0x7f41e8198a80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e8072d50 0x7f41e819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 --2- 172.21.15.99:0/3899741189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e8073670 0x7f41e819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3899741189 >> 172.21.15.99:0/3899741189 conn(0x7f41e80fc380 msgr2=0x7f41e810f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3899741189 shutdown_connections 2023-08-25T09:08:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:35.951+0000 7f41ed9ff700 1 -- 172.21.15.99:0/3899741189 wait complete. 2023-08-25T09:08:36.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 381255 4.20 MB/sec execute 402 sec latency 20.342 ms 2023-08-25T09:08:37.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:37 smithi157 ceph-mon[130879]: pgmap v292: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.8 MiB/s wr, 131 op/s 2023-08-25T09:08:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:37 smithi181 ceph-mon[131296]: pgmap v292: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.8 MiB/s wr, 131 op/s 2023-08-25T09:08:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:37 smithi099 ceph-mon[97641]: pgmap v292: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.8 MiB/s wr, 131 op/s 2023-08-25T09:08:37.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 381255 4.19 MB/sec execute 403 sec latency 1017.784 ms 2023-08-25T09:08:38.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 381255 4.18 MB/sec execute 404 sec latency 2017.936 ms 2023-08-25T09:08:39.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:39 smithi157 ceph-mon[130879]: pgmap v293: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:08:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:39 smithi181 ceph-mon[131296]: pgmap v293: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:08:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:39 smithi099 ceph-mon[97641]: pgmap v293: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:08:39.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 382167 4.17 MB/sec execute 405 sec latency 2567.510 ms 2023-08-25T09:08:40.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 383656 4.19 MB/sec execute 406 sec latency 13.281 ms 2023-08-25T09:08:41.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:41 smithi157 ceph-mon[130879]: pgmap v294: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:08:41.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:41 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:41 smithi181 ceph-mon[131296]: pgmap v294: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:08:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:41 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:41 smithi099 ceph-mon[97641]: pgmap v294: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 80 op/s 2023-08-25T09:08:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:41 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:41.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 384796 4.19 MB/sec execute 407 sec latency 300.415 ms 2023-08-25T09:08:42.934 INFO:tasks.workunit.client.0.smithi099.stdout: 1 384796 4.18 MB/sec execute 408 sec latency 1300.526 ms 2023-08-25T09:08:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:43 smithi181 ceph-mon[131296]: pgmap v295: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2023-08-25T09:08:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:43 smithi099 ceph-mon[97641]: pgmap v295: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2023-08-25T09:08:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:43 smithi157 ceph-mon[130879]: pgmap v295: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2023-08-25T09:08:43.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 384796 4.17 MB/sec execute 409 sec latency 2300.672 ms 2023-08-25T09:08:44.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:44 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:44.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:44 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:44.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:44 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:44.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 385607 4.17 MB/sec execute 410 sec latency 2847.485 ms 2023-08-25T09:08:45.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:45 smithi181 ceph-mon[131296]: pgmap v296: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2023-08-25T09:08:45.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:45 smithi099 ceph-mon[97641]: pgmap v296: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2023-08-25T09:08:45.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:45 smithi157 ceph-mon[130879]: pgmap v296: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2023-08-25T09:08:45.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 387124 4.19 MB/sec execute 411 sec latency 41.441 ms 2023-08-25T09:08:46.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 388368 4.19 MB/sec execute 412 sec latency 201.743 ms 2023-08-25T09:08:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:47 smithi181 ceph-mon[131296]: pgmap v297: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.3 MiB/s wr, 163 op/s 2023-08-25T09:08:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:47 smithi099 ceph-mon[97641]: pgmap v297: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.3 MiB/s wr, 163 op/s 2023-08-25T09:08:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:47 smithi157 ceph-mon[130879]: pgmap v297: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.3 MiB/s wr, 163 op/s 2023-08-25T09:08:47.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 388368 4.18 MB/sec execute 413 sec latency 1201.787 ms 2023-08-25T09:08:48.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 388368 4.17 MB/sec execute 414 sec latency 2201.901 ms 2023-08-25T09:08:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 -- 172.21.15.99:0/2395797557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35001068b0 msgr2=0x7f3500106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 --2- 172.21.15.99:0/2395797557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35001068b0 0x7f3500106c90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009990 tx=0x7f34fc009ca0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 -- 172.21.15.99:0/2395797557 shutdown_connections 2023-08-25T09:08:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 --2- 172.21.15.99:0/2395797557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3500101260 0x7f350010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 --2- 172.21.15.99:0/2395797557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35001068b0 0x7f3500106c90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 --2- 172.21.15.99:0/2395797557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35001008a0 0x7f3500100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 -- 172.21.15.99:0/2395797557 >> 172.21.15.99:0/2395797557 conn(0x7f35000fc370 msgr2=0x7f35000fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 -- 172.21.15.99:0/2395797557 shutdown_connections 2023-08-25T09:08:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.106+0000 7f3507fff700 1 -- 172.21.15.99:0/2395797557 wait complete. 2023-08-25T09:08:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.107+0000 7f3507fff700 1 Processor -- start 2023-08-25T09:08:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.108+0000 7f3507fff700 1 -- start start 2023-08-25T09:08:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.108+0000 7f3507fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35001008a0 0x7f350019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.108+0000 7f3507fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3500101260 0x7f350019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.108+0000 7f3507fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35001068b0 0x7f35001989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.108+0000 7f3507fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3500199180 con 0x7f3500101260 2023-08-25T09:08:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.108+0000 7f3507fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35001992f0 con 0x7f35001068b0 2023-08-25T09:08:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.108+0000 7f3507fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35001994b0 con 0x7f35001008a0 2023-08-25T09:08:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35001008a0 0x7f350019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35001008a0 0x7f350019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:53578/0 (socket says 172.21.15.99:53578) 2023-08-25T09:08:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 -- 172.21.15.99:0/3251379680 learned_addr learned my addr 172.21.15.99:0/3251379680 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f350559a700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3500101260 0x7f350019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f350659c700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35001068b0 0x7f35001989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 -- 172.21.15.99:0/3251379680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35001068b0 msgr2=0x7f35001989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35001068b0 0x7f35001989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 -- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3500101260 msgr2=0x7f350019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3500101260 0x7f350019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 -- 172.21.15.99:0/3251379680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc0095f0 con 0x7f35001008a0 2023-08-25T09:08:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.109+0000 7f3505d9b700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35001008a0 0x7f350019ebe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f34fc005910 tx=0x7f34fc004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.110+0000 7f34f6ffd700 1 -- 172.21.15.99:0/3251379680 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc01d070 con 0x7f35001008a0 2023-08-25T09:08:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.110+0000 7f34f6ffd700 1 -- 172.21.15.99:0/3251379680 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc022470 con 0x7f35001008a0 2023-08-25T09:08:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.110+0000 7f34f6ffd700 1 -- 172.21.15.99:0/3251379680 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc017650 con 0x7f35001008a0 2023-08-25T09:08:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.110+0000 7f3507fff700 1 -- 172.21.15.99:0/3251379680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3500199730 con 0x7f35001008a0 2023-08-25T09:08:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.110+0000 7f3507fff700 1 -- 172.21.15.99:0/3251379680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35001a5b60 con 0x7f35001008a0 2023-08-25T09:08:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.113+0000 7f34f6ffd700 1 -- 172.21.15.99:0/3251379680 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f34fc0225e0 con 0x7f35001008a0 2023-08-25T09:08:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.114+0000 7f351b928b80 1 Processor -- start 2023-08-25T09:08:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.114+0000 7f351b928b80 1 -- start start 2023-08-25T09:08:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.114+0000 7f351b928b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e353320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.114+0000 7f351b928b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 0x55907e34d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f351b928b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e34ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f351b928b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e2b5750 con 0x55907e352f40 2023-08-25T09:08:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f351b928b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e2a6720 con 0x55907e353860 2023-08-25T09:08:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f351b928b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e2a4040 con 0x55907e34d990 2023-08-25T09:08:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e353320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e353320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58688/0 (socket says 172.21.15.99:58688) 2023-08-25T09:08:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350659c700 1 -- 172.21.15.99:0/4151939116 learned_addr learned my addr 172.21.15.99:0/4151939116 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350559a700 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e34ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f3505d9b700 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 0x55907e34d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f34f6ffd700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f34ec0a0140 0x7f34ec0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350559a700 1 -- 172.21.15.99:0/4151939116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 msgr2=0x55907e34d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350559a700 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 0x55907e34d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350559a700 1 -- 172.21.15.99:0/4151939116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 msgr2=0x55907e353320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350559a700 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e353320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f350559a700 1 -- 172.21.15.99:0/4151939116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55907e11f5d0 con 0x55907e34d990 2023-08-25T09:08:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.115+0000 7f34f6ffd700 1 -- 172.21.15.99:0/3251379680 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f34fc0cc830 con 0x7f35001008a0 2023-08-25T09:08:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.116+0000 7f350659c700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f34ec0a0140 0x7f34ec0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.116+0000 7f350559a700 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e34ddd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f350019a480 tx=0x7f34f0009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91fd2fd742ad7c16 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.116+0000 7f34ebfff700 1 -- 172.21.15.99:0/4151939116 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0011c80 con 0x55907e34d990 2023-08-25T09:08:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.116+0000 7f34ebfff700 1 -- 172.21.15.99:0/4151939116 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34f0004510 con 0x55907e34d990 2023-08-25T09:08:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.116+0000 7f34ebfff700 1 -- 172.21.15.99:0/4151939116 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0005280 con 0x55907e34d990 2023-08-25T09:08:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 -- 172.21.15.99:0/4151939116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 msgr2=0x55907e34ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f350659c700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f34ec0a0140 0x7f34ec0a2600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009550 tx=0x7f34f8009040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e34ddd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f350019a480 tx=0x7f34f0009f60 comp rx=0 tx=0).stop 2023-08-25T09:08:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 -- 172.21.15.99:0/4151939116 shutdown_connections 2023-08-25T09:08:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e34ddd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e353320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 --2- 172.21.15.99:0/4151939116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 0x55907e34d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 -- 172.21.15.99:0/4151939116 >> 172.21.15.99:0/4151939116 conn(0x55907e2c2080 msgr2=0x55907e347d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 -- 172.21.15.99:0/4151939116 shutdown_connections 2023-08-25T09:08:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 -- 172.21.15.99:0/4151939116 wait complete. 2023-08-25T09:08:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 -- start start 2023-08-25T09:08:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.117+0000 7f351b928b80 1 Processor -- start 2023-08-25T09:08:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f351b928b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e359690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f351b928b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e356910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f351b928b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 0x55907e356f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f351b928b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e29f930 con 0x55907e352f40 2023-08-25T09:08:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f351b928b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e23db50 con 0x55907e353860 2023-08-25T09:08:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f351b928b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e23d850 con 0x55907e34d990 2023-08-25T09:08:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f351b928b80 10 client.0 _ll_register_callbacks cb 0x55907e187a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:08:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f350559a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e356910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f350559a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e356910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58692/0 (socket says 172.21.15.99:58692) 2023-08-25T09:08:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f350559a700 1 -- 172.21.15.99:0/3162612277 learned_addr learned my addr 172.21.15.99:0/3162612277 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f3505d9b700 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e359690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.118+0000 7f350659c700 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 0x55907e356f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f350559a700 1 -- 172.21.15.99:0/3162612277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 msgr2=0x55907e359690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f350559a700 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e359690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f350559a700 1 -- 172.21.15.99:0/3162612277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 msgr2=0x55907e356f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f350559a700 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 0x55907e356f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f350559a700 1 -- 172.21.15.99:0/3162612277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f00098e0 con 0x55907e352f40 2023-08-25T09:08:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f3505d9b700 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e359690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:08:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f350559a700 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e356910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f34f0004ad0 tx=0x7f34f0011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba714f4fe05cbd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f34e97fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:08:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f34e97fa700 1 -- 172.21.15.99:0/3162612277 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0004050 con 0x55907e352f40 2023-08-25T09:08:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f351b928b80 1 -- 172.21.15.99:0/3162612277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55907e31d160 con 0x55907e352f40 2023-08-25T09:08:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f34e97fa700 1 -- 172.21.15.99:0/3162612277 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f34f00041b0 con 0x55907e352f40 2023-08-25T09:08:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f34e97fa700 1 -- 172.21.15.99:0/3162612277 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34f00043b0 con 0x55907e352f40 2023-08-25T09:08:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.119+0000 7f351b928b80 10 client.15993 fetch_fsmap learned FSMap version 35 2023-08-25T09:08:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f351b928b80 1 -- 172.21.15.99:0/3162612277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55907e0cbd80 con 0x55907e352f40 2023-08-25T09:08:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f34e97fa700 1 -- 172.21.15.99:0/3162612277 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55907e31d160 con 0x55907e352f40 2023-08-25T09:08:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f34e97fa700 1 -- 172.21.15.99:0/3162612277 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 35) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f34f0005070 con 0x55907e352f40 2023-08-25T09:08:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f351b928b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T09:08:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f351b928b80 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:08:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f351b928b80 20 client.15993 populate_metadata read hostname 'smithi099' 2023-08-25T09:08:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f351b928b80 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55907e367040 0x55907e369460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f351b928b80 4 client.15993 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:08:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f351b928b80 1 -- 172.21.15.99:0/3162612277 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55907e245b40 con 0x55907e367040 2023-08-25T09:08:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.120+0000 7f3505d9b700 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55907e367040 0x55907e369460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.121+0000 7f3505d9b700 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55907e367040 0x55907e369460 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf39b1fd6cc590d4 server_cookie=96378bbe58934632 in_seq=0 out_seq=0 2023-08-25T09:08:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.121+0000 7f34e97fa700 10 client.15993 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.121+0000 7f34e97fa700 1 -- 172.21.15.99:0/3162612277 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55907e245b40 con 0x55907e367040 2023-08-25T09:08:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.121+0000 7f34e97fa700 10 client.15993 handle_command_reply: tid=0 2023-08-25T09:08:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 client.15993 shutdown 2023-08-25T09:08:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 -- 172.21.15.99:0/3162612277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 msgr2=0x55907e356910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e356910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f34f0004ad0 tx=0x7f34f0011f70 comp rx=0 tx=0).stop 2023-08-25T09:08:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 -- 172.21.15.99:0/3162612277 shutdown_connections 2023-08-25T09:08:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55907e367040 0x55907e369460 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e353860 0x55907e356f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e359690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 --2- 172.21.15.99:0/3162612277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e352f40 0x55907e356910 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 -- 172.21.15.99:0/3162612277 >> 172.21.15.99:0/3162612277 conn(0x55907e2c2080 msgr2=0x55907e342820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f34e97fa700 0 client.15993 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 -- 172.21.15.99:0/3162612277 shutdown_connections 2023-08-25T09:08:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 1 -- 172.21.15.99:0/3162612277 wait complete. 2023-08-25T09:08:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.122+0000 7f351b928b80 20 client.15993 trim_cache size 0 max 16384 2023-08-25T09:08:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.137+0000 7f351b928b80 1 Processor -- start 2023-08-25T09:08:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.137+0000 7f351b928b80 1 -- start start 2023-08-25T09:08:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.137+0000 7f351b928b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e34d990 0x55907e34d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.137+0000 7f351b928b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e352f40 0x55907e35e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.137+0000 7f351b928b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e3591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.137+0000 7f351b928b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e1218c0 con 0x55907e353860 2023-08-25T09:08:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.137+0000 7f351b928b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e35e7b0 con 0x55907e34d990 2023-08-25T09:08:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.137+0000 7f351b928b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e354e70 con 0x55907e352f40 2023-08-25T09:08:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e3591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e3591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58694/0 (socket says 172.21.15.99:58694) 2023-08-25T09:08:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350659c700 1 -- 172.21.15.99:0/14862188 learned_addr learned my addr 172.21.15.99:0/14862188 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350559a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e352f40 0x55907e35e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f3505d9b700 1 --2- 172.21.15.99:0/14862188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e34d990 0x55907e34d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350559a700 1 -- 172.21.15.99:0/14862188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e34d990 msgr2=0x55907e34d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350559a700 1 --2- 172.21.15.99:0/14862188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e34d990 0x55907e34d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350559a700 1 -- 172.21.15.99:0/14862188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 msgr2=0x55907e3591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350559a700 1 --2- 172.21.15.99:0/14862188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e3591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350559a700 1 -- 172.21.15.99:0/14862188 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55907e0b4c00 con 0x55907e352f40 2023-08-25T09:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f350559a700 1 --2- 172.21.15.99:0/14862188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e352f40 0x55907e35e270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f34f00053d0 tx=0x7f34f00117a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30767a5aa2ef1c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.138+0000 7f34f4ff9700 1 -- 172.21.15.99:0/14862188 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0005640 con 0x55907e352f40 2023-08-25T09:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f34f4ff9700 1 -- 172.21.15.99:0/14862188 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34f0004fc0 con 0x55907e352f40 2023-08-25T09:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f34f4ff9700 1 -- 172.21.15.99:0/14862188 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0022cc0 con 0x55907e352f40 2023-08-25T09:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 -- 172.21.15.99:0/14862188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e352f40 msgr2=0x55907e35e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 --2- 172.21.15.99:0/14862188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e352f40 0x55907e35e270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f34f00053d0 tx=0x7f34f00117a0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 -- 172.21.15.99:0/14862188 shutdown_connections 2023-08-25T09:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 --2- 172.21.15.99:0/14862188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e3591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 --2- 172.21.15.99:0/14862188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e34d990 0x55907e34d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 --2- 172.21.15.99:0/14862188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e352f40 0x55907e35e270 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 -- 172.21.15.99:0/14862188 >> 172.21.15.99:0/14862188 conn(0x55907e2c2080 msgr2=0x55907e350a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 -- 172.21.15.99:0/14862188 shutdown_connections 2023-08-25T09:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.139+0000 7f351b928b80 1 -- 172.21.15.99:0/14862188 wait complete. 2023-08-25T09:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 1 -- start start 2023-08-25T09:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 1 Processor -- start 2023-08-25T09:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e350130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e352f40 0x55907e358d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e359300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e359a40 con 0x55907e353860 2023-08-25T09:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e34e950 con 0x55907e352f40 2023-08-25T09:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907e34eb10 con 0x55907e34d990 2023-08-25T09:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.140+0000 7f351b928b80 10 client.0 _ll_register_callbacks cb 0x7f34f8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e359300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e359300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58708/0 (socket says 172.21.15.99:58708) 2023-08-25T09:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 -- 172.21.15.99:0/4116052288 learned_addr learned my addr 172.21.15.99:0/4116052288 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f3505d9b700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e350130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350559a700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e352f40 0x55907e358d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 -- 172.21.15.99:0/4116052288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 msgr2=0x55907e350130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e350130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 -- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e352f40 msgr2=0x55907e358d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e352f40 0x55907e358d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 -- 172.21.15.99:0/4116052288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55907e0cbd80 con 0x55907e353860 2023-08-25T09:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350559a700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e352f40 0x55907e358d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f3505d9b700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e350130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f350659c700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e359300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f34f8024680 tx=0x7f34f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d404c2e3bb3b83 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.141+0000 7f34e97fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f34e97fa700 1 -- 172.21.15.99:0/4116052288 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f800e040 con 0x55907e353860 2023-08-25T09:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 1 -- 172.21.15.99:0/4116052288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55907e340600 con 0x55907e353860 2023-08-25T09:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f34e97fa700 1 -- 172.21.15.99:0/4116052288 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f34f8003c00 con 0x55907e353860 2023-08-25T09:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f34e97fa700 1 -- 172.21.15.99:0/4116052288 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34f8003e00 con 0x55907e353860 2023-08-25T09:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 10 client.15999 fetch_fsmap learned FSMap version 35 2023-08-25T09:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 1 -- 172.21.15.99:0/4116052288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55907e0f21f0 con 0x55907e353860 2023-08-25T09:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f34e97fa700 1 -- 172.21.15.99:0/4116052288 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34f8038080 con 0x55907e353860 2023-08-25T09:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f34e97fa700 1 -- 172.21.15.99:0/4116052288 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 35) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f34f80344b0 con 0x55907e353860 2023-08-25T09:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T09:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:08:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 20 client.15999 populate_metadata read hostname 'smithi099' 2023-08-25T09:08:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55907e3bb7f0 0x55907e3bdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:08:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 4 client.15999 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:08:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.142+0000 7f351b928b80 1 -- 172.21.15.99:0/4116052288 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55907e298910 con 0x55907e3bb7f0 2023-08-25T09:08:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.143+0000 7f3505d9b700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55907e3bb7f0 0x55907e3bdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:08:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.143+0000 7f34e97fa700 10 client.15999 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.143+0000 7f3505d9b700 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55907e3bb7f0 0x55907e3bdc10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efedc11b848eacb1 server_cookie=2b068710c739d9 in_seq=0 out_seq=0 2023-08-25T09:08:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f34e97fa700 1 -- 172.21.15.99:0/4116052288 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55907e298910 con 0x55907e3bb7f0 2023-08-25T09:08:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f34e97fa700 10 client.15999 handle_command_reply: tid=0 2023-08-25T09:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 client.15999 shutdown 2023-08-25T09:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 -- 172.21.15.99:0/4116052288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 msgr2=0x55907e359300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e359300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f34f8024680 tx=0x7f34f8004ad0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 -- 172.21.15.99:0/4116052288 shutdown_connections 2023-08-25T09:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55907e353860 0x55907e359300 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55907e34d990 0x55907e350130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55907e3bb7f0 0x55907e3bdc10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 --2- 172.21.15.99:0/4116052288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55907e352f40 0x55907e358d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f34e97fa700 0 client.15999 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:08:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 -- 172.21.15.99:0/4116052288 >> 172.21.15.99:0/4116052288 conn(0x55907e2c2080 msgr2=0x55907e3561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 -- 172.21.15.99:0/4116052288 shutdown_connections 2023-08-25T09:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.193+0000 7f351b928b80 1 -- 172.21.15.99:0/4116052288 wait complete. 2023-08-25T09:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f351b928b80 20 client.15999 trim_cache size 0 max 16384 2023-08-25T09:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 -- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f34ec0a0140 msgr2=0x7f34ec0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f34ec0a0140 0x7f34ec0a2600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009550 tx=0x7f34f8009040 comp rx=0 tx=0).stop 2023-08-25T09:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 -- 172.21.15.99:0/3251379680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35001008a0 msgr2=0x7f350019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35001008a0 0x7f350019ebe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f34fc005910 tx=0x7f34fc004710 comp rx=0 tx=0).stop 2023-08-25T09:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 -- 172.21.15.99:0/3251379680 shutdown_connections 2023-08-25T09:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f34ec0a0140 0x7f34ec0a2600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35001068b0 0x7f35001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35001008a0 0x7f350019ebe0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 --2- 172.21.15.99:0/3251379680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3500101260 0x7f350019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:08:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 -- 172.21.15.99:0/3251379680 >> 172.21.15.99:0/3251379680 conn(0x7f35000fc370 msgr2=0x7f350010bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:08:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 -- 172.21.15.99:0/3251379680 shutdown_connections 2023-08-25T09:08:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:08:49.194+0000 7f3507fff700 1 -- 172.21.15.99:0/3251379680 wait complete. 2023-08-25T09:08:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:49 smithi181 ceph-mon[131296]: pgmap v298: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-08-25T09:08:49.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:49 smithi099 ceph-mon[97641]: pgmap v298: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-08-25T09:08:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:49 smithi157 ceph-mon[130879]: pgmap v298: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-08-25T09:08:49.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 389453 4.17 MB/sec execute 415 sec latency 2750.425 ms 2023-08-25T09:08:50.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 390733 4.18 MB/sec execute 416 sec latency 45.339 ms 2023-08-25T09:08:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:51 smithi181 ceph-mon[131296]: pgmap v299: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-08-25T09:08:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:51 smithi099 ceph-mon[97641]: pgmap v299: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-08-25T09:08:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:51 smithi157 ceph-mon[130879]: pgmap v299: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-08-25T09:08:51.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 391911 4.18 MB/sec execute 417 sec latency 243.629 ms 2023-08-25T09:08:52.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 391911 4.17 MB/sec execute 418 sec latency 1243.763 ms 2023-08-25T09:08:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:53 smithi181 ceph-mon[131296]: pgmap v300: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:08:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:53 smithi099 ceph-mon[97641]: pgmap v300: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:08:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:53 smithi157 ceph-mon[130879]: pgmap v300: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-08-25T09:08:53.935 INFO:tasks.workunit.client.0.smithi099.stdout: 1 391911 4.16 MB/sec execute 419 sec latency 2243.931 ms 2023-08-25T09:08:54.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:54 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:54.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:54 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:54.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:54 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:08:54.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 392985 4.17 MB/sec execute 420 sec latency 2785.562 ms 2023-08-25T09:08:55.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:55 smithi181 ceph-mon[131296]: pgmap v301: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2023-08-25T09:08:55.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:55 smithi099 ceph-mon[97641]: pgmap v301: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2023-08-25T09:08:55.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:55 smithi157 ceph-mon[130879]: pgmap v301: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2023-08-25T09:08:55.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 394330 4.18 MB/sec execute 421 sec latency 12.201 ms 2023-08-25T09:08:56.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 395459 4.18 MB/sec execute 422 sec latency 284.778 ms 2023-08-25T09:08:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:57 smithi181 ceph-mon[131296]: pgmap v302: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2023-08-25T09:08:57.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:57 smithi099 ceph-mon[97641]: pgmap v302: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2023-08-25T09:08:57.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:57 smithi157 ceph-mon[130879]: pgmap v302: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2023-08-25T09:08:57.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 395459 4.17 MB/sec execute 423 sec latency 1284.926 ms 2023-08-25T09:08:58.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 395459 4.16 MB/sec execute 424 sec latency 2285.049 ms 2023-08-25T09:08:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:08:59 smithi181 ceph-mon[131296]: pgmap v303: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-08-25T09:08:59.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:08:59 smithi099 ceph-mon[97641]: pgmap v303: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-08-25T09:08:59.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:08:59 smithi157 ceph-mon[130879]: pgmap v303: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-08-25T09:08:59.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 396383 4.16 MB/sec execute 425 sec latency 2824.475 ms 2023-08-25T09:09:00.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 397867 4.17 MB/sec execute 426 sec latency 10.986 ms 2023-08-25T09:09:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:01 smithi181 ceph-mon[131296]: pgmap v304: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-08-25T09:09:01.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:01 smithi099 ceph-mon[97641]: pgmap v304: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-08-25T09:09:01.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:01 smithi157 ceph-mon[130879]: pgmap v304: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-08-25T09:09:01.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 398761 4.18 MB/sec execute 427 sec latency 50.721 ms 2023-08-25T09:09:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 -- 172.21.15.99:0/1318261748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a8101260 msgr2=0x7f40a810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 --2- 172.21.15.99:0/1318261748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a8101260 0x7f40a810d770 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f40a000b0a0 tx=0x7f40a000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 -- 172.21.15.99:0/1318261748 shutdown_connections 2023-08-25T09:09:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 --2- 172.21.15.99:0/1318261748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a8101260 0x7f40a810d770 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 --2- 172.21.15.99:0/1318261748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a81068b0 0x7f40a8106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 --2- 172.21.15.99:0/1318261748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a81008a0 0x7f40a8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 -- 172.21.15.99:0/1318261748 >> 172.21.15.99:0/1318261748 conn(0x7f40a80fc370 msgr2=0x7f40a80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 -- 172.21.15.99:0/1318261748 shutdown_connections 2023-08-25T09:09:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.340+0000 7f40adfff700 1 -- 172.21.15.99:0/1318261748 wait complete. 2023-08-25T09:09:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.341+0000 7f40adfff700 1 Processor -- start 2023-08-25T09:09:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.341+0000 7f40adfff700 1 -- start start 2023-08-25T09:09:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.341+0000 7f40adfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a81008a0 0x7f40a819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.341+0000 7f40adfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8101260 0x7f40a819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.341+0000 7f40adfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a81068b0 0x7f40a8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.341+0000 7f40adfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8198fa0 con 0x7f40a81008a0 2023-08-25T09:09:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.341+0000 7f40adfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8199110 con 0x7f40a8101260 2023-08-25T09:09:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.341+0000 7f40adfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a81992d0 con 0x7f40a81068b0 2023-08-25T09:09:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a81008a0 0x7f40a819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a81068b0 0x7f40a8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a81008a0 0x7f40a819ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36256/0 (socket says 172.21.15.99:36256) 2023-08-25T09:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a81068b0 0x7f40a8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:47304/0 (socket says 172.21.15.99:47304) 2023-08-25T09:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 -- 172.21.15.99:0/1171861309 learned_addr learned my addr 172.21.15.99:0/1171861309 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8101260 0x7f40a819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 -- 172.21.15.99:0/1171861309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a81068b0 msgr2=0x7f40a8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a81068b0 0x7f40a8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 -- 172.21.15.99:0/1171861309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8101260 msgr2=0x7f40a819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8101260 0x7f40a819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 -- 172.21.15.99:0/1171861309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a0009d00 con 0x7f40a81008a0 2023-08-25T09:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a7fff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a81068b0 0x7f40a8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.342+0000 7f40a77fe700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a81008a0 0x7f40a819ead0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f409800e9a0 tx=0x7f409800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.343+0000 7f40a4ff9700 1 -- 172.21.15.99:0/1171861309 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098009790 con 0x7f40a81008a0 2023-08-25T09:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.343+0000 7f40adfff700 1 -- 172.21.15.99:0/1171861309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a81995b0 con 0x7f40a81008a0 2023-08-25T09:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.343+0000 7f40a4ff9700 1 -- 172.21.15.99:0/1171861309 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4098010430 con 0x7f40a81008a0 2023-08-25T09:09:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.343+0000 7f40a4ff9700 1 -- 172.21.15.99:0/1171861309 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098018680 con 0x7f40a81008a0 2023-08-25T09:09:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.343+0000 7f40adfff700 1 -- 172.21.15.99:0/1171861309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a81a5ac0 con 0x7f40a81008a0 2023-08-25T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.346+0000 7f40a4ff9700 1 -- 172.21.15.99:0/1171861309 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f40980105a0 con 0x7f40a81008a0 2023-08-25T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.347+0000 7f40c19e3b80 1 Processor -- start 2023-08-25T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.347+0000 7f40c19e3b80 1 -- start start 2023-08-25T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.347+0000 7f40a4ff9700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f40900a00c0 0x7f40900a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.347+0000 7f40a4ff9700 1 -- 172.21.15.99:0/1171861309 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4098014070 con 0x7f40a81008a0 2023-08-25T09:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.347+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.347+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805ddd2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a7fff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f40900a00c0 0x7f40900a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd2990 0x55805ddd2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805dd3a750 con 0x55805ddd7f40 2023-08-25T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805dd2b720 con 0x55805ddd2990 2023-08-25T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805dd29040 con 0x55805ddd8860 2023-08-25T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36266/0 (socket says 172.21.15.99:36266) 2023-08-25T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805ddd2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a77fe700 1 -- 172.21.15.99:0/933505122 learned_addr learned my addr 172.21.15.99:0/933505122 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a7fff700 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd2990 0x55805ddd2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a6ffd700 1 -- 172.21.15.99:0/933505122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd2990 msgr2=0x55805ddd2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd2990 0x55805ddd2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a6ffd700 1 -- 172.21.15.99:0/933505122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 msgr2=0x55805ddd8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a6ffd700 1 -- 172.21.15.99:0/933505122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55805dba45d0 con 0x55805ddd8860 2023-08-25T09:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a77fe700 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.348+0000 7f40a7fff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f40900a00c0 0x7f40900a2580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40a000b780 tx=0x7f40a000bf00 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.349+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805ddd2450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f409c009c00 tx=0x7f409c009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a1d9ae4f2d2b41b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.349+0000 7f408dffb700 1 -- 172.21.15.99:0/933505122 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c011b60 con 0x55805ddd8860 2023-08-25T09:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.349+0000 7f408dffb700 1 -- 172.21.15.99:0/933505122 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f409c004510 con 0x55805ddd8860 2023-08-25T09:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.349+0000 7f408dffb700 1 -- 172.21.15.99:0/933505122 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c005020 con 0x55805ddd8860 2023-08-25T09:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.349+0000 7f40c19e3b80 1 -- 172.21.15.99:0/933505122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 msgr2=0x55805ddd2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.349+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805ddd2450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f409c009c00 tx=0x7f409c009f10 comp rx=0 tx=0).stop 2023-08-25T09:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 -- 172.21.15.99:0/933505122 shutdown_connections 2023-08-25T09:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd2990 0x55805ddd2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/933505122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805ddd2450 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 -- 172.21.15.99:0/933505122 >> 172.21.15.99:0/933505122 conn(0x55805dd47080 msgr2=0x55805ddccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 -- 172.21.15.99:0/933505122 shutdown_connections 2023-08-25T09:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 -- 172.21.15.99:0/933505122 wait complete. 2023-08-25T09:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 -- start start 2023-08-25T09:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.350+0000 7f40c19e3b80 1 Processor -- start 2023-08-25T09:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd2990 0x55805ddde5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd7f40 0x55805dddb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805dddbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805dd24930 con 0x55805ddd2990 2023-08-25T09:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805dcc2b50 con 0x55805ddd7f40 2023-08-25T09:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805dcc2850 con 0x55805ddd8860 2023-08-25T09:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40c19e3b80 10 client.0 _ll_register_callbacks cb 0x55805dc0ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd2990 0x55805ddde5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40a6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd7f40 0x55805dddb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd2990 0x55805ddde5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36276/0 (socket says 172.21.15.99:36276) 2023-08-25T09:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40a77fe700 1 -- 172.21.15.99:0/360487908 learned_addr learned my addr 172.21.15.99:0/360487908 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.351+0000 7f40a7fff700 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805dddbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40a77fe700 1 -- 172.21.15.99:0/360487908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 msgr2=0x55805dddbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40a77fe700 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805dddbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40a77fe700 1 -- 172.21.15.99:0/360487908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd7f40 msgr2=0x55805dddb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40a77fe700 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd7f40 0x55805dddb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40a77fe700 1 -- 172.21.15.99:0/360487908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a81a5ac0 con 0x55805ddd2990 2023-08-25T09:09:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40a77fe700 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd2990 0x55805ddde5f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f40980c8f80 tx=0x7f4098099610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5e1f73114fb04a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f4086ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:09:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f4086ffd700 1 -- 172.21.15.99:0/360487908 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098099ea0 con 0x55805ddd2990 2023-08-25T09:09:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40c19e3b80 1 -- 172.21.15.99:0/360487908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55805dda2160 con 0x55805ddd2990 2023-08-25T09:09:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f4086ffd700 1 -- 172.21.15.99:0/360487908 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f409809ccc0 con 0x55805ddd2990 2023-08-25T09:09:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f4086ffd700 1 -- 172.21.15.99:0/360487908 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40980a0ca0 con 0x55805ddd2990 2023-08-25T09:09:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40c19e3b80 10 client.16011 fetch_fsmap learned FSMap version 36 2023-08-25T09:09:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f40c19e3b80 1 -- 172.21.15.99:0/360487908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f409c0098e0 con 0x55805ddd2990 2023-08-25T09:09:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.352+0000 7f4086ffd700 1 -- 172.21.15.99:0/360487908 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55805dda2160 con 0x55805ddd2990 2023-08-25T09:09:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.353+0000 7f4086ffd700 1 -- 172.21.15.99:0/360487908 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f40980a0810 con 0x55805ddd2990 2023-08-25T09:09:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.353+0000 7f40c19e3b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 36 2023-08-25T09:09:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.353+0000 7f40c19e3b80 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.353+0000 7f40c19e3b80 20 client.16011 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.353+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55805ddec210 0x55805ddee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.353+0000 7f40c19e3b80 4 client.16011 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:09:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.353+0000 7f40c19e3b80 1 -- 172.21.15.99:0/360487908 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55805dccab40 con 0x55805ddec210 2023-08-25T09:09:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.353+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55805ddec210 0x55805ddee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.354+0000 7f4086ffd700 10 client.16011 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.354+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55805ddec210 0x55805ddee600 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3326764977a68583 server_cookie=47d8957c191afa23 in_seq=0 out_seq=0 2023-08-25T09:09:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.354+0000 7f4086ffd700 1 -- 172.21.15.99:0/360487908 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55805dccab40 con 0x55805ddec210 2023-08-25T09:09:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.354+0000 7f4086ffd700 10 client.16011 handle_command_reply: tid=0 2023-08-25T09:09:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.354+0000 7f40c19e3b80 1 client.16011 shutdown 2023-08-25T09:09:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 -- 172.21.15.99:0/360487908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd2990 msgr2=0x55805ddde5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd2990 0x55805ddde5f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f40980c8f80 tx=0x7f4098099610 comp rx=0 tx=0).stop 2023-08-25T09:09:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 -- 172.21.15.99:0/360487908 shutdown_connections 2023-08-25T09:09:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd8860 0x55805dddbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd2990 0x55805ddde5f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55805ddec210 0x55805ddee600 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/360487908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd7f40 0x55805dddb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f4086ffd700 0 client.16011 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 -- 172.21.15.99:0/360487908 >> 172.21.15.99:0/360487908 conn(0x55805dd47080 msgr2=0x55805ddc7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 -- 172.21.15.99:0/360487908 shutdown_connections 2023-08-25T09:09:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 1 -- 172.21.15.99:0/360487908 wait complete. 2023-08-25T09:09:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.355+0000 7f40c19e3b80 20 client.16011 trim_cache size 0 max 16384 2023-08-25T09:09:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40c19e3b80 1 Processor -- start 2023-08-25T09:09:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40c19e3b80 1 -- start start 2023-08-25T09:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805dde3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddde190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805dba68c0 con 0x55805ddd7f40 2023-08-25T09:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805dddce50 con 0x55805ddd8860 2023-08-25T09:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805ddcde80 con 0x55805ddd2990 2023-08-25T09:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddde190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40a77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805dde3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddde190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36286/0 (socket says 172.21.15.99:36286) 2023-08-25T09:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40a6ffd700 1 -- 172.21.15.99:0/3302129056 learned_addr learned my addr 172.21.15.99:0/3302129056 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.370+0000 7f40a7fff700 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f40a6ffd700 1 -- 172.21.15.99:0/3302129056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 msgr2=0x55805dde3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805dde3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f40a6ffd700 1 -- 172.21.15.99:0/3302129056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 msgr2=0x55805ddd5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f40a6ffd700 1 -- 172.21.15.99:0/3302129056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55805db50d80 con 0x55805ddd7f40 2023-08-25T09:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f40a77fe700 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805dde3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f40a7fff700 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddde190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f409c009c00 tx=0x7f409c004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dba024de6f346c0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f408e7fc700 1 -- 172.21.15.99:0/3302129056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c005b00 con 0x55805ddd7f40 2023-08-25T09:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f408e7fc700 1 -- 172.21.15.99:0/3302129056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f409c0049b0 con 0x55805ddd7f40 2023-08-25T09:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.371+0000 7f408e7fc700 1 -- 172.21.15.99:0/3302129056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c003a00 con 0x55805ddd7f40 2023-08-25T09:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 -- 172.21.15.99:0/3302129056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 msgr2=0x55805ddde190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddde190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f409c009c00 tx=0x7f409c004890 comp rx=0 tx=0).stop 2023-08-25T09:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 -- 172.21.15.99:0/3302129056 shutdown_connections 2023-08-25T09:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805dde3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/3302129056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddde190 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 -- 172.21.15.99:0/3302129056 >> 172.21.15.99:0/3302129056 conn(0x55805dd47080 msgr2=0x55805ddccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 -- 172.21.15.99:0/3302129056 shutdown_connections 2023-08-25T09:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 -- 172.21.15.99:0/3302129056 wait complete. 2023-08-25T09:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.372+0000 7f40c19e3b80 1 -- start start 2023-08-25T09:09:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40c19e3b80 1 Processor -- start 2023-08-25T09:09:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805ddd4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40c19e3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805ddd5670 con 0x55805ddd7f40 2023-08-25T09:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805ddd3e90 con 0x55805ddd8860 2023-08-25T09:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40c19e3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805ddd4050 con 0x55805ddd2990 2023-08-25T09:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40c19e3b80 10 client.0 _ll_register_callbacks cb 0x7f40980989d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36290/0 (socket says 172.21.15.99:36290) 2023-08-25T09:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40a6ffd700 1 -- 172.21.15.99:0/1572833565 learned_addr learned my addr 172.21.15.99:0/1572833565 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40a77fe700 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805ddd4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.373+0000 7f40a7fff700 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40a6ffd700 1 -- 172.21.15.99:0/1572833565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 msgr2=0x55805ddd4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805ddd4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40a6ffd700 1 -- 172.21.15.99:0/1572833565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 msgr2=0x55805ddd3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40a6ffd700 1 -- 172.21.15.99:0/1572833565 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c0098e0 con 0x55805ddd7f40 2023-08-25T09:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40a6ffd700 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd4f80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f409c009bd0 tx=0x7f409c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e258dd4266ec7563 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f4086ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f4086ffd700 1 -- 172.21.15.99:0/1572833565 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c011bc0 con 0x55805ddd7f40 2023-08-25T09:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40c19e3b80 1 -- 172.21.15.99:0/1572833565 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55805ddc5600 con 0x55805ddd7f40 2023-08-25T09:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f4086ffd700 1 -- 172.21.15.99:0/1572833565 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f409c004050 con 0x55805ddd7f40 2023-08-25T09:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f4086ffd700 1 -- 172.21.15.99:0/1572833565 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f409c011d20 con 0x55805ddd7f40 2023-08-25T09:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40c19e3b80 10 client.16017 fetch_fsmap learned FSMap version 36 2023-08-25T09:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.374+0000 7f40c19e3b80 1 -- 172.21.15.99:0/1572833565 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f409c01d040 con 0x55805ddd7f40 2023-08-25T09:09:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f4086ffd700 1 -- 172.21.15.99:0/1572833565 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55805ddc5600 con 0x55805ddd7f40 2023-08-25T09:09:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f4086ffd700 1 -- 172.21.15.99:0/1572833565 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f409c033460 con 0x55805ddd7f40 2023-08-25T09:09:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f40c19e3b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 36 2023-08-25T09:09:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f40c19e3b80 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f40c19e3b80 20 client.16017 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55805de409c0 0x55805de42db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f40c19e3b80 4 client.16017 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:09:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f40c19e3b80 1 -- 172.21.15.99:0/1572833565 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55805dd1d910 con 0x55805de409c0 2023-08-25T09:09:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.375+0000 7f40a77fe700 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55805de409c0 0x55805de42db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.376+0000 7f40a77fe700 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55805de409c0 0x55805de42db0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d09955aac61fa0 server_cookie=7e62e893a59cc0ab in_seq=0 out_seq=0 2023-08-25T09:09:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.376+0000 7f4086ffd700 10 client.16017 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f4086ffd700 1 -- 172.21.15.99:0/1572833565 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55805dd1d910 con 0x55805de409c0 2023-08-25T09:09:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f4086ffd700 10 client.16017 handle_command_reply: tid=0 2023-08-25T09:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 client.16017 shutdown 2023-08-25T09:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 -- 172.21.15.99:0/1572833565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 msgr2=0x55805ddd4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd4f80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f409c009bd0 tx=0x7f409c009490 comp rx=0 tx=0).stop 2023-08-25T09:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 -- 172.21.15.99:0/1572833565 shutdown_connections 2023-08-25T09:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55805de409c0 0x55805de42db0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55805ddd8860 0x55805ddd3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55805ddd2990 0x55805ddd4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 --2- 172.21.15.99:0/1572833565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55805ddd7f40 0x55805ddd4f80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f40c19e3b80 1 -- 172.21.15.99:0/1572833565 >> 172.21.15.99:0/1572833565 conn(0x55805dd47080 msgr2=0x55805ddd5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.430+0000 7f4086ffd700 0 client.16017 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.431+0000 7f40c19e3b80 1 -- 172.21.15.99:0/1572833565 shutdown_connections 2023-08-25T09:09:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.431+0000 7f40c19e3b80 1 -- 172.21.15.99:0/1572833565 wait complete. 2023-08-25T09:09:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.431+0000 7f40c19e3b80 20 client.16017 trim_cache size 0 max 16384 2023-08-25T09:09:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:09:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:09:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:09:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:09:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.431+0000 7f40adfff700 1 -- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f40900a00c0 msgr2=0x7f40900a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.431+0000 7f40adfff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f40900a00c0 0x7f40900a2580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40a000b780 tx=0x7f40a000bf00 comp rx=0 tx=0).stop 2023-08-25T09:09:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.431+0000 7f40adfff700 1 -- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a81008a0 msgr2=0x7f40a819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.431+0000 7f40adfff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a81008a0 0x7f40a819ead0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f409800e9a0 tx=0x7f409800ecb0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.432+0000 7f40adfff700 1 -- 172.21.15.99:0/1171861309 shutdown_connections 2023-08-25T09:09:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.432+0000 7f40adfff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f40900a00c0 0x7f40900a2580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.432+0000 7f40adfff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a81068b0 0x7f40a8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.432+0000 7f40adfff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40a81008a0 0x7f40a819ead0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.432+0000 7f40adfff700 1 --2- 172.21.15.99:0/1171861309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8101260 0x7f40a819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.432+0000 7f40adfff700 1 -- 172.21.15.99:0/1171861309 >> 172.21.15.99:0/1171861309 conn(0x7f40a80fc370 msgr2=0x7f40a810bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.432+0000 7f40adfff700 1 -- 172.21.15.99:0/1171861309 shutdown_connections 2023-08-25T09:09:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:02.432+0000 7f40adfff700 1 -- 172.21.15.99:0/1171861309 wait complete. 2023-08-25T09:09:02.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 399001 4.17 MB/sec execute 428 sec latency 815.472 ms 2023-08-25T09:09:03.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:03 smithi181 ceph-mon[131296]: pgmap v305: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2023-08-25T09:09:03.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:03 smithi099 ceph-mon[97641]: pgmap v305: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2023-08-25T09:09:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:03 smithi157 ceph-mon[130879]: pgmap v305: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2023-08-25T09:09:03.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 399656 4.16 MB/sec execute 429 sec latency 1351.141 ms 2023-08-25T09:09:04.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:04 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:09:04.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:04 smithi181 ceph-mon[131296]: osdmap e95: 12 total, 12 up, 12 in 2023-08-25T09:09:04.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:04 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:09:04.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:04 smithi099 ceph-mon[97641]: osdmap e95: 12 total, 12 up, 12 in 2023-08-25T09:09:04.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:04 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:09:04.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:04 smithi157 ceph-mon[130879]: osdmap e95: 12 total, 12 up, 12 in 2023-08-25T09:09:04.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 401246 4.18 MB/sec execute 430 sec latency 12.380 ms 2023-08-25T09:09:05.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:05 smithi181 ceph-mon[131296]: pgmap v307: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 834 KiB/s wr, 108 op/s 2023-08-25T09:09:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:05 smithi099 ceph-mon[97641]: pgmap v307: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 834 KiB/s wr, 108 op/s 2023-08-25T09:09:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:05 smithi157 ceph-mon[130879]: pgmap v307: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 834 KiB/s wr, 108 op/s 2023-08-25T09:09:05.936 INFO:tasks.workunit.client.0.smithi099.stdout: 1 402364 4.18 MB/sec execute 431 sec latency 21.860 ms 2023-08-25T09:09:06.937 INFO:tasks.workunit.client.0.smithi099.stdout: 1 402551 4.17 MB/sec execute 432 sec latency 768.219 ms 2023-08-25T09:09:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:07 smithi181 ceph-mon[131296]: pgmap v308: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.0 MiB/s wr, 173 op/s 2023-08-25T09:09:07.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:07 smithi099 ceph-mon[97641]: pgmap v308: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.0 MiB/s wr, 173 op/s 2023-08-25T09:09:07.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:07 smithi157 ceph-mon[130879]: pgmap v308: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.0 MiB/s wr, 173 op/s 2023-08-25T09:09:07.937 INFO:tasks.workunit.client.0.smithi099.stdout: 1 402551 4.16 MB/sec execute 433 sec latency 1768.334 ms 2023-08-25T09:09:08.937 INFO:tasks.workunit.client.0.smithi099.stdout: 1 402551 4.15 MB/sec execute 434 sec latency 2768.517 ms 2023-08-25T09:09:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:09 smithi181 ceph-mon[131296]: pgmap v309: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.0 MiB/s wr, 173 op/s 2023-08-25T09:09:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:09 smithi181 ceph-mon[131296]: osdmap e96: 12 total, 12 up, 12 in 2023-08-25T09:09:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:09 smithi099 ceph-mon[97641]: pgmap v309: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.0 MiB/s wr, 173 op/s 2023-08-25T09:09:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:09 smithi099 ceph-mon[97641]: osdmap e96: 12 total, 12 up, 12 in 2023-08-25T09:09:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:09 smithi157 ceph-mon[130879]: pgmap v309: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.0 MiB/s wr, 173 op/s 2023-08-25T09:09:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:09 smithi157 ceph-mon[130879]: osdmap e96: 12 total, 12 up, 12 in 2023-08-25T09:09:09.937 INFO:tasks.workunit.client.0.smithi099.stdout: 1 403631 4.16 MB/sec execute 435 sec latency 3317.576 ms 2023-08-25T09:09:10.937 INFO:tasks.workunit.client.0.smithi099.stdout: 1 404993 4.17 MB/sec execute 436 sec latency 12.389 ms 2023-08-25T09:09:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:11 smithi181 ceph-mon[131296]: pgmap v311: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 804 KiB/s wr, 141 op/s 2023-08-25T09:09:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:11 smithi099 ceph-mon[97641]: pgmap v311: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 804 KiB/s wr, 141 op/s 2023-08-25T09:09:11.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:11 smithi157 ceph-mon[130879]: pgmap v311: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 804 KiB/s wr, 141 op/s 2023-08-25T09:09:11.937 INFO:tasks.workunit.client.0.smithi099.stdout: 1 405469 4.16 MB/sec execute 437 sec latency 689.145 ms 2023-08-25T09:09:12.937 INFO:tasks.workunit.client.0.smithi099.stdout: 1 405469 4.15 MB/sec execute 438 sec latency 1689.321 ms 2023-08-25T09:09:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:13 smithi181 ceph-mon[131296]: pgmap v312: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 184 op/s 2023-08-25T09:09:13.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:13 smithi099 ceph-mon[97641]: pgmap v312: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 184 op/s 2023-08-25T09:09:13.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:13 smithi157 ceph-mon[130879]: pgmap v312: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 184 op/s 2023-08-25T09:09:13.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 405469 4.14 MB/sec execute 439 sec latency 2689.477 ms 2023-08-25T09:09:14.938 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.14 MB/sec execute 440 sec latency 3235.668 ms 2023-08-25T09:09:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:15 smithi181 ceph-mon[131296]: pgmap v313: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2023-08-25T09:09:15.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:15 smithi099 ceph-mon[97641]: pgmap v313: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2023-08-25T09:09:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 -- 172.21.15.99:0/4101670061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941068c0 msgr2=0x7f9894106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 --2- 172.21.15.99:0/4101670061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941068c0 0x7f9894106ca0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9884009990 tx=0x7f9884009ca0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 -- 172.21.15.99:0/4101670061 shutdown_connections 2023-08-25T09:09:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 --2- 172.21.15.99:0/4101670061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9894101270 0x7f989410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 --2- 172.21.15.99:0/4101670061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941068c0 0x7f9894106ca0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 --2- 172.21.15.99:0/4101670061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98941008b0 0x7f9894100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 -- 172.21.15.99:0/4101670061 >> 172.21.15.99:0/4101670061 conn(0x7f98940fc380 msgr2=0x7f98940fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 -- 172.21.15.99:0/4101670061 shutdown_connections 2023-08-25T09:09:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.589+0000 7f9899dff700 1 -- 172.21.15.99:0/4101670061 wait complete. 2023-08-25T09:09:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.590+0000 7f9899dff700 1 Processor -- start 2023-08-25T09:09:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.590+0000 7f9899dff700 1 -- start start 2023-08-25T09:09:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.590+0000 7f9899dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941008b0 0x7f9894072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.590+0000 7f9899dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9894101270 0x7f9894073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f9899dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98941068c0 0x7f98941a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f9899dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98941a5160 con 0x7f98941008b0 2023-08-25T09:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f9899dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98941a52a0 con 0x7f98941068c0 2023-08-25T09:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f9899dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98941a53e0 con 0x7f9894101270 2023-08-25T09:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941008b0 0x7f9894072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941008b0 0x7f9894072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42580/0 (socket says 172.21.15.99:42580) 2023-08-25T09:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 -- 172.21.15.99:0/4108212913 learned_addr learned my addr 172.21.15.99:0/4108212913 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f9893fff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98941068c0 0x7f98941a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f9892ffd700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9894101270 0x7f9894073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 -- 172.21.15.99:0/4108212913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9894101270 msgr2=0x7f9894073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9894101270 0x7f9894073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 -- 172.21.15.99:0/4108212913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98941068c0 msgr2=0x7f98941a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98941068c0 0x7f98941a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 -- 172.21.15.99:0/4108212913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840095f0 con 0x7f98941008b0 2023-08-25T09:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.591+0000 7f98937fe700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941008b0 0x7f9894072e00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9884000c00 tx=0x7f9884004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.592+0000 7f9890ff9700 1 -- 172.21.15.99:0/4108212913 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988401d070 con 0x7f98941008b0 2023-08-25T09:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.592+0000 7f9899dff700 1 -- 172.21.15.99:0/4108212913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98941a5690 con 0x7f98941008b0 2023-08-25T09:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.592+0000 7f9890ff9700 1 -- 172.21.15.99:0/4108212913 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9884021410 con 0x7f98941008b0 2023-08-25T09:09:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.592+0000 7f9890ff9700 1 -- 172.21.15.99:0/4108212913 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884017650 con 0x7f98941008b0 2023-08-25T09:09:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.592+0000 7f9899dff700 1 -- 172.21.15.99:0/4108212913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98941a5ba0 con 0x7f98941008b0 2023-08-25T09:09:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.596+0000 7f9890ff9700 1 -- 172.21.15.99:0/4108212913 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9884021580 con 0x7f98941008b0 2023-08-25T09:09:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.597+0000 7f98ad694b80 1 Processor -- start 2023-08-25T09:09:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f98ad694b80 1 -- start start 2023-08-25T09:09:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99bf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99bf860 0x55f4e99b9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99b9990 0x55f4e99b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f98ad694b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e9921750 con 0x55f4e99b9990 2023-08-25T09:09:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f98ad694b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e9912720 con 0x55f4e99bef40 2023-08-25T09:09:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f98ad694b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e9910040 con 0x55f4e99bf860 2023-08-25T09:09:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f9893fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99bf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99b9990 0x55f4e99b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f9893fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99bf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52594/0 (socket says 172.21.15.99:52594) 2023-08-25T09:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f9893fff700 1 -- 172.21.15.99:0/1533132130 learned_addr learned my addr 172.21.15.99:0/1533132130 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f9890ff9700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f987c0a00c0 0x7f987c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.598+0000 7f98937fe700 1 --2- 172.21.15.99:0/1533132130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99bf860 0x55f4e99b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9890ff9700 1 -- 172.21.15.99:0/4108212913 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f98840cc9f0 con 0x7f98941008b0 2023-08-25T09:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9892ffd700 1 -- 172.21.15.99:0/1533132130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99bf860 msgr2=0x55f4e99b9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9892ffd700 1 --2- 172.21.15.99:0/1533132130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99bf860 0x55f4e99b9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9893fff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f987c0a00c0 0x7f987c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9892ffd700 1 -- 172.21.15.99:0/1533132130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 msgr2=0x55f4e99bf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9892ffd700 1 --2- 172.21.15.99:0/1533132130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99bf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9892ffd700 1 -- 172.21.15.99:0/1533132130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4e978b5d0 con 0x55f4e99b9990 2023-08-25T09:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9892ffd700 1 --2- 172.21.15.99:0/1533132130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99b9990 0x55f4e99b9dd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9888009c80 tx=0x7f9888009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440145e04f2432f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9879ffb700 1 -- 172.21.15.99:0/1533132130 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888011cb0 con 0x55f4e99b9990 2023-08-25T09:09:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9893fff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f987c0a00c0 0x7f987c0a2580 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f988c0060b0 tx=0x7f988c004b40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9879ffb700 1 -- 172.21.15.99:0/1533132130 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9888011e10 con 0x55f4e99b9990 2023-08-25T09:09:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.599+0000 7f9879ffb700 1 -- 172.21.15.99:0/1533132130 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888005280 con 0x55f4e99b9990 2023-08-25T09:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 -- 172.21.15.99:0/1533132130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99b9990 msgr2=0x55f4e99b9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 --2- 172.21.15.99:0/1533132130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99b9990 0x55f4e99b9dd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9888009c80 tx=0x7f9888009f90 comp rx=0 tx=0).stop 2023-08-25T09:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 -- 172.21.15.99:0/1533132130 shutdown_connections 2023-08-25T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 --2- 172.21.15.99:0/1533132130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99b9990 0x55f4e99b9dd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 --2- 172.21.15.99:0/1533132130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99bf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 --2- 172.21.15.99:0/1533132130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99bf860 0x55f4e99b9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 -- 172.21.15.99:0/1533132130 >> 172.21.15.99:0/1533132130 conn(0x55f4e992e080 msgr2=0x55f4e99b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 -- 172.21.15.99:0/1533132130 shutdown_connections 2023-08-25T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.600+0000 7f98ad694b80 1 -- 172.21.15.99:0/1533132130 wait complete. 2023-08-25T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.601+0000 7f98ad694b80 1 -- start start 2023-08-25T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.601+0000 7f98ad694b80 1 Processor -- start 2023-08-25T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.601+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.601+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.601+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.601+0000 7f98ad694b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e990b930 con 0x55f4e99bef40 2023-08-25T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.601+0000 7f98ad694b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e98a9b50 con 0x55f4e99bf860 2023-08-25T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.601+0000 7f98ad694b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e98a9850 con 0x55f4e99b9990 2023-08-25T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98ad694b80 10 client.0 _ll_register_callbacks cb 0x55f4e97f3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:55004/0 (socket says 172.21.15.99:55004) 2023-08-25T09:09:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99c28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42588/0 (socket says 172.21.15.99:42588) 2023-08-25T09:09:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 -- 172.21.15.99:0/3173407579 learned_addr learned my addr 172.21.15.99:0/3173407579 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f9893fff700 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 -- 172.21.15.99:0/3173407579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 msgr2=0x55f4e99c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 -- 172.21.15.99:0/3173407579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 msgr2=0x55f4e99c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 -- 172.21.15.99:0/3173407579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98880098e0 con 0x55f4e99b9990 2023-08-25T09:09:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f98937fe700 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c56c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f988409a050 tx=0x7f9884021c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de1501a6f1ee6390 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f9872ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:09:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.602+0000 7f9872ffd700 1 -- 172.21.15.99:0/3173407579 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884099c00 con 0x55f4e99b9990 2023-08-25T09:09:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f9872ffd700 1 -- 172.21.15.99:0/3173407579 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f988409a7c0 con 0x55f4e99b9990 2023-08-25T09:09:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 1 -- 172.21.15.99:0/3173407579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4e9989160 con 0x55f4e99b9990 2023-08-25T09:09:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f9872ffd700 1 -- 172.21.15.99:0/3173407579 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9884099a20 con 0x55f4e99b9990 2023-08-25T09:09:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 10 client.26071 fetch_fsmap learned FSMap version 37 2023-08-25T09:09:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 1 -- 172.21.15.99:0/3173407579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f4e9737d80 con 0x55f4e99b9990 2023-08-25T09:09:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f9872ffd700 1 -- 172.21.15.99:0/3173407579 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4e9989160 con 0x55f4e99b9990 2023-08-25T09:09:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f9872ffd700 1 -- 172.21.15.99:0/3173407579 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f98840995b0 con 0x55f4e99b9990 2023-08-25T09:09:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:09:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 20 client.26071 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f4e99d2e40 0x55f4e99d5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 4 client.26071 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.603+0000 7f98ad694b80 1 -- 172.21.15.99:0/3173407579 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4e98b1b40 con 0x55f4e99d2e40 2023-08-25T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.604+0000 7f9892ffd700 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f4e99d2e40 0x55f4e99d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.604+0000 7f9892ffd700 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f4e99d2e40 0x55f4e99d5260 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0813ee928c91a9 server_cookie=29315016a43d578e in_seq=0 out_seq=0 2023-08-25T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.604+0000 7f9872ffd700 10 client.26071 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f9872ffd700 1 -- 172.21.15.99:0/3173407579 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f4e98b1b40 con 0x55f4e99d2e40 2023-08-25T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f9872ffd700 10 client.26071 handle_command_reply: tid=0 2023-08-25T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 client.26071 shutdown 2023-08-25T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 -- 172.21.15.99:0/3173407579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 msgr2=0x55f4e99c56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c56c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f988409a050 tx=0x7f9884021c00 comp rx=0 tx=0).stop 2023-08-25T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 -- 172.21.15.99:0/3173407579 shutdown_connections 2023-08-25T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f4e99d2e40 0x55f4e99d5260 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c56c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 --2- 172.21.15.99:0/3173407579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f9872ffd700 0 client.26071 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 -- 172.21.15.99:0/3173407579 >> 172.21.15.99:0/3173407579 conn(0x55f4e992e080 msgr2=0x55f4e99ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 -- 172.21.15.99:0/3173407579 shutdown_connections 2023-08-25T09:09:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 1 -- 172.21.15.99:0/3173407579 wait complete. 2023-08-25T09:09:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.605+0000 7f98ad694b80 20 client.26071 trim_cache size 0 max 16384 2023-08-25T09:09:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.620+0000 7f98ad694b80 1 Processor -- start 2023-08-25T09:09:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.620+0000 7f98ad694b80 1 -- start start 2023-08-25T09:09:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99b9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98ad694b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e978d8c0 con 0x55f4e99bef40 2023-08-25T09:09:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98ad694b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e99bf320 con 0x55f4e99bf860 2023-08-25T09:09:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98ad694b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e99c6030 con 0x55f4e99b9990 2023-08-25T09:09:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f9893fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f9893fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52600/0 (socket says 172.21.15.99:52600) 2023-08-25T09:09:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f9893fff700 1 -- 172.21.15.99:0/1401927638 learned_addr learned my addr 172.21.15.99:0/1401927638 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98937fe700 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98937fe700 1 -- 172.21.15.99:0/1401927638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 msgr2=0x55f4e99c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98937fe700 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98937fe700 1 -- 172.21.15.99:0/1401927638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 msgr2=0x55f4e99ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98937fe700 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f98937fe700 1 -- 172.21.15.99:0/1401927638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4e99adb40 con 0x55f4e99b9990 2023-08-25T09:09:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.621+0000 7f9892ffd700 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98937fe700 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99b9450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f98840ce040 tx=0x7f9884009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e7e434f8cf1d678 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f987a7fc700 1 -- 172.21.15.99:0/1401927638 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884095eb0 con 0x55f4e99b9990 2023-08-25T09:09:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f987a7fc700 1 -- 172.21.15.99:0/1401927638 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98840b2140 con 0x55f4e99b9990 2023-08-25T09:09:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f987a7fc700 1 -- 172.21.15.99:0/1401927638 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840a1e40 con 0x55f4e99b9990 2023-08-25T09:09:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 -- 172.21.15.99:0/1401927638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 msgr2=0x55f4e99b9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99b9450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f98840ce040 tx=0x7f9884009ca0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 -- 172.21.15.99:0/1401927638 shutdown_connections 2023-08-25T09:09:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bf860 0x55f4e99c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99b9450 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 --2- 172.21.15.99:0/1401927638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bef40 0x55f4e99ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 -- 172.21.15.99:0/1401927638 >> 172.21.15.99:0/1401927638 conn(0x55f4e992e080 msgr2=0x55f4e99bca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 -- 172.21.15.99:0/1401927638 shutdown_connections 2023-08-25T09:09:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.622+0000 7f98ad694b80 1 -- 172.21.15.99:0/1401927638 wait complete. 2023-08-25T09:09:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.623+0000 7f98ad694b80 1 -- start start 2023-08-25T09:09:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.623+0000 7f98ad694b80 1 Processor -- start 2023-08-25T09:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.623+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.623+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99c1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98ad694b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bf860 0x55f4e99c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98ad694b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e99c2190 con 0x55f4e99bf860 2023-08-25T09:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98ad694b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e99c2300 con 0x55f4e99bef40 2023-08-25T09:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98ad694b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e99c24c0 con 0x55f4e99b9990 2023-08-25T09:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98ad694b80 10 client.0 _ll_register_callbacks cb 0x55f4e97f3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:55016/0 (socket says 172.21.15.99:55016) 2023-08-25T09:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 -- 172.21.15.99:0/2300985729 learned_addr learned my addr 172.21.15.99:0/2300985729 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f9893fff700 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bf860 0x55f4e99c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f9892ffd700 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 -- 172.21.15.99:0/2300985729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 msgr2=0x55f4e99c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99c1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 -- 172.21.15.99:0/2300985729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bf860 msgr2=0x55f4e99c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bf860 0x55f4e99c1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 -- 172.21.15.99:0/2300985729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988801d040 con 0x55f4e99b9990 2023-08-25T09:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f9893fff700 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bf860 0x55f4e99c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f98937fe700 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c0e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98840af040 tx=0x7f98840b22b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8e771a7e24c0eb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f9872ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.624+0000 7f9872ffd700 1 -- 172.21.15.99:0/2300985729 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884095eb0 con 0x55f4e99b9990 2023-08-25T09:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 1 -- 172.21.15.99:0/2300985729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4e99ac600 con 0x55f4e99b9990 2023-08-25T09:09:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f9872ffd700 1 -- 172.21.15.99:0/2300985729 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f988409a0c0 con 0x55f4e99b9990 2023-08-25T09:09:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f9872ffd700 1 -- 172.21.15.99:0/2300985729 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9884099a20 con 0x55f4e99b9990 2023-08-25T09:09:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 10 client.26077 fetch_fsmap learned FSMap version 37 2023-08-25T09:09:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 1 -- 172.21.15.99:0/2300985729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f4e9737d80 con 0x55f4e99b9990 2023-08-25T09:09:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f9872ffd700 1 -- 172.21.15.99:0/2300985729 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4e99ac600 con 0x55f4e99b9990 2023-08-25T09:09:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f9872ffd700 1 -- 172.21.15.99:0/2300985729 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f98840ad0b0 con 0x55f4e99b9990 2023-08-25T09:09:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 10 client.26077 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 10 client.26077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 20 client.26077 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f4e9a2fc10 0x55f4e9a32000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 4 client.26077 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f98ad694b80 1 -- 172.21.15.99:0/2300985729 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4e9904910 con 0x55f4e9a2fc10 2023-08-25T09:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.625+0000 7f9892ffd700 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f4e9a2fc10 0x55f4e9a32000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.626+0000 7f9892ffd700 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f4e9a2fc10 0x55f4e9a32000 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dbf4e137e42f97d server_cookie=bc8946183e8f9b2a in_seq=0 out_seq=0 2023-08-25T09:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.626+0000 7f9872ffd700 10 client.26077 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:15.634 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:15 smithi157 ceph-mon[130879]: pgmap v313: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2023-08-25T09:09:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f9872ffd700 1 -- 172.21.15.99:0/2300985729 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4e9904910 con 0x55f4e9a2fc10 2023-08-25T09:09:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f9872ffd700 10 client.26077 handle_command_reply: tid=0 2023-08-25T09:09:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 client.26077 shutdown 2023-08-25T09:09:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 -- 172.21.15.99:0/2300985729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 msgr2=0x55f4e99c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c0e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98840af040 tx=0x7f98840b22b0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 -- 172.21.15.99:0/2300985729 shutdown_connections 2023-08-25T09:09:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4e99bf860 0x55f4e99c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f4e99b9990 0x55f4e99c0e30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f4e9a2fc10 0x55f4e9a32000 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 --2- 172.21.15.99:0/2300985729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4e99bef40 0x55f4e99c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f9872ffd700 0 client.26077 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.664+0000 7f98ad694b80 1 -- 172.21.15.99:0/2300985729 >> 172.21.15.99:0/2300985729 conn(0x55f4e992e080 msgr2=0x55f4e99bfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f98ad694b80 1 -- 172.21.15.99:0/2300985729 shutdown_connections 2023-08-25T09:09:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f98ad694b80 1 -- 172.21.15.99:0/2300985729 wait complete. 2023-08-25T09:09:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f98ad694b80 20 client.26077 trim_cache size 0 max 16384 2023-08-25T09:09:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:09:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:09:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:09:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:09:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 -- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f987c0a00c0 msgr2=0x7f987c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f987c0a00c0 0x7f987c0a2580 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f988c0060b0 tx=0x7f988c004b40 comp rx=0 tx=0).stop 2023-08-25T09:09:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 -- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941008b0 msgr2=0x7f9894072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941008b0 0x7f9894072e00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9884000c00 tx=0x7f9884004790 comp rx=0 tx=0).stop 2023-08-25T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 -- 172.21.15.99:0/4108212913 shutdown_connections 2023-08-25T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f987c0a00c0 0x7f987c0a2580 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98941068c0 0x7f98941a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98941008b0 0x7f9894072e00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 --2- 172.21.15.99:0/4108212913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9894101270 0x7f9894073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.665+0000 7f9899dff700 1 -- 172.21.15.99:0/4108212913 >> 172.21.15.99:0/4108212913 conn(0x7f98940fc380 msgr2=0x7f98940fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.666+0000 7f9899dff700 1 -- 172.21.15.99:0/4108212913 shutdown_connections 2023-08-25T09:09:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:15.666+0000 7f9899dff700 1 -- 172.21.15.99:0/4108212913 wait complete. 2023-08-25T09:09:15.938 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.13 MB/sec execute 441 sec latency 1039.337 ms 2023-08-25T09:09:16.938 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.12 MB/sec execute 442 sec latency 2039.449 ms 2023-08-25T09:09:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:17 smithi181 ceph-mon[131296]: pgmap v314: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-08-25T09:09:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:17 smithi099 ceph-mon[97641]: pgmap v314: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-08-25T09:09:17.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:17 smithi157 ceph-mon[130879]: pgmap v314: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-08-25T09:09:17.938 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.11 MB/sec execute 443 sec latency 3039.579 ms 2023-08-25T09:09:18.938 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.10 MB/sec execute 444 sec latency 4039.740 ms 2023-08-25T09:09:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:19 smithi181 ceph-mon[131296]: pgmap v315: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-08-25T09:09:19.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:19 smithi099 ceph-mon[97641]: pgmap v315: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-08-25T09:09:19.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:19 smithi157 ceph-mon[130879]: pgmap v315: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-08-25T09:09:19.938 INFO:tasks.workunit.client.0.smithi099.stdout: 1 407175 4.11 MB/sec execute 445 sec latency 4584.300 ms 2023-08-25T09:09:20.938 INFO:tasks.workunit.client.0.smithi099.stdout: 1 408542 4.12 MB/sec execute 446 sec latency 13.648 ms 2023-08-25T09:09:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:21 smithi181 ceph-mon[131296]: pgmap v316: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2023-08-25T09:09:21.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:21 smithi099 ceph-mon[97641]: pgmap v316: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2023-08-25T09:09:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:21 smithi157 ceph-mon[130879]: pgmap v316: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2023-08-25T09:09:21.938 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409018 4.11 MB/sec execute 447 sec latency 703.794 ms 2023-08-25T09:09:22.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409018 4.10 MB/sec execute 448 sec latency 1703.914 ms 2023-08-25T09:09:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:23 smithi181 ceph-mon[131296]: pgmap v317: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2023-08-25T09:09:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:23 smithi099 ceph-mon[97641]: pgmap v317: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2023-08-25T09:09:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:23 smithi157 ceph-mon[130879]: pgmap v317: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2023-08-25T09:09:23.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409018 4.09 MB/sec execute 449 sec latency 2704.059 ms 2023-08-25T09:09:24.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409635 4.09 MB/sec execute 450 sec latency 3259.452 ms 2023-08-25T09:09:25.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:25 smithi181 ceph-mon[131296]: pgmap v318: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 997 KiB/s wr, 81 op/s 2023-08-25T09:09:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:25 smithi099 ceph-mon[97641]: pgmap v318: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 997 KiB/s wr, 81 op/s 2023-08-25T09:09:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:25 smithi157 ceph-mon[130879]: pgmap v318: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 997 KiB/s wr, 81 op/s 2023-08-25T09:09:25.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409641 4.08 MB/sec execute 451 sec latency 997.394 ms 2023-08-25T09:09:26.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409641 4.07 MB/sec execute 452 sec latency 1997.520 ms 2023-08-25T09:09:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:27 smithi181 ceph-mon[131296]: pgmap v319: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2023-08-25T09:09:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:27 smithi099 ceph-mon[97641]: pgmap v319: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2023-08-25T09:09:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:27 smithi157 ceph-mon[130879]: pgmap v319: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2023-08-25T09:09:27.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409641 4.06 MB/sec execute 453 sec latency 2997.644 ms 2023-08-25T09:09:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 -- 172.21.15.99:0/3097122717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01008b0 msgr2=0x7fc7b0100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/3097122717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01008b0 0x7fc7b0100d30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009990 tx=0x7fc7a4009ca0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 -- 172.21.15.99:0/3097122717 shutdown_connections 2023-08-25T09:09:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/3097122717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7b0101270 0x7fc7b010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/3097122717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b01068c0 0x7fc7b0106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/3097122717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01008b0 0x7fc7b0100d30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 -- 172.21.15.99:0/3097122717 >> 172.21.15.99:0/3097122717 conn(0x7fc7b00fc380 msgr2=0x7fc7b00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 -- 172.21.15.99:0/3097122717 shutdown_connections 2023-08-25T09:09:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.830+0000 7fc7b65ff700 1 -- 172.21.15.99:0/3097122717 wait complete. 2023-08-25T09:09:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.831+0000 7fc7b65ff700 1 Processor -- start 2023-08-25T09:09:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b65ff700 1 -- start start 2023-08-25T09:09:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b65ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7b01008b0 0x7fc7b019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b65ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b0101270 0x7fc7b019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b65ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01068c0 0x7fc7b01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b65ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0199020 con 0x7fc7b01068c0 2023-08-25T09:09:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b65ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0199190 con 0x7fc7b0101270 2023-08-25T09:09:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b65ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0199350 con 0x7fc7b01008b0 2023-08-25T09:09:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b4b9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01068c0 0x7fc7b01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b4b9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01068c0 0x7fc7b01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53292/0 (socket says 172.21.15.99:53292) 2023-08-25T09:09:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7b4b9c700 1 -- 172.21.15.99:0/4018055787 learned_addr learned my addr 172.21.15.99:0/4018055787 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7affff700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7b01008b0 0x7fc7b019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.832+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b0101270 0x7fc7b019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7b4b9c700 1 -- 172.21.15.99:0/4018055787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7b01008b0 msgr2=0x7fc7b019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7b4b9c700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7b01008b0 0x7fc7b019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7b4b9c700 1 -- 172.21.15.99:0/4018055787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b0101270 msgr2=0x7fc7b019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7b4b9c700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b0101270 0x7fc7b019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7b4b9c700 1 -- 172.21.15.99:0/4018055787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0009520 con 0x7fc7b01068c0 2023-08-25T09:09:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b0101270 0x7fc7b019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7b4b9c700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01068c0 0x7fc7b01989b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800ef30 tx=0x7fc7a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7ad7fa700 1 -- 172.21.15.99:0/4018055787 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8018070 con 0x7fc7b01068c0 2023-08-25T09:09:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7b65ff700 1 -- 172.21.15.99:0/4018055787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a40095f0 con 0x7fc7b01068c0 2023-08-25T09:09:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7ad7fa700 1 -- 172.21.15.99:0/4018055787 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a800f040 con 0x7fc7b01068c0 2023-08-25T09:09:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7ad7fa700 1 -- 172.21.15.99:0/4018055787 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a800cd00 con 0x7fc7b01068c0 2023-08-25T09:09:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.833+0000 7fc7b65ff700 1 -- 172.21.15.99:0/4018055787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b01a5960 con 0x7fc7b01068c0 2023-08-25T09:09:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.836+0000 7fc7ad7fa700 1 -- 172.21.15.99:0/4018055787 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fc7a8003a00 con 0x7fc7b01068c0 2023-08-25T09:09:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7c9e7cb80 1 Processor -- start 2023-08-25T09:09:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7ad7fa700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc7980a00c0 0x7fc7980a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7c9e7cb80 1 -- start start 2023-08-25T09:09:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7ad7fa700 1 -- 172.21.15.99:0/4018055787 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fc7a80cb180 con 0x7fc7b01068c0 2023-08-25T09:09:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc7980a00c0 0x7fc7980a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd001320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcf63750 con 0x55cbfd001860 2023-08-25T09:09:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcf54720 con 0x55cbfd000f40 2023-08-25T09:09:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.838+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcf52040 con 0x55cbfcffb990 2023-08-25T09:09:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.839+0000 7fc7affff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd001320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.839+0000 7fc7affff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd001320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:37164/0 (socket says 172.21.15.99:37164) 2023-08-25T09:09:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.839+0000 7fc7affff700 1 -- 172.21.15.99:0/1157289547 learned_addr learned my addr 172.21.15.99:0/1157289547 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.839+0000 7fc7b4b9c700 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.839+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.840+0000 7fc7affff700 1 -- 172.21.15.99:0/1157289547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 msgr2=0x55cbfcffbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.840+0000 7fc7affff700 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.840+0000 7fc7affff700 1 -- 172.21.15.99:0/1157289547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 msgr2=0x55cbfcffb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.840+0000 7fc7affff700 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.840+0000 7fc7affff700 1 -- 172.21.15.99:0/1157289547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbfcdcd5d0 con 0x55cbfd000f40 2023-08-25T09:09:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.840+0000 7fc7affff700 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd001320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009e90 tx=0x7fc7a0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32690061cd8cf956 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.840+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc7980a00c0 0x7fc7980a2580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0105f80 tx=0x7fc7a400bf00 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.841+0000 7fc7967fc700 1 -- 172.21.15.99:0/1157289547 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000e670 con 0x55cbfd000f40 2023-08-25T09:09:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.841+0000 7fc7967fc700 1 -- 172.21.15.99:0/1157289547 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a000ec90 con 0x55cbfd000f40 2023-08-25T09:09:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.841+0000 7fc7967fc700 1 -- 172.21.15.99:0/1157289547 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0019d00 con 0x55cbfd000f40 2023-08-25T09:09:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1157289547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 msgr2=0x55cbfd001320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd001320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009e90 tx=0x7fc7a0011720 comp rx=0 tx=0).stop 2023-08-25T09:09:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1157289547 shutdown_connections 2023-08-25T09:09:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd001320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1157289547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1157289547 >> 172.21.15.99:0/1157289547 conn(0x55cbfcf70080 msgr2=0x55cbfcff5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1157289547 shutdown_connections 2023-08-25T09:09:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1157289547 wait complete. 2023-08-25T09:09:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.842+0000 7fc7c9e7cb80 1 -- start start 2023-08-25T09:09:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7c9e7cb80 1 Processor -- start 2023-08-25T09:09:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfcffb990 0x55cbfd007690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd000f40 0x55cbfd0048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfd001860 0x55cbfd004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcf4d930 con 0x55cbfd000f40 2023-08-25T09:09:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfceebb50 con 0x55cbfcffb990 2023-08-25T09:09:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfceeb850 con 0x55cbfd001860 2023-08-25T09:09:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7c9e7cb80 10 client.0 _ll_register_callbacks cb 0x55cbfce35a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7affff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfcffb990 0x55cbfd007690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7affff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfcffb990 0x55cbfd007690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:37174/0 (socket says 172.21.15.99:37174) 2023-08-25T09:09:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7b4b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfd001860 0x55cbfd004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7affff700 1 -- 172.21.15.99:0/1074272508 learned_addr learned my addr 172.21.15.99:0/1074272508 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.843+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd000f40 0x55cbfd0048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc7affff700 1 -- 172.21.15.99:0/1074272508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfd001860 msgr2=0x55cbfd004eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc7affff700 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfd001860 0x55cbfd004eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc7affff700 1 -- 172.21.15.99:0/1074272508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd000f40 msgr2=0x55cbfd0048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc7affff700 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd000f40 0x55cbfd0048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc7affff700 1 -- 172.21.15.99:0/1074272508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a0009af0 con 0x55cbfcffb990 2023-08-25T09:09:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc7b4b9c700 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfd001860 0x55cbfd004eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc7affff700 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfcffb990 0x55cbfd007690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0019880 tx=0x7fc7a00195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb8467e035999ce server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc78f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:09:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc78f7fe700 1 -- 172.21.15.99:0/1074272508 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0019cf0 con 0x55cbfcffb990 2023-08-25T09:09:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1074272508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbfcfcb160 con 0x55cbfcffb990 2023-08-25T09:09:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.844+0000 7fc78f7fe700 1 -- 172.21.15.99:0/1074272508 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fc7a0003680 con 0x55cbfcffb990 2023-08-25T09:09:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc78f7fe700 1 -- 172.21.15.99:0/1074272508 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a0019e50 con 0x55cbfcffb990 2023-08-25T09:09:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc7c9e7cb80 10 client.26105 fetch_fsmap learned FSMap version 37 2023-08-25T09:09:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1074272508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cbfcd79d80 con 0x55cbfcffb990 2023-08-25T09:09:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc78f7fe700 1 -- 172.21.15.99:0/1074272508 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cbfcfcb160 con 0x55cbfcffb990 2023-08-25T09:09:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc78f7fe700 1 -- 172.21.15.99:0/1074272508 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc7a0023770 con 0x55cbfcffb990 2023-08-25T09:09:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc7c9e7cb80 10 client.26105 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:09:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc7c9e7cb80 10 client.26105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc7c9e7cb80 20 client.26105 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55cbfd015130 0x55cbfd017550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc7c9e7cb80 4 client.26105 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:09:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.845+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1074272508 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbfcef3b40 con 0x55cbfd015130 2023-08-25T09:09:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.846+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55cbfd015130 0x55cbfd017550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.846+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55cbfd015130 0x55cbfd017550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6169971d310cafc server_cookie=684c939655aa1e96 in_seq=0 out_seq=0 2023-08-25T09:09:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.846+0000 7fc78f7fe700 10 client.26105 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc78f7fe700 1 -- 172.21.15.99:0/1074272508 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cbfcef3b40 con 0x55cbfd015130 2023-08-25T09:09:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc78f7fe700 10 client.26105 handle_command_reply: tid=0 2023-08-25T09:09:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 client.26105 shutdown 2023-08-25T09:09:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1074272508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfcffb990 msgr2=0x55cbfd007690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfcffb990 0x55cbfd007690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0019880 tx=0x7fc7a00195f0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1074272508 shutdown_connections 2023-08-25T09:09:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfd001860 0x55cbfd004eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55cbfd015130 0x55cbfd017550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfcffb990 0x55cbfd007690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc78f7fe700 0 client.26105 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/1074272508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd000f40 0x55cbfd0048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1074272508 >> 172.21.15.99:0/1074272508 conn(0x55cbfcf70080 msgr2=0x55cbfcff0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1074272508 shutdown_connections 2023-08-25T09:09:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/1074272508 wait complete. 2023-08-25T09:09:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.847+0000 7fc7c9e7cb80 20 client.26105 trim_cache size 0 max 16384 2023-08-25T09:09:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.862+0000 7fc7c9e7cb80 1 Processor -- start 2023-08-25T09:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.862+0000 7fc7c9e7cb80 1 -- start start 2023-08-25T09:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd00c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfd0071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcdcf8c0 con 0x55cbfd001860 2023-08-25T09:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd005e60 con 0x55cbfd000f40 2023-08-25T09:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd001320 con 0x55cbfcffb990 2023-08-25T09:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7b4b9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfd0071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7b4b9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfd0071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53328/0 (socket says 172.21.15.99:53328) 2023-08-25T09:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7b4b9c700 1 -- 172.21.15.99:0/522631290 learned_addr learned my addr 172.21.15.99:0/522631290 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7affff700 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd00c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7affff700 1 -- 172.21.15.99:0/522631290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 msgr2=0x55cbfd00c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7affff700 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd00c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7affff700 1 -- 172.21.15.99:0/522631290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 msgr2=0x55cbfd0071c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7affff700 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfd0071c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7affff700 1 -- 172.21.15.99:0/522631290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbfcd62c00 con 0x55cbfcffb990 2023-08-25T09:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.863+0000 7fc7b4b9c700 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfd0071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7affff700 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffb450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009de0 tx=0x7fc7a0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6917668a5a3df5ee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc796ffd700 1 -- 172.21.15.99:0/522631290 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0023c20 con 0x55cbfcffb990 2023-08-25T09:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc796ffd700 1 -- 172.21.15.99:0/522631290 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a0019430 con 0x55cbfcffb990 2023-08-25T09:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc796ffd700 1 -- 172.21.15.99:0/522631290 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00052d0 con 0x55cbfcffb990 2023-08-25T09:09:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/522631290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 msgr2=0x55cbfcffb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffb450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009de0 tx=0x7fc7a0011720 comp rx=0 tx=0).stop 2023-08-25T09:09:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/522631290 shutdown_connections 2023-08-25T09:09:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfd0071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffb450 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/522631290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfd00c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/522631290 >> 172.21.15.99:0/522631290 conn(0x55cbfcf70080 msgr2=0x55cbfcffeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/522631290 shutdown_connections 2023-08-25T09:09:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.864+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/522631290 wait complete. 2023-08-25T09:09:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.865+0000 7fc7c9e7cb80 1 -- start start 2023-08-25T09:09:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.865+0000 7fc7c9e7cb80 1 Processor -- start 2023-08-25T09:09:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.865+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.865+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfcffcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.865+0000 7fc7c9e7cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcffd840 con 0x55cbfd001860 2023-08-25T09:09:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcffd9b0 con 0x55cbfd000f40 2023-08-25T09:09:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7c9e7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcffdb70 con 0x55cbfcffb990 2023-08-25T09:09:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7c9e7cb80 10 client.0 _ll_register_callbacks cb 0x7fc7a0009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:53146/0 (socket says 172.21.15.99:53146) 2023-08-25T09:09:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 -- 172.21.15.99:0/3679933629 learned_addr learned my addr 172.21.15.99:0/3679933629 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfcffcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7b4b9c700 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 -- 172.21.15.99:0/3679933629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 msgr2=0x55cbfcffcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfcffcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 -- 172.21.15.99:0/3679933629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 msgr2=0x55cbfcffd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 -- 172.21.15.99:0/3679933629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b01a5960 con 0x55cbfcffb990 2023-08-25T09:09:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7b4b9c700 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:09:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc7affff700 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffc880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009920 tx=0x7fc7a0004370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e981eadefb356b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.866+0000 7fc78f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:09:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc78f7fe700 1 -- 172.21.15.99:0/3679933629 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00092b0 con 0x55cbfcffb990 2023-08-25T09:09:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/3679933629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbfcfee600 con 0x55cbfcffb990 2023-08-25T09:09:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc78f7fe700 1 -- 172.21.15.99:0/3679933629 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fc7a003d030 con 0x55cbfcffb990 2023-08-25T09:09:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc78f7fe700 1 -- 172.21.15.99:0/3679933629 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a0019af0 con 0x55cbfcffb990 2023-08-25T09:09:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 10 client.26095 fetch_fsmap learned FSMap version 37 2023-08-25T09:09:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/3679933629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cbfcd79d80 con 0x55cbfcffb990 2023-08-25T09:09:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc78f7fe700 1 -- 172.21.15.99:0/3679933629 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cbfcfee600 con 0x55cbfcffb990 2023-08-25T09:09:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc78f7fe700 1 -- 172.21.15.99:0/3679933629 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc7a00046a0 con 0x55cbfcffb990 2023-08-25T09:09:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 10 client.26095 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:09:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 10 client.26095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 20 client.26095 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55cbfd069500 0x55cbfd06b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 4 client.26095 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:09:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.867+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/3679933629 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cbfcf46910 con 0x55cbfd069500 2023-08-25T09:09:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.868+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55cbfd069500 0x55cbfd06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.868+0000 7fc7af7fe700 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55cbfd069500 0x55cbfd06b920 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff21f2a39efde73f server_cookie=1ce62b4922af7f0b in_seq=0 out_seq=0 2023-08-25T09:09:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.868+0000 7fc78f7fe700 10 client.26095 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc78f7fe700 1 -- 172.21.15.99:0/3679933629 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cbfcf46910 con 0x55cbfd069500 2023-08-25T09:09:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc78f7fe700 10 client.26095 handle_command_reply: tid=0 2023-08-25T09:09:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 client.26095 shutdown 2023-08-25T09:09:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/3679933629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 msgr2=0x55cbfcffc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffc880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009920 tx=0x7fc7a0004370 comp rx=0 tx=0).stop 2023-08-25T09:09:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/3679933629 shutdown_connections 2023-08-25T09:09:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbfd001860 0x55cbfcffd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbfcffb990 0x55cbfcffc880 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55cbfd069500 0x55cbfd06b920 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 --2- 172.21.15.99:0/3679933629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbfd000f40 0x55cbfcffcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc78f7fe700 0 client.26095 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/3679933629 >> 172.21.15.99:0/3679933629 conn(0x55cbfcf70080 msgr2=0x55cbfd004190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/3679933629 shutdown_connections 2023-08-25T09:09:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 1 -- 172.21.15.99:0/3679933629 wait complete. 2023-08-25T09:09:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.910+0000 7fc7c9e7cb80 20 client.26095 trim_cache size 0 max 16384 2023-08-25T09:09:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:09:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:09:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:09:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:09:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 -- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc7980a00c0 msgr2=0x7fc7980a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc7980a00c0 0x7fc7980a2580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0105f80 tx=0x7fc7a400bf00 comp rx=0 tx=0).stop 2023-08-25T09:09:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 -- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01068c0 msgr2=0x7fc7b01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01068c0 0x7fc7b01989b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800ef30 tx=0x7fc7a800c3a0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 -- 172.21.15.99:0/4018055787 shutdown_connections 2023-08-25T09:09:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc7980a00c0 0x7fc7980a2580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b01068c0 0x7fc7b01989b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7b01008b0 0x7fc7b019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 --2- 172.21.15.99:0/4018055787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b0101270 0x7fc7b019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 -- 172.21.15.99:0/4018055787 >> 172.21.15.99:0/4018055787 conn(0x7fc7b00fc380 msgr2=0x7fc7b010bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 -- 172.21.15.99:0/4018055787 shutdown_connections 2023-08-25T09:09:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:28.911+0000 7fc7b65ff700 1 -- 172.21.15.99:0/4018055787 wait complete. 2023-08-25T09:09:28.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409641 4.05 MB/sec execute 454 sec latency 3997.747 ms 2023-08-25T09:09:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:29 smithi181 ceph-mon[131296]: pgmap v320: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 849 KiB/s wr, 62 op/s 2023-08-25T09:09:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:29 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:09:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:29 smithi099 ceph-mon[97641]: pgmap v320: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 849 KiB/s wr, 62 op/s 2023-08-25T09:09:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:29 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:09:29.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:29 smithi157 ceph-mon[130879]: pgmap v320: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 849 KiB/s wr, 62 op/s 2023-08-25T09:09:29.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:29 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:09:29.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 410530 4.05 MB/sec execute 455 sec latency 4532.865 ms 2023-08-25T09:09:30.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 411909 4.07 MB/sec execute 456 sec latency 20.359 ms 2023-08-25T09:09:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:31 smithi181 ceph-mon[131296]: pgmap v321: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 849 KiB/s wr, 62 op/s 2023-08-25T09:09:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:31 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:31 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:31 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:31 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:31 smithi099 ceph-mon[97641]: pgmap v321: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 849 KiB/s wr, 62 op/s 2023-08-25T09:09:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:31 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:31 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:31 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:31 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:31 smithi157 ceph-mon[130879]: pgmap v321: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 849 KiB/s wr, 62 op/s 2023-08-25T09:09:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:31 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:31 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:31 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:31 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:31.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 412560 4.06 MB/sec execute 457 sec latency 394.894 ms 2023-08-25T09:09:32.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi157 to 196.2M 2023-08-25T09:09:32.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: Unable to set osd_memory_target on smithi157 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T09:09:32.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-08-25T09:09:32.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-08-25T09:09:32.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:32.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:32 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:32.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi157 to 196.2M 2023-08-25T09:09:32.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: Unable to set osd_memory_target on smithi157 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T09:09:32.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-08-25T09:09:32.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-08-25T09:09:32.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:32.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:32 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:32.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi157 to 196.2M 2023-08-25T09:09:32.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: Unable to set osd_memory_target on smithi157 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T09:09:32.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:32.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-08-25T09:09:32.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-08-25T09:09:32.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:32.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:32 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:32.939 INFO:tasks.workunit.client.0.smithi099.stdout: 1 412560 4.05 MB/sec execute 458 sec latency 1394.930 ms 2023-08-25T09:09:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:33 smithi181 ceph-mon[131296]: pgmap v322: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2023-08-25T09:09:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:33 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:33 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:33 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:33 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:33 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:09:33.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:33 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:09:33.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:33 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:33 smithi099 ceph-mon[97641]: pgmap v322: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2023-08-25T09:09:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:33 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:33 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:33 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:33 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:33 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:09:33.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:33 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:09:33.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:33 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:33 smithi157 ceph-mon[130879]: pgmap v322: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2023-08-25T09:09:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:33 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:33 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:33 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:33 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T09:09:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:33 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:09:33.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:33 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:09:33.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:33 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:09:33.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 412560 4.04 MB/sec execute 459 sec latency 2395.037 ms 2023-08-25T09:09:34.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:34 smithi181 ceph-mon[131296]: Adjusting osd_memory_target on smithi099 to 220.9M 2023-08-25T09:09:34.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:34 smithi181 ceph-mon[131296]: Unable to set osd_memory_target on smithi099 to 231651840: error parsing value: Value '231651840' is below minimum 939524096 2023-08-25T09:09:34.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:34 smithi099 ceph-mon[97641]: Adjusting osd_memory_target on smithi099 to 220.9M 2023-08-25T09:09:34.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:34 smithi099 ceph-mon[97641]: Unable to set osd_memory_target on smithi099 to 231651840: error parsing value: Value '231651840' is below minimum 939524096 2023-08-25T09:09:34.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:34 smithi157 ceph-mon[130879]: Adjusting osd_memory_target on smithi099 to 220.9M 2023-08-25T09:09:34.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:34 smithi157 ceph-mon[130879]: Unable to set osd_memory_target on smithi099 to 231651840: error parsing value: Value '231651840' is below minimum 939524096 2023-08-25T09:09:34.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 413183 4.04 MB/sec execute 460 sec latency 2942.969 ms 2023-08-25T09:09:35.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:35 smithi181 ceph-mon[131296]: pgmap v323: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 879 KiB/s wr, 68 op/s 2023-08-25T09:09:35.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:35 smithi099 ceph-mon[97641]: pgmap v323: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 879 KiB/s wr, 68 op/s 2023-08-25T09:09:35.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:35 smithi157 ceph-mon[130879]: pgmap v323: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 879 KiB/s wr, 68 op/s 2023-08-25T09:09:35.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 413183 4.03 MB/sec execute 461 sec latency 1021.802 ms 2023-08-25T09:09:36.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 413183 4.03 MB/sec execute 462 sec latency 2021.913 ms 2023-08-25T09:09:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:37 smithi181 ceph-mon[131296]: pgmap v324: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2023-08-25T09:09:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:37 smithi099 ceph-mon[97641]: pgmap v324: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2023-08-25T09:09:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:37 smithi157 ceph-mon[130879]: pgmap v324: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2023-08-25T09:09:37.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 413183 4.02 MB/sec execute 463 sec latency 3022.062 ms 2023-08-25T09:09:38.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 413183 4.01 MB/sec execute 464 sec latency 4022.176 ms 2023-08-25T09:09:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:39 smithi181 ceph-mon[131296]: pgmap v325: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 918 KiB/s wr, 73 op/s 2023-08-25T09:09:39.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:39 smithi099 ceph-mon[97641]: pgmap v325: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 918 KiB/s wr, 73 op/s 2023-08-25T09:09:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:39 smithi157 ceph-mon[130879]: pgmap v325: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 918 KiB/s wr, 73 op/s 2023-08-25T09:09:39.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 414112 4.01 MB/sec execute 465 sec latency 4562.835 ms 2023-08-25T09:09:40.940 INFO:tasks.workunit.client.0.smithi099.stdout: 1 415610 4.02 MB/sec execute 466 sec latency 13.022 ms 2023-08-25T09:09:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:41 smithi181 ceph-mon[131296]: pgmap v326: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 918 KiB/s wr, 73 op/s 2023-08-25T09:09:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:41 smithi099 ceph-mon[97641]: pgmap v326: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 918 KiB/s wr, 73 op/s 2023-08-25T09:09:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:41 smithi157 ceph-mon[130879]: pgmap v326: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 918 KiB/s wr, 73 op/s 2023-08-25T09:09:41.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416109 4.02 MB/sec execute 467 sec latency 668.406 ms 2023-08-25T09:09:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 -- 172.21.15.99:0/1514708024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb741068b0 msgr2=0x7feb74106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 --2- 172.21.15.99:0/1514708024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb741068b0 0x7feb74106c90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feb64009990 tx=0x7feb64009ca0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 -- 172.21.15.99:0/1514708024 shutdown_connections 2023-08-25T09:09:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 --2- 172.21.15.99:0/1514708024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb74101260 0x7feb7410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 --2- 172.21.15.99:0/1514708024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb741068b0 0x7feb74106c90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 --2- 172.21.15.99:0/1514708024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb741008a0 0x7feb74100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 -- 172.21.15.99:0/1514708024 >> 172.21.15.99:0/1514708024 conn(0x7feb740fc370 msgr2=0x7feb740fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 -- 172.21.15.99:0/1514708024 shutdown_connections 2023-08-25T09:09:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.066+0000 7feb793ff700 1 -- 172.21.15.99:0/1514708024 wait complete. 2023-08-25T09:09:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.067+0000 7feb793ff700 1 Processor -- start 2023-08-25T09:09:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.067+0000 7feb793ff700 1 -- start start 2023-08-25T09:09:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb793ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb741008a0 0x7feb7419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb793ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb74101260 0x7feb7419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb793ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb741068b0 0x7feb74198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb793ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74198fd0 con 0x7feb74101260 2023-08-25T09:09:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb793ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74199140 con 0x7feb741068b0 2023-08-25T09:09:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb793ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74199300 con 0x7feb741008a0 2023-08-25T09:09:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb727fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb74101260 0x7feb7419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb737fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb741068b0 0x7feb74198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb741008a0 0x7feb7419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb727fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb74101260 0x7feb7419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38330/0 (socket says 172.21.15.99:38330) 2023-08-25T09:09:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb737fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb741068b0 0x7feb74198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:37458/0 (socket says 172.21.15.99:37458) 2023-08-25T09:09:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.068+0000 7feb727fc700 1 -- 172.21.15.99:0/3662478013 learned_addr learned my addr 172.21.15.99:0/3662478013 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb72ffd700 1 -- 172.21.15.99:0/3662478013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb741068b0 msgr2=0x7feb74198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb72ffd700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb741068b0 0x7feb74198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb72ffd700 1 -- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb74101260 msgr2=0x7feb7419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb72ffd700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb74101260 0x7feb7419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb72ffd700 1 -- 172.21.15.99:0/3662478013 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb640095f0 con 0x7feb741008a0 2023-08-25T09:09:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb737fe700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb741068b0 0x7feb74198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb727fc700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb74101260 0x7feb7419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb72ffd700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb741008a0 0x7feb7419eb00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feb64000c00 tx=0x7feb64009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb5bfff700 1 -- 172.21.15.99:0/3662478013 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6401d070 con 0x7feb741008a0 2023-08-25T09:09:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb793ff700 1 -- 172.21.15.99:0/3662478013 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb74199580 con 0x7feb741008a0 2023-08-25T09:09:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb5bfff700 1 -- 172.21.15.99:0/3662478013 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb64022470 con 0x7feb741008a0 2023-08-25T09:09:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb5bfff700 1 -- 172.21.15.99:0/3662478013 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64017650 con 0x7feb741008a0 2023-08-25T09:09:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.069+0000 7feb793ff700 1 -- 172.21.15.99:0/3662478013 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb741a5af0 con 0x7feb741008a0 2023-08-25T09:09:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.073+0000 7feb5bfff700 1 -- 172.21.15.99:0/3662478013 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7feb640225e0 con 0x7feb741008a0 2023-08-25T09:09:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.073+0000 7feb8cd06b80 1 Processor -- start 2023-08-25T09:09:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb8cd06b80 1 -- start start 2023-08-25T09:09:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b96dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37afe750 con 0x55fd37b9bf40 2023-08-25T09:09:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37aef720 con 0x55fd37b9c860 2023-08-25T09:09:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37aed040 con 0x55fd37b96990 2023-08-25T09:09:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb5bfff700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7feb5c0a00c0 0x7feb5c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb737fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb5bfff700 1 -- 172.21.15.99:0/3662478013 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7feb640cc960 con 0x7feb741008a0 2023-08-25T09:09:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb737fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38334/0 (socket says 172.21.15.99:38334) 2023-08-25T09:09:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb727fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b96dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb737fe700 1 -- 172.21.15.99:0/3015150814 learned_addr learned my addr 172.21.15.99:0/3015150814 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.074+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb737fe700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7feb5c0a00c0 0x7feb5c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb727fc700 1 -- 172.21.15.99:0/3015150814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 msgr2=0x55fd37b96450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb727fc700 1 --2- 172.21.15.99:0/3015150814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b96450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb727fc700 1 -- 172.21.15.99:0/3015150814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 msgr2=0x55fd37b9c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb727fc700 1 --2- 172.21.15.99:0/3015150814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb727fc700 1 -- 172.21.15.99:0/3015150814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd379685d0 con 0x55fd37b96990 2023-08-25T09:09:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb72ffd700 1 --2- 172.21.15.99:0/3015150814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b96450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb737fe700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7feb5c0a00c0 0x7feb5c0a2580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00efd0 tx=0x7feb6c00b040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb727fc700 1 --2- 172.21.15.99:0/3015150814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b96dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb68011540 tx=0x7feb68011850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4358905fc9dc94 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.075+0000 7feb597fa700 1 -- 172.21.15.99:0/3015150814 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68004230 con 0x55fd37b96990 2023-08-25T09:09:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb597fa700 1 -- 172.21.15.99:0/3015150814 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb68004390 con 0x55fd37b96990 2023-08-25T09:09:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb597fa700 1 -- 172.21.15.99:0/3015150814 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68019860 con 0x55fd37b96990 2023-08-25T09:09:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 -- 172.21.15.99:0/3015150814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 msgr2=0x55fd37b96dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/3015150814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b96dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb68011540 tx=0x7feb68011850 comp rx=0 tx=0).stop 2023-08-25T09:09:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 -- 172.21.15.99:0/3015150814 shutdown_connections 2023-08-25T09:09:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/3015150814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b96dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/3015150814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/3015150814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b96450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 -- 172.21.15.99:0/3015150814 >> 172.21.15.99:0/3015150814 conn(0x55fd37b0b080 msgr2=0x55fd37b90d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 -- 172.21.15.99:0/3015150814 shutdown_connections 2023-08-25T09:09:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.076+0000 7feb8cd06b80 1 -- 172.21.15.99:0/3015150814 wait complete. 2023-08-25T09:09:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.077+0000 7feb8cd06b80 1 -- start start 2023-08-25T09:09:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.077+0000 7feb8cd06b80 1 Processor -- start 2023-08-25T09:09:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37ba2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b9fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37ae8930 con 0x55fd37b9bf40 2023-08-25T09:09:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37a86b50 con 0x55fd37b9c860 2023-08-25T09:09:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37a86850 con 0x55fd37b96990 2023-08-25T09:09:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb8cd06b80 10 client.0 _ll_register_callbacks cb 0x55fd379d0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37ba2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37ba2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52308/0 (socket says 172.21.15.99:52308) 2023-08-25T09:09:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb72ffd700 1 -- 172.21.15.99:0/4225019802 learned_addr learned my addr 172.21.15.99:0/4225019802 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb737fe700 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b9fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb727fc700 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb72ffd700 1 -- 172.21.15.99:0/4225019802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 msgr2=0x55fd37b9fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb72ffd700 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b9fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb72ffd700 1 -- 172.21.15.99:0/4225019802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 msgr2=0x55fd37b9f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb72ffd700 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.078+0000 7feb72ffd700 1 -- 172.21.15.99:0/4225019802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb741a5af0 con 0x55fd37b96990 2023-08-25T09:09:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb72ffd700 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37ba2670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feb64096aa0 tx=0x7feb64022bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72335d3dcb916b4f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:09:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb56ff5700 1 -- 172.21.15.99:0/4225019802 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640b1070 con 0x55fd37b96990 2023-08-25T09:09:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb8cd06b80 1 -- 172.21.15.99:0/4225019802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd37b66160 con 0x55fd37b96990 2023-08-25T09:09:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb56ff5700 1 -- 172.21.15.99:0/4225019802 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7feb6409a030 con 0x55fd37b96990 2023-08-25T09:09:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb56ff5700 1 -- 172.21.15.99:0/4225019802 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb6409de60 con 0x55fd37b96990 2023-08-25T09:09:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb8cd06b80 10 client.26107 fetch_fsmap learned FSMap version 37 2023-08-25T09:09:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb8cd06b80 1 -- 172.21.15.99:0/4225019802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7feb68009d60 con 0x55fd37b96990 2023-08-25T09:09:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb56ff5700 1 -- 172.21.15.99:0/4225019802 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb64064020 con 0x55fd37b96990 2023-08-25T09:09:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb56ff5700 1 -- 172.21.15.99:0/4225019802 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7feb640a1920 con 0x55fd37b96990 2023-08-25T09:09:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.079+0000 7feb8cd06b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:09:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.080+0000 7feb8cd06b80 10 client.26107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.080+0000 7feb8cd06b80 20 client.26107 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.080+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fd37bafe80 0x55fd37bb2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.080+0000 7feb8cd06b80 4 client.26107 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:09:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.080+0000 7feb8cd06b80 1 -- 172.21.15.99:0/4225019802 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd37a8eb40 con 0x55fd37bafe80 2023-08-25T09:09:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.080+0000 7feb727fc700 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fd37bafe80 0x55fd37bb2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.080+0000 7feb727fc700 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fd37bafe80 0x55fd37bb2270 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd00880767c5f7ba server_cookie=ebec8242b90972db in_seq=0 out_seq=0 2023-08-25T09:09:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.080+0000 7feb56ff5700 10 client.26107 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb56ff5700 1 -- 172.21.15.99:0/4225019802 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd37a8eb40 con 0x55fd37bafe80 2023-08-25T09:09:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb56ff5700 10 client.26107 handle_command_reply: tid=0 2023-08-25T09:09:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 client.26107 shutdown 2023-08-25T09:09:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 -- 172.21.15.99:0/4225019802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 msgr2=0x55fd37ba2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37ba2670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feb64096aa0 tx=0x7feb64022bc0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 -- 172.21.15.99:0/4225019802 shutdown_connections 2023-08-25T09:09:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fd37bafe80 0x55fd37bb2270 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37b9fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37ba2670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb56ff5700 0 client.26107 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/4225019802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9bf40 0x55fd37b9f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.081+0000 7feb8cd06b80 1 -- 172.21.15.99:0/4225019802 >> 172.21.15.99:0/4225019802 conn(0x55fd37b0b080 msgr2=0x55fd37b8b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.082+0000 7feb8cd06b80 1 -- 172.21.15.99:0/4225019802 shutdown_connections 2023-08-25T09:09:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.082+0000 7feb8cd06b80 1 -- 172.21.15.99:0/4225019802 wait complete. 2023-08-25T09:09:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.082+0000 7feb8cd06b80 20 client.26107 trim_cache size 0 max 16384 2023-08-25T09:09:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.096+0000 7feb8cd06b80 1 Processor -- start 2023-08-25T09:09:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.096+0000 7feb8cd06b80 1 -- start start 2023-08-25T09:09:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b96990 0x55fd37b96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b9bf40 0x55fd37ba72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37ba21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3796a8c0 con 0x55fd37b96990 2023-08-25T09:09:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37b91c30 con 0x55fd37b9c860 2023-08-25T09:09:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37b9c6b0 con 0x55fd37b9bf40 2023-08-25T09:09:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b96990 0x55fd37b96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b96990 0x55fd37b96450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38342/0 (socket says 172.21.15.99:38342) 2023-08-25T09:09:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb72ffd700 1 -- 172.21.15.99:0/1033730424 learned_addr learned my addr 172.21.15.99:0/1033730424 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb727fc700 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b9bf40 0x55fd37ba72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb737fe700 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37ba21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb72ffd700 1 -- 172.21.15.99:0/1033730424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b9bf40 msgr2=0x55fd37ba72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb72ffd700 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b9bf40 0x55fd37ba72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb72ffd700 1 -- 172.21.15.99:0/1033730424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 msgr2=0x55fd37ba21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb72ffd700 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37ba21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.097+0000 7feb72ffd700 1 -- 172.21.15.99:0/1033730424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd37914d80 con 0x55fd37b96990 2023-08-25T09:09:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb72ffd700 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b96990 0x55fd37b96450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feb64098380 tx=0x7feb64099ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0e215e372c20f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb59ffb700 1 -- 172.21.15.99:0/1033730424 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640b1040 con 0x55fd37b96990 2023-08-25T09:09:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb59ffb700 1 -- 172.21.15.99:0/1033730424 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb64064030 con 0x55fd37b96990 2023-08-25T09:09:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb59ffb700 1 -- 172.21.15.99:0/1033730424 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640a1920 con 0x55fd37b96990 2023-08-25T09:09:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1033730424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b96990 msgr2=0x55fd37b96450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b96990 0x55fd37b96450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feb64098380 tx=0x7feb64099ca0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1033730424 shutdown_connections 2023-08-25T09:09:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9c860 0x55fd37ba21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b96990 0x55fd37b96450 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1033730424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b9bf40 0x55fd37ba72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.098+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1033730424 >> 172.21.15.99:0/1033730424 conn(0x55fd37b0b080 msgr2=0x55fd37b90bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.099+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1033730424 shutdown_connections 2023-08-25T09:09:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.099+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1033730424 wait complete. 2023-08-25T09:09:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.099+0000 7feb8cd06b80 1 -- start start 2023-08-25T09:09:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.099+0000 7feb8cd06b80 1 Processor -- start 2023-08-25T09:09:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.099+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b98e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9bf40 0x55fd37b97880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb8cd06b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9c860 0x55fd37b97dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37b994e0 con 0x55fd37b9c860 2023-08-25T09:09:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37b98300 con 0x55fd37b9bf40 2023-08-25T09:09:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb8cd06b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd37b984c0 con 0x55fd37b96990 2023-08-25T09:09:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb8cd06b80 10 client.0 _ll_register_callbacks cb 0x7feb64096460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb737fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9c860 0x55fd37b97dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b98e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb727fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9bf40 0x55fd37b97880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b98e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:52334/0 (socket says 172.21.15.99:52334) 2023-08-25T09:09:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 -- 172.21.15.99:0/1145199534 learned_addr learned my addr 172.21.15.99:0/1145199534 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb737fe700 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9c860 0x55fd37b97dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38344/0 (socket says 172.21.15.99:38344) 2023-08-25T09:09:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 -- 172.21.15.99:0/1145199534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9bf40 msgr2=0x55fd37b97880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9bf40 0x55fd37b97880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 -- 172.21.15.99:0/1145199534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9c860 msgr2=0x55fd37b97dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9c860 0x55fd37b97dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 -- 172.21.15.99:0/1145199534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb68009d60 con 0x55fd37b96990 2023-08-25T09:09:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb737fe700 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9c860 0x55fd37b97dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb727fc700 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9bf40 0x55fd37b97880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb72ffd700 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b98e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb640ca5e0 tx=0x7feb64095cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7868129811966a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.100+0000 7feb56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:09:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb56ff5700 1 -- 172.21.15.99:0/1145199534 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640b1040 con 0x55fd37b96990 2023-08-25T09:09:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1145199534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd37b89600 con 0x55fd37b96990 2023-08-25T09:09:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb56ff5700 1 -- 172.21.15.99:0/1145199534 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7feb64065080 con 0x55fd37b96990 2023-08-25T09:09:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb56ff5700 1 -- 172.21.15.99:0/1145199534 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb6409a030 con 0x55fd37b96990 2023-08-25T09:09:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 10 client.26113 fetch_fsmap learned FSMap version 37 2023-08-25T09:09:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1145199534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55fd378fdc00 con 0x55fd37b96990 2023-08-25T09:09:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb56ff5700 1 -- 172.21.15.99:0/1145199534 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd37b89600 con 0x55fd37b96990 2023-08-25T09:09:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb56ff5700 1 -- 172.21.15.99:0/1145199534 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7feb64096000 con 0x55fd37b96990 2023-08-25T09:09:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 10 client.26113 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:09:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 10 client.26113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 20 client.26113 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fd37c04b10 0x55fd37c06f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 4 client.26113 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:09:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.101+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1145199534 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd37ae1910 con 0x55fd37c04b10 2023-08-25T09:09:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.102+0000 7feb727fc700 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fd37c04b10 0x55fd37c06f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.102+0000 7feb56ff5700 10 client.26113 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.102+0000 7feb727fc700 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fd37c04b10 0x55fd37c06f30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906ba7b1d5f0ef23 server_cookie=acb4e8877f4b4c03 in_seq=0 out_seq=0 2023-08-25T09:09:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb56ff5700 1 -- 172.21.15.99:0/1145199534 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd37ae1910 con 0x55fd37c04b10 2023-08-25T09:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb56ff5700 10 client.26113 handle_command_reply: tid=0 2023-08-25T09:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 client.26113 shutdown 2023-08-25T09:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1145199534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 msgr2=0x55fd37b98e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b98e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb640ca5e0 tx=0x7feb64095cd0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1145199534 shutdown_connections 2023-08-25T09:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd37b9c860 0x55fd37b97dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd37b96990 0x55fd37b98e50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fd37c04b10 0x55fd37c06f30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 --2- 172.21.15.99:0/1145199534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd37b9bf40 0x55fd37b97880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb56ff5700 0 client.26113 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1145199534 >> 172.21.15.99:0/1145199534 conn(0x55fd37b0b080 msgr2=0x55fd37b99990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1145199534 shutdown_connections 2023-08-25T09:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.141+0000 7feb8cd06b80 1 -- 172.21.15.99:0/1145199534 wait complete. 2023-08-25T09:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb8cd06b80 20 client.26113 trim_cache size 0 max 16384 2023-08-25T09:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:09:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:09:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 -- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7feb5c0a00c0 msgr2=0x7feb5c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7feb5c0a00c0 0x7feb5c0a2580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00efd0 tx=0x7feb6c00b040 comp rx=0 tx=0).stop 2023-08-25T09:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 -- 172.21.15.99:0/3662478013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb741008a0 msgr2=0x7feb7419eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb741008a0 0x7feb7419eb00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feb64000c00 tx=0x7feb64009580 comp rx=0 tx=0).stop 2023-08-25T09:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 -- 172.21.15.99:0/3662478013 shutdown_connections 2023-08-25T09:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7feb5c0a00c0 0x7feb5c0a2580 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb741068b0 0x7feb74198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb741008a0 0x7feb7419eb00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 --2- 172.21.15.99:0/3662478013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb74101260 0x7feb7419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 -- 172.21.15.99:0/3662478013 >> 172.21.15.99:0/3662478013 conn(0x7feb740fc370 msgr2=0x7feb740fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 -- 172.21.15.99:0/3662478013 shutdown_connections 2023-08-25T09:09:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:42.142+0000 7feb793ff700 1 -- 172.21.15.99:0/3662478013 wait complete. 2023-08-25T09:09:42.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416109 4.01 MB/sec execute 468 sec latency 1668.520 ms 2023-08-25T09:09:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:43 smithi181 ceph-mon[131296]: pgmap v327: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2023-08-25T09:09:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:43 smithi099 ceph-mon[97641]: pgmap v327: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2023-08-25T09:09:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:43 smithi157 ceph-mon[130879]: pgmap v327: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2023-08-25T09:09:43.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416109 4.00 MB/sec execute 469 sec latency 2668.679 ms 2023-08-25T09:09:44.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416732 4.00 MB/sec execute 470 sec latency 3209.835 ms 2023-08-25T09:09:45.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:45 smithi181 ceph-mon[131296]: pgmap v328: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 884 KiB/s wr, 66 op/s 2023-08-25T09:09:45.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:45 smithi099 ceph-mon[97641]: pgmap v328: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 884 KiB/s wr, 66 op/s 2023-08-25T09:09:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:45 smithi157 ceph-mon[130879]: pgmap v328: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 884 KiB/s wr, 66 op/s 2023-08-25T09:09:45.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416732 3.99 MB/sec execute 471 sec latency 1031.961 ms 2023-08-25T09:09:46.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416732 3.98 MB/sec execute 472 sec latency 2032.099 ms 2023-08-25T09:09:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:47 smithi181 ceph-mon[131296]: pgmap v329: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2023-08-25T09:09:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:47 smithi099 ceph-mon[97641]: pgmap v329: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2023-08-25T09:09:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:47 smithi157 ceph-mon[130879]: pgmap v329: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2023-08-25T09:09:47.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416732 3.97 MB/sec execute 473 sec latency 3032.214 ms 2023-08-25T09:09:48.942 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416732 3.96 MB/sec execute 474 sec latency 4032.352 ms 2023-08-25T09:09:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:49 smithi181 ceph-mon[131296]: pgmap v330: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 850 KiB/s wr, 74 op/s 2023-08-25T09:09:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:49 smithi099 ceph-mon[97641]: pgmap v330: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 850 KiB/s wr, 74 op/s 2023-08-25T09:09:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:49 smithi157 ceph-mon[130879]: pgmap v330: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 850 KiB/s wr, 74 op/s 2023-08-25T09:09:49.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 417807 3.97 MB/sec execute 475 sec latency 4567.895 ms 2023-08-25T09:09:50.941 INFO:tasks.workunit.client.0.smithi099.stdout: 1 419203 3.98 MB/sec execute 476 sec latency 12.550 ms 2023-08-25T09:09:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:51 smithi181 ceph-mon[131296]: pgmap v331: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 850 KiB/s wr, 74 op/s 2023-08-25T09:09:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:51 smithi099 ceph-mon[97641]: pgmap v331: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 850 KiB/s wr, 74 op/s 2023-08-25T09:09:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:51 smithi157 ceph-mon[130879]: pgmap v331: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 850 KiB/s wr, 74 op/s 2023-08-25T09:09:51.942 INFO:tasks.workunit.client.0.smithi099.stdout: 1 419652 3.97 MB/sec execute 477 sec latency 706.558 ms 2023-08-25T09:09:52.942 INFO:tasks.workunit.client.0.smithi099.stdout: 1 419652 3.96 MB/sec execute 478 sec latency 1706.725 ms 2023-08-25T09:09:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:53 smithi181 ceph-mon[131296]: pgmap v332: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2023-08-25T09:09:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:53 smithi099 ceph-mon[97641]: pgmap v332: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2023-08-25T09:09:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:53 smithi157 ceph-mon[130879]: pgmap v332: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2023-08-25T09:09:53.942 INFO:tasks.workunit.client.0.smithi099.stdout: 1 419652 3.96 MB/sec execute 479 sec latency 2706.898 ms 2023-08-25T09:09:54.942 INFO:tasks.workunit.client.0.smithi099.stdout: 1 420275 3.95 MB/sec execute 480 sec latency 3245.837 ms 2023-08-25T09:09:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 -- 172.21.15.99:0/1861337566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c1008a0 msgr2=0x7fd52c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 --2- 172.21.15.99:0/1861337566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c1008a0 0x7fd52c100d20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd518009990 tx=0x7fd518009ca0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 -- 172.21.15.99:0/1861337566 shutdown_connections 2023-08-25T09:09:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 --2- 172.21.15.99:0/1861337566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c101260 0x7fd52c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 --2- 172.21.15.99:0/1861337566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52c1068b0 0x7fd52c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 --2- 172.21.15.99:0/1861337566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c1008a0 0x7fd52c100d20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 -- 172.21.15.99:0/1861337566 >> 172.21.15.99:0/1861337566 conn(0x7fd52c0fc370 msgr2=0x7fd52c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 -- 172.21.15.99:0/1861337566 shutdown_connections 2023-08-25T09:09:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.299+0000 7fd532dff700 1 -- 172.21.15.99:0/1861337566 wait complete. 2023-08-25T09:09:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.300+0000 7fd532dff700 1 Processor -- start 2023-08-25T09:09:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.301+0000 7fd532dff700 1 -- start start 2023-08-25T09:09:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.301+0000 7fd532dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c1008a0 0x7fd52c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.301+0000 7fd532dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101260 0x7fd52c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.301+0000 7fd532dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52c1068b0 0x7fd52c1967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.301+0000 7fd532dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c196e20 con 0x7fd52c101260 2023-08-25T09:09:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.301+0000 7fd532dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c196f90 con 0x7fd52c1008a0 2023-08-25T09:09:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.301+0000 7fd532dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c197150 con 0x7fd52c1068b0 2023-08-25T09:09:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.301+0000 7fd52bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101260 0x7fd52c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd530b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c1008a0 0x7fd52c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd52bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101260 0x7fd52c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48444/0 (socket says 172.21.15.99:48444) 2023-08-25T09:09:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd52bfff700 1 -- 172.21.15.99:0/2060489903 learned_addr learned my addr 172.21.15.99:0/2060489903 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd53139c700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52c1068b0 0x7fd52c1967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd52bfff700 1 -- 172.21.15.99:0/2060489903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52c1068b0 msgr2=0x7fd52c1967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd52bfff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52c1068b0 0x7fd52c1967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd52bfff700 1 -- 172.21.15.99:0/2060489903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c1008a0 msgr2=0x7fd52c19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd52bfff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c1008a0 0x7fd52c19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd52bfff700 1 -- 172.21.15.99:0/2060489903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180095f0 con 0x7fd52c101260 2023-08-25T09:09:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd52bfff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101260 0x7fd52c19ce90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd518005910 tx=0x7fd518005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd529ffb700 1 -- 172.21.15.99:0/2060489903 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51801d070 con 0x7fd52c101260 2023-08-25T09:09:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd529ffb700 1 -- 172.21.15.99:0/2060489903 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd518022470 con 0x7fd52c101260 2023-08-25T09:09:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd532dff700 1 -- 172.21.15.99:0/2060489903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c1973d0 con 0x7fd52c101260 2023-08-25T09:09:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.302+0000 7fd529ffb700 1 -- 172.21.15.99:0/2060489903 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518017650 con 0x7fd52c101260 2023-08-25T09:09:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.303+0000 7fd532dff700 1 -- 172.21.15.99:0/2060489903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c1a3940 con 0x7fd52c101260 2023-08-25T09:09:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.306+0000 7fd529ffb700 1 -- 172.21.15.99:0/2060489903 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fd5180225e0 con 0x7fd52c101260 2023-08-25T09:09:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.307+0000 7fd5467d4b80 1 Processor -- start 2023-08-25T09:09:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.307+0000 7fd5467d4b80 1 -- start start 2023-08-25T09:09:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.307+0000 7fd529ffb700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd51c0a00c0 0x7fd51c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.307+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b72320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.307+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b72860 0x560b31b6c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.307+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.307+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31ad4750 con 0x560b31b6c990 2023-08-25T09:09:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.307+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31ac5720 con 0x560b31b72860 2023-08-25T09:09:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31ac3040 con 0x560b31b71f40 2023-08-25T09:09:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd529ffb700 1 -- 172.21.15.99:0/2060489903 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd5180cc970 con 0x7fd52c101260 2023-08-25T09:09:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b72320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b72320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34510/0 (socket says 172.21.15.99:34510) 2023-08-25T09:09:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 -- 172.21.15.99:0/1819293402 learned_addr learned my addr 172.21.15.99:0/1819293402 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd53139c700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd51c0a00c0 0x7fd51c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd52bfff700 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b72860 0x560b31b6c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd53139c700 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 -- 172.21.15.99:0/1819293402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b72860 msgr2=0x560b31b6c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b72860 0x560b31b6c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 -- 172.21.15.99:0/1819293402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 msgr2=0x560b31b6cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 -- 172.21.15.99:0/1819293402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b3193e5d0 con 0x560b31b71f40 2023-08-25T09:09:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd53139c700 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:09:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.308+0000 7fd530b9b700 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b72320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd520009d00 tx=0x7fd520011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73caf4c353ec8a08 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.309+0000 7fd53139c700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd51c0a00c0 0x7fd51c0a2580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd524009ec0 tx=0x7fd524009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.309+0000 7fd516ffd700 1 -- 172.21.15.99:0/1819293402 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520004230 con 0x560b31b71f40 2023-08-25T09:09:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.309+0000 7fd516ffd700 1 -- 172.21.15.99:0/1819293402 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd520004390 con 0x560b31b71f40 2023-08-25T09:09:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.309+0000 7fd516ffd700 1 -- 172.21.15.99:0/1819293402 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520019880 con 0x560b31b71f40 2023-08-25T09:09:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.309+0000 7fd5467d4b80 1 -- 172.21.15.99:0/1819293402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 msgr2=0x560b31b72320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.309+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b72320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd520009d00 tx=0x7fd520011720 comp rx=0 tx=0).stop 2023-08-25T09:09:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.310+0000 7fd5467d4b80 1 -- 172.21.15.99:0/1819293402 shutdown_connections 2023-08-25T09:09:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.310+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.310+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b72320 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.310+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/1819293402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b72860 0x560b31b6c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.310+0000 7fd5467d4b80 1 -- 172.21.15.99:0/1819293402 >> 172.21.15.99:0/1819293402 conn(0x560b31ae1080 msgr2=0x560b31b66d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.310+0000 7fd5467d4b80 1 -- 172.21.15.99:0/1819293402 shutdown_connections 2023-08-25T09:09:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.310+0000 7fd5467d4b80 1 -- 172.21.15.99:0/1819293402 wait complete. 2023-08-25T09:09:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.310+0000 7fd5467d4b80 1 -- start start 2023-08-25T09:09:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.311+0000 7fd5467d4b80 1 Processor -- start 2023-08-25T09:09:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.311+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b78680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.311+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b75890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.311+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 0x560b31b75ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.311+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31abe930 con 0x560b31b72860 2023-08-25T09:09:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.311+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31a5cb50 con 0x560b31b6c990 2023-08-25T09:09:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.311+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31a5c850 con 0x560b31b71f40 2023-08-25T09:09:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.311+0000 7fd5467d4b80 10 client.0 _ll_register_callbacks cb 0x560b319a6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b75890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd530b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b78680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b75890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34522/0 (socket says 172.21.15.99:34522) 2023-08-25T09:09:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 -- 172.21.15.99:0/3498516290 learned_addr learned my addr 172.21.15.99:0/3498516290 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd53139c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 0x560b31b75ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 -- 172.21.15.99:0/3498516290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 msgr2=0x560b31b78680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b78680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 -- 172.21.15.99:0/3498516290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 msgr2=0x560b31b75ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 0x560b31b75ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 -- 172.21.15.99:0/3498516290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd520009960 con 0x560b31b71f40 2023-08-25T09:09:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd530b9b700 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b78680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:09:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd52bfff700 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b75890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd51809a030 tx=0x7fd518099ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6977ea0ed9a59bc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd50bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:09:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd50bfff700 1 -- 172.21.15.99:0/3498516290 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51809f040 con 0x560b31b71f40 2023-08-25T09:09:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd50bfff700 1 -- 172.21.15.99:0/3498516290 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd51809a7a0 con 0x560b31b71f40 2023-08-25T09:09:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.312+0000 7fd5467d4b80 1 -- 172.21.15.99:0/3498516290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b31b3c160 con 0x560b31b71f40 2023-08-25T09:09:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd50bfff700 1 -- 172.21.15.99:0/3498516290 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd51809a0a0 con 0x560b31b71f40 2023-08-25T09:09:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd5467d4b80 10 client.26125 fetch_fsmap learned FSMap version 37 2023-08-25T09:09:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd5467d4b80 1 -- 172.21.15.99:0/3498516290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x560b318ead80 con 0x560b31b71f40 2023-08-25T09:09:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd50bfff700 1 -- 172.21.15.99:0/3498516290 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b31b3c160 con 0x560b31b71f40 2023-08-25T09:09:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd50bfff700 1 -- 172.21.15.99:0/3498516290 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd51808ca60 con 0x560b31b71f40 2023-08-25T09:09:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd5467d4b80 10 client.26125 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:09:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd5467d4b80 10 client.26125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd5467d4b80 20 client.26125 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560b31b85df0 0x560b31b88210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd5467d4b80 4 client.26125 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:09:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd5467d4b80 1 -- 172.21.15.99:0/3498516290 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b31a64b40 con 0x560b31b85df0 2023-08-25T09:09:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.313+0000 7fd530b9b700 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560b31b85df0 0x560b31b88210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.314+0000 7fd50bfff700 10 client.26125 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.314+0000 7fd530b9b700 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560b31b85df0 0x560b31b88210 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48fe6fa74cb00cbe server_cookie=3fceda83cb9d7ae8 in_seq=0 out_seq=0 2023-08-25T09:09:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.314+0000 7fd50bfff700 1 -- 172.21.15.99:0/3498516290 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560b31a64b40 con 0x560b31b85df0 2023-08-25T09:09:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.314+0000 7fd50bfff700 10 client.26125 handle_command_reply: tid=0 2023-08-25T09:09:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 client.26125 shutdown 2023-08-25T09:09:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 -- 172.21.15.99:0/3498516290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 msgr2=0x560b31b75890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b75890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd51809a030 tx=0x7fd518099ef0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 -- 172.21.15.99:0/3498516290 shutdown_connections 2023-08-25T09:09:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 0x560b31b75ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560b31b85df0 0x560b31b88210 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b78680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/3498516290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b75890 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd50bfff700 0 client.26125 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 -- 172.21.15.99:0/3498516290 >> 172.21.15.99:0/3498516290 conn(0x560b31ae1080 msgr2=0x560b31b61820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 -- 172.21.15.99:0/3498516290 shutdown_connections 2023-08-25T09:09:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 1 -- 172.21.15.99:0/3498516290 wait complete. 2023-08-25T09:09:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.315+0000 7fd5467d4b80 20 client.26125 trim_cache size 0 max 16384 2023-08-25T09:09:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.330+0000 7fd5467d4b80 1 Processor -- start 2023-08-25T09:09:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.330+0000 7fd5467d4b80 1 -- start start 2023-08-25T09:09:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b71f40 0x560b31b7d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b72860 0x560b31b781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b319408c0 con 0x560b31b6c990 2023-08-25T09:09:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31b68770 con 0x560b31b71f40 2023-08-25T09:09:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31b67250 con 0x560b31b72860 2023-08-25T09:09:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd530b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd53139c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b72860 0x560b31b781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd53139c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b72860 0x560b31b781b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34526/0 (socket says 172.21.15.99:34526) 2023-08-25T09:09:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd53139c700 1 -- 172.21.15.99:0/2525317226 learned_addr learned my addr 172.21.15.99:0/2525317226 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd530b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48456/0 (socket says 172.21.15.99:48456) 2023-08-25T09:09:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd52bfff700 1 --2- 172.21.15.99:0/2525317226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b71f40 0x560b31b7d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd530b9b700 1 -- 172.21.15.99:0/2525317226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b72860 msgr2=0x560b31b781b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd530b9b700 1 --2- 172.21.15.99:0/2525317226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b72860 0x560b31b781b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd530b9b700 1 -- 172.21.15.99:0/2525317226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b71f40 msgr2=0x560b31b7d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd530b9b700 1 --2- 172.21.15.99:0/2525317226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b71f40 0x560b31b7d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.331+0000 7fd530b9b700 1 -- 172.21.15.99:0/2525317226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b31b6d040 con 0x560b31b6c990 2023-08-25T09:09:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd530b9b700 1 --2- 172.21.15.99:0/2525317226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6c450 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd520011a70 tx=0x7fd520005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bceb720d1793de7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5177fe700 1 -- 172.21.15.99:0/2525317226 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520003b70 con 0x560b31b6c990 2023-08-25T09:09:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5177fe700 1 -- 172.21.15.99:0/2525317226 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd520003d10 con 0x560b31b6c990 2023-08-25T09:09:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5177fe700 1 -- 172.21.15.99:0/2525317226 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5200198f0 con 0x560b31b6c990 2023-08-25T09:09:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5467d4b80 1 -- 172.21.15.99:0/2525317226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 msgr2=0x560b31b6c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/2525317226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6c450 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd520011a70 tx=0x7fd520005e00 comp rx=0 tx=0).stop 2023-08-25T09:09:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5467d4b80 1 -- 172.21.15.99:0/2525317226 shutdown_connections 2023-08-25T09:09:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/2525317226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b72860 0x560b31b781b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/2525317226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b6c990 0x560b31b6c450 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/2525317226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b71f40 0x560b31b7d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.332+0000 7fd5467d4b80 1 -- 172.21.15.99:0/2525317226 >> 172.21.15.99:0/2525317226 conn(0x560b31ae1080 msgr2=0x560b31b6fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.333+0000 7fd5467d4b80 1 -- 172.21.15.99:0/2525317226 shutdown_connections 2023-08-25T09:09:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.333+0000 7fd5467d4b80 1 -- 172.21.15.99:0/2525317226 wait complete. 2023-08-25T09:09:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.333+0000 7fd5467d4b80 1 -- start start 2023-08-25T09:09:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.333+0000 7fd5467d4b80 1 Processor -- start 2023-08-25T09:09:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b6d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd5467d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 0x560b31b6e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31b6e840 con 0x560b31b72860 2023-08-25T09:09:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31b6e9b0 con 0x560b31b6c990 2023-08-25T09:09:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd5467d4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31b6eb70 con 0x560b31b71f40 2023-08-25T09:09:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd5467d4b80 10 client.0 _ll_register_callbacks cb 0x7fd5180963e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:09:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd52bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd53139c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 0x560b31b6e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd52bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b6ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34542/0 (socket says 172.21.15.99:34542) 2023-08-25T09:09:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd52bfff700 1 -- 172.21.15.99:0/4143744878 learned_addr learned my addr 172.21.15.99:0/4143744878 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:09:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.334+0000 7fd530b9b700 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b6d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd52bfff700 1 -- 172.21.15.99:0/4143744878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 msgr2=0x560b31b6d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd52bfff700 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b6d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd52bfff700 1 -- 172.21.15.99:0/4143744878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 msgr2=0x560b31b6e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:09:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd52bfff700 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 0x560b31b6e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd52bfff700 1 -- 172.21.15.99:0/4143744878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b3193e5d0 con 0x560b31b71f40 2023-08-25T09:09:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd52bfff700 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b6ddc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd518096b10 tx=0x7fd518022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5fb5720d872b14 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:09:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd50bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:09:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd50bfff700 1 -- 172.21.15.99:0/4143744878 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51809f070 con 0x560b31b71f40 2023-08-25T09:09:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd50bfff700 1 -- 172.21.15.99:0/4143744878 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd51809a7a0 con 0x560b31b71f40 2023-08-25T09:09:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd5467d4b80 1 -- 172.21.15.99:0/4143744878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b31b5f600 con 0x560b31b71f40 2023-08-25T09:09:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd50bfff700 1 -- 172.21.15.99:0/4143744878 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd518099be0 con 0x560b31b71f40 2023-08-25T09:09:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd5467d4b80 10 client.26131 fetch_fsmap learned FSMap version 37 2023-08-25T09:09:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd5467d4b80 1 -- 172.21.15.99:0/4143744878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x560b318d3c00 con 0x560b31b71f40 2023-08-25T09:09:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.335+0000 7fd50bfff700 1 -- 172.21.15.99:0/4143744878 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b31b5f600 con 0x560b31b71f40 2023-08-25T09:09:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd50bfff700 1 -- 172.21.15.99:0/4143744878 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd51809a300 con 0x560b31b71f40 2023-08-25T09:09:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd5467d4b80 10 client.26131 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:09:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd5467d4b80 10 client.26131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:09:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd5467d4b80 20 client.26131 populate_metadata read hostname 'smithi099' 2023-08-25T09:09:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560b31bda270 0x560b31bdc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:09:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd5467d4b80 4 client.26131 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:09:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd5467d4b80 1 -- 172.21.15.99:0/4143744878 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560b31ab7910 con 0x560b31bda270 2023-08-25T09:09:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd530b9b700 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560b31bda270 0x560b31bdc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:09:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd530b9b700 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560b31bda270 0x560b31bdc660 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bf62dbf0e935b23 server_cookie=d6724a80f92cf8bf in_seq=0 out_seq=0 2023-08-25T09:09:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.336+0000 7fd50bfff700 10 client.26131 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd50bfff700 1 -- 172.21.15.99:0/4143744878 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560b31ab7910 con 0x560b31bda270 2023-08-25T09:09:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd50bfff700 10 client.26131 handle_command_reply: tid=0 2023-08-25T09:09:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 client.26131 shutdown 2023-08-25T09:09:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 -- 172.21.15.99:0/4143744878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 msgr2=0x560b31b6ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b6ddc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd518096b10 tx=0x7fd518022af0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 -- 172.21.15.99:0/4143744878 shutdown_connections 2023-08-25T09:09:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b31b72860 0x560b31b6e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560b31bda270 0x560b31bdc660 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b31b6c990 0x560b31b6d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 --2- 172.21.15.99:0/4143744878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b31b71f40 0x560b31b6ddc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd50bfff700 0 client.26131 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:09:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 -- 172.21.15.99:0/4143744878 >> 172.21.15.99:0/4143744878 conn(0x560b31ae1080 msgr2=0x560b31b72c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.366+0000 7fd5467d4b80 1 -- 172.21.15.99:0/4143744878 shutdown_connections 2023-08-25T09:09:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd5467d4b80 1 -- 172.21.15.99:0/4143744878 wait complete. 2023-08-25T09:09:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd5467d4b80 20 client.26131 trim_cache size 0 max 16384 2023-08-25T09:09:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:09:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:09:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:09:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:09:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 -- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd51c0a00c0 msgr2=0x7fd51c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd51c0a00c0 0x7fd51c0a2580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd524009ec0 tx=0x7fd524009920 comp rx=0 tx=0).stop 2023-08-25T09:09:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 -- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101260 msgr2=0x7fd52c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:09:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101260 0x7fd52c19ce90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd518005910 tx=0x7fd518005720 comp rx=0 tx=0).stop 2023-08-25T09:09:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 -- 172.21.15.99:0/2060489903 shutdown_connections 2023-08-25T09:09:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd51c0a00c0 0x7fd51c0a2580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52c1068b0 0x7fd52c1967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c1008a0 0x7fd52c19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 --2- 172.21.15.99:0/2060489903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101260 0x7fd52c19ce90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:09:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.367+0000 7fd532dff700 1 -- 172.21.15.99:0/2060489903 >> 172.21.15.99:0/2060489903 conn(0x7fd52c0fc370 msgr2=0x7fd52c0fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:09:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.368+0000 7fd532dff700 1 -- 172.21.15.99:0/2060489903 shutdown_connections 2023-08-25T09:09:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:09:55.368+0000 7fd532dff700 1 -- 172.21.15.99:0/2060489903 wait complete. 2023-08-25T09:09:55.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:55 smithi181 ceph-mon[131296]: pgmap v333: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 641 KiB/s wr, 68 op/s 2023-08-25T09:09:55.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:55 smithi099 ceph-mon[97641]: pgmap v333: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 641 KiB/s wr, 68 op/s 2023-08-25T09:09:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:55 smithi157 ceph-mon[130879]: pgmap v333: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 641 KiB/s wr, 68 op/s 2023-08-25T09:09:55.942 INFO:tasks.workunit.client.0.smithi099.stdout: 1 420275 3.95 MB/sec execute 481 sec latency 1039.597 ms 2023-08-25T09:09:56.942 INFO:tasks.workunit.client.0.smithi099.stdout: 1 420275 3.94 MB/sec execute 482 sec latency 2039.703 ms 2023-08-25T09:09:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:57 smithi181 ceph-mon[131296]: pgmap v334: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-08-25T09:09:57.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:57 smithi099 ceph-mon[97641]: pgmap v334: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-08-25T09:09:57.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:57 smithi157 ceph-mon[130879]: pgmap v334: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-08-25T09:09:57.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 420275 3.93 MB/sec execute 483 sec latency 3039.791 ms 2023-08-25T09:09:58.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 420275 3.92 MB/sec execute 484 sec latency 4039.932 ms 2023-08-25T09:09:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:09:59 smithi181 ceph-mon[131296]: pgmap v335: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 710 KiB/s wr, 65 op/s 2023-08-25T09:09:59.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:09:59 smithi099 ceph-mon[97641]: pgmap v335: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 710 KiB/s wr, 65 op/s 2023-08-25T09:09:59.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:09:59 smithi157 ceph-mon[130879]: pgmap v335: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 710 KiB/s wr, 65 op/s 2023-08-25T09:09:59.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 421357 3.93 MB/sec execute 485 sec latency 4576.434 ms 2023-08-25T09:10:00.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:00 smithi181 ceph-mon[131296]: overall HEALTH_OK 2023-08-25T09:10:00.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:00 smithi099 ceph-mon[97641]: overall HEALTH_OK 2023-08-25T09:10:00.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:00 smithi157 ceph-mon[130879]: overall HEALTH_OK 2023-08-25T09:10:00.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 422486 3.93 MB/sec execute 486 sec latency 32.058 ms 2023-08-25T09:10:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:01 smithi181 ceph-mon[131296]: pgmap v336: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 710 KiB/s wr, 65 op/s 2023-08-25T09:10:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:01 smithi099 ceph-mon[97641]: pgmap v336: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 710 KiB/s wr, 65 op/s 2023-08-25T09:10:01.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:01 smithi157 ceph-mon[130879]: pgmap v336: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 710 KiB/s wr, 65 op/s 2023-08-25T09:10:01.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 423638 3.94 MB/sec execute 487 sec latency 19.607 ms 2023-08-25T09:10:02.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 423824 3.93 MB/sec execute 488 sec latency 751.829 ms 2023-08-25T09:10:03.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:03 smithi181 ceph-mon[131296]: pgmap v337: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2023-08-25T09:10:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:03 smithi099 ceph-mon[97641]: pgmap v337: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2023-08-25T09:10:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:03 smithi157 ceph-mon[130879]: pgmap v337: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2023-08-25T09:10:03.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 423824 3.92 MB/sec execute 489 sec latency 1751.875 ms 2023-08-25T09:10:04.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:04 smithi181 ceph-mon[131296]: osdmap e97: 12 total, 12 up, 12 in 2023-08-25T09:10:04.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:04 smithi099 ceph-mon[97641]: osdmap e97: 12 total, 12 up, 12 in 2023-08-25T09:10:04.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:04 smithi157 ceph-mon[130879]: osdmap e97: 12 total, 12 up, 12 in 2023-08-25T09:10:04.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 425152 3.93 MB/sec execute 490 sec latency 2169.245 ms 2023-08-25T09:10:05.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:05 smithi181 ceph-mon[131296]: pgmap v339: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 868 KiB/s wr, 108 op/s 2023-08-25T09:10:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:05 smithi099 ceph-mon[97641]: pgmap v339: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 868 KiB/s wr, 108 op/s 2023-08-25T09:10:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:05 smithi157 ceph-mon[130879]: pgmap v339: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 868 KiB/s wr, 108 op/s 2023-08-25T09:10:05.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426463 3.93 MB/sec execute 491 sec latency 22.865 ms 2023-08-25T09:10:06.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426742 3.93 MB/sec execute 492 sec latency 834.220 ms 2023-08-25T09:10:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:07 smithi181 ceph-mon[131296]: pgmap v340: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-08-25T09:10:07.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:07 smithi099 ceph-mon[97641]: pgmap v340: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-08-25T09:10:07.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:07 smithi157 ceph-mon[130879]: pgmap v340: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-08-25T09:10:07.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426742 3.92 MB/sec execute 493 sec latency 1834.281 ms 2023-08-25T09:10:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 -- 172.21.15.99:0/930806949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97681008a0 msgr2=0x7f9768100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 --2- 172.21.15.99:0/930806949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97681008a0 0x7f9768100d20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9758009990 tx=0x7f9758009ca0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 -- 172.21.15.99:0/930806949 shutdown_connections 2023-08-25T09:10:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 --2- 172.21.15.99:0/930806949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768101260 0x7f976810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 --2- 172.21.15.99:0/930806949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97681068b0 0x7f9768106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 --2- 172.21.15.99:0/930806949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97681008a0 0x7f9768100d20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 -- 172.21.15.99:0/930806949 >> 172.21.15.99:0/930806949 conn(0x7f97680fc370 msgr2=0x7f97680fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 -- 172.21.15.99:0/930806949 shutdown_connections 2023-08-25T09:10:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.521+0000 7f976ffff700 1 -- 172.21.15.99:0/930806949 wait complete. 2023-08-25T09:10:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.522+0000 7f976ffff700 1 Processor -- start 2023-08-25T09:10:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.522+0000 7f976ffff700 1 -- start start 2023-08-25T09:10:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.522+0000 7f976ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97681008a0 0x7f976819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.522+0000 7f976ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9768101260 0x7f976819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.522+0000 7f976ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97681068b0 0x7f97681989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976ffff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768199040 con 0x7f9768101260 2023-08-25T09:10:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976ffff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97681991b0 con 0x7f97681068b0 2023-08-25T09:10:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768199370 con 0x7f97681008a0 2023-08-25T09:10:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9768101260 0x7f976819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9768101260 0x7f976819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52368/0 (socket says 172.21.15.99:52368) 2023-08-25T09:10:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97681008a0 0x7f976819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 -- 172.21.15.99:0/3865627260 learned_addr learned my addr 172.21.15.99:0/3865627260 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976e59c700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97681068b0 0x7f97681989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 -- 172.21.15.99:0/3865627260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97681008a0 msgr2=0x7f976819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97681008a0 0x7f976819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 -- 172.21.15.99:0/3865627260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97681068b0 msgr2=0x7f97681989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97681068b0 0x7f97681989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 -- 172.21.15.99:0/3865627260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580095f0 con 0x7f9768101260 2023-08-25T09:10:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.523+0000 7f976d59a700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9768101260 0x7f976819f0b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9758005910 tx=0x7f9758005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.524+0000 7f975effd700 1 -- 172.21.15.99:0/3865627260 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975801d070 con 0x7f9768101260 2023-08-25T09:10:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.524+0000 7f975effd700 1 -- 172.21.15.99:0/3865627260 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9758022470 con 0x7f9768101260 2023-08-25T09:10:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.524+0000 7f976ffff700 1 -- 172.21.15.99:0/3865627260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97681995f0 con 0x7f9768101260 2023-08-25T09:10:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.524+0000 7f975effd700 1 -- 172.21.15.99:0/3865627260 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758017650 con 0x7f9768101260 2023-08-25T09:10:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.524+0000 7f976ffff700 1 -- 172.21.15.99:0/3865627260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97681a5b60 con 0x7f9768101260 2023-08-25T09:10:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.527+0000 7f975effd700 1 -- 172.21.15.99:0/3865627260 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f97580225e0 con 0x7f9768101260 2023-08-25T09:10:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.528+0000 7f975effd700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f97540a00c0 0x7f97540a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.528+0000 7f9783a13b80 1 Processor -- start 2023-08-25T09:10:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.528+0000 7f975effd700 1 -- 172.21.15.99:0/3865627260 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f97580cc9a0 con 0x7f9768101260 2023-08-25T09:10:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.528+0000 7f976dd9b700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f97540a00c0 0x7f97540a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f9783a13b80 1 -- start start 2023-08-25T09:10:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad752f40 0x55c4ad753320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f976dd9b700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f97540a00c0 0x7f97540a2580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9764009be0 tx=0x7f9764009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad753860 0x55c4ad74d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad74ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f9783a13b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad6b5750 con 0x55c4ad752f40 2023-08-25T09:10:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f9783a13b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad6a6720 con 0x55c4ad74d990 2023-08-25T09:10:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f9783a13b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad6a4040 con 0x55c4ad753860 2023-08-25T09:10:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f976dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad752f40 0x55c4ad753320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f976dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad752f40 0x55c4ad753320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52380/0 (socket says 172.21.15.99:52380) 2023-08-25T09:10:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f976dd9b700 1 -- 172.21.15.99:0/2937334886 learned_addr learned my addr 172.21.15.99:0/2937334886 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f976e59c700 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad74ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.529+0000 7f976d59a700 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad753860 0x55c4ad74d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f976dd9b700 1 -- 172.21.15.99:0/2937334886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad753860 msgr2=0x55c4ad74d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad753860 0x55c4ad74d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f976dd9b700 1 -- 172.21.15.99:0/2937334886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 msgr2=0x55c4ad74ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad74ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f976dd9b700 1 -- 172.21.15.99:0/2937334886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4ad51f5d0 con 0x55c4ad752f40 2023-08-25T09:10:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad752f40 0x55c4ad753320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9764011d70 tx=0x7f9764014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880bb2e3cfee775a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f9753fff700 1 -- 172.21.15.99:0/2937334886 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764011da0 con 0x55c4ad752f40 2023-08-25T09:10:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f9753fff700 1 -- 172.21.15.99:0/2937334886 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9764004020 con 0x55c4ad752f40 2023-08-25T09:10:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.530+0000 7f9753fff700 1 -- 172.21.15.99:0/2937334886 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764003cb0 con 0x55c4ad752f40 2023-08-25T09:10:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 -- 172.21.15.99:0/2937334886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad752f40 msgr2=0x55c4ad753320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad752f40 0x55c4ad753320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9764011d70 tx=0x7f9764014000 comp rx=0 tx=0).stop 2023-08-25T09:10:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 -- 172.21.15.99:0/2937334886 shutdown_connections 2023-08-25T09:10:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad74ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad752f40 0x55c4ad753320 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2937334886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad753860 0x55c4ad74d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 -- 172.21.15.99:0/2937334886 >> 172.21.15.99:0/2937334886 conn(0x55c4ad6c2080 msgr2=0x55c4ad747d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 -- 172.21.15.99:0/2937334886 shutdown_connections 2023-08-25T09:10:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.531+0000 7f9783a13b80 1 -- 172.21.15.99:0/2937334886 wait complete. 2023-08-25T09:10:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.532+0000 7f9783a13b80 1 -- start start 2023-08-25T09:10:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.532+0000 7f9783a13b80 1 Processor -- start 2023-08-25T09:10:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.532+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad759700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.532+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad756910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad753860 0x55c4ad757000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f9783a13b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad69f930 con 0x55c4ad753860 2023-08-25T09:10:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f9783a13b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad63db50 con 0x55c4ad74d990 2023-08-25T09:10:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f9783a13b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad63d850 con 0x55c4ad752f40 2023-08-25T09:10:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f9783a13b80 10 client.0 _ll_register_callbacks cb 0x55c4ad587a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:10:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad759700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad759700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:53822/0 (socket says 172.21.15.99:53822) 2023-08-25T09:10:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976dd9b700 1 -- 172.21.15.99:0/2946359572 learned_addr learned my addr 172.21.15.99:0/2946359572 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976e59c700 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad753860 0x55c4ad757000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976d59a700 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad756910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976dd9b700 1 -- 172.21.15.99:0/2946359572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 msgr2=0x55c4ad756910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad756910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976dd9b700 1 -- 172.21.15.99:0/2946359572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad753860 msgr2=0x55c4ad757000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad753860 0x55c4ad757000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.533+0000 7f976dd9b700 1 -- 172.21.15.99:0/2946359572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9764017070 con 0x55c4ad74d990 2023-08-25T09:10:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad759700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9764009210 tx=0x7f9764003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b052ad2147c86c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f97517fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:10:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f97517fa700 1 -- 172.21.15.99:0/2946359572 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976401e930 con 0x55c4ad74d990 2023-08-25T09:10:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f9783a13b80 1 -- 172.21.15.99:0/2946359572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4ad71d160 con 0x55c4ad74d990 2023-08-25T09:10:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f97517fa700 1 -- 172.21.15.99:0/2946359572 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f976401ea90 con 0x55c4ad74d990 2023-08-25T09:10:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f97517fa700 1 -- 172.21.15.99:0/2946359572 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f976401ec50 con 0x55c4ad74d990 2023-08-25T09:10:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f9783a13b80 10 client.26159 fetch_fsmap learned FSMap version 37 2023-08-25T09:10:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f9783a13b80 1 -- 172.21.15.99:0/2946359572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c4ad4cbd80 con 0x55c4ad74d990 2023-08-25T09:10:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.534+0000 7f97517fa700 1 -- 172.21.15.99:0/2946359572 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f976403c080 con 0x55c4ad74d990 2023-08-25T09:10:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f97517fa700 1 -- 172.21.15.99:0/2946359572 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9764024030 con 0x55c4ad74d990 2023-08-25T09:10:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f9783a13b80 10 client.26159 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:10:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f9783a13b80 10 client.26159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:10:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f9783a13b80 20 client.26159 populate_metadata read hostname 'smithi099' 2023-08-25T09:10:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c4ad767130 0x55c4ad769550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f9783a13b80 4 client.26159 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:10:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f9783a13b80 1 -- 172.21.15.99:0/2946359572 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4ad645b40 con 0x55c4ad767130 2023-08-25T09:10:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f976d59a700 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c4ad767130 0x55c4ad769550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f97517fa700 10 client.26159 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.535+0000 7f976d59a700 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c4ad767130 0x55c4ad769550 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce87e2a53c49717 server_cookie=78c6a4740f94a9a9 in_seq=0 out_seq=0 2023-08-25T09:10:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.536+0000 7f97517fa700 1 -- 172.21.15.99:0/2946359572 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c4ad645b40 con 0x55c4ad767130 2023-08-25T09:10:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.536+0000 7f97517fa700 10 client.26159 handle_command_reply: tid=0 2023-08-25T09:10:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.536+0000 7f9783a13b80 1 client.26159 shutdown 2023-08-25T09:10:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.536+0000 7f9783a13b80 1 -- 172.21.15.99:0/2946359572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 msgr2=0x55c4ad759700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.536+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad759700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9764009210 tx=0x7f9764003680 comp rx=0 tx=0).stop 2023-08-25T09:10:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 1 -- 172.21.15.99:0/2946359572 shutdown_connections 2023-08-25T09:10:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad753860 0x55c4ad757000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c4ad767130 0x55c4ad769550 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad74d990 0x55c4ad759700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f97517fa700 0 client.26159 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2946359572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad756910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 1 -- 172.21.15.99:0/2946359572 >> 172.21.15.99:0/2946359572 conn(0x55c4ad6c2080 msgr2=0x55c4ad742820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 1 -- 172.21.15.99:0/2946359572 shutdown_connections 2023-08-25T09:10:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 1 -- 172.21.15.99:0/2946359572 wait complete. 2023-08-25T09:10:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.537+0000 7f9783a13b80 20 client.26159 trim_cache size 0 max 16384 2023-08-25T09:10:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f9783a13b80 1 Processor -- start 2023-08-25T09:10:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f9783a13b80 1 -- start start 2023-08-25T09:10:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad74d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad75e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad759230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f9783a13b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad5218c0 con 0x55c4ad74d990 2023-08-25T09:10:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f9783a13b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad749670 con 0x55c4ad753860 2023-08-25T09:10:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f9783a13b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad740420 con 0x55c4ad752f40 2023-08-25T09:10:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f976dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad74d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f976dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad74d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52398/0 (socket says 172.21.15.99:52398) 2023-08-25T09:10:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f976dd9b700 1 -- 172.21.15.99:0/2867327459 learned_addr learned my addr 172.21.15.99:0/2867327459 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.552+0000 7f976d59a700 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad75e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f976e59c700 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad759230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f976dd9b700 1 -- 172.21.15.99:0/2867327459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 msgr2=0x55c4ad75e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad75e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f976dd9b700 1 -- 172.21.15.99:0/2867327459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 msgr2=0x55c4ad759230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad759230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f976dd9b700 1 -- 172.21.15.99:0/2867327459 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4ad4b4c00 con 0x55c4ad74d990 2023-08-25T09:10:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad74d450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f97640099c0 tx=0x7f97640092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265e9e668268e7c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f975cff9700 1 -- 172.21.15.99:0/2867327459 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976403f070 con 0x55c4ad74d990 2023-08-25T09:10:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f975cff9700 1 -- 172.21.15.99:0/2867327459 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c4ad71d160 con 0x55c4ad74d990 2023-08-25T09:10:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.553+0000 7f975cff9700 1 -- 172.21.15.99:0/2867327459 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764003950 con 0x55c4ad74d990 2023-08-25T09:10:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 -- 172.21.15.99:0/2867327459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 msgr2=0x55c4ad74d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad74d450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f97640099c0 tx=0x7f97640092d0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 -- 172.21.15.99:0/2867327459 shutdown_connections 2023-08-25T09:10:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad759230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad74d450 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2867327459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad75e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 -- 172.21.15.99:0/2867327459 >> 172.21.15.99:0/2867327459 conn(0x55c4ad6c2080 msgr2=0x55c4ad750a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 -- 172.21.15.99:0/2867327459 shutdown_connections 2023-08-25T09:10:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 -- 172.21.15.99:0/2867327459 wait complete. 2023-08-25T09:10:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 -- start start 2023-08-25T09:10:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.554+0000 7f9783a13b80 1 Processor -- start 2023-08-25T09:10:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad755fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad756510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f9783a13b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad754e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f9783a13b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad756c00 con 0x55c4ad74d990 2023-08-25T09:10:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f9783a13b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad7553b0 con 0x55c4ad753860 2023-08-25T09:10:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f9783a13b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ad755570 con 0x55c4ad752f40 2023-08-25T09:10:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f9783a13b80 10 client.0 _ll_register_callbacks cb 0x7f9760009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:10:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f976e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad754e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f976dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad755fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f976e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad754e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:53836/0 (socket says 172.21.15.99:53836) 2023-08-25T09:10:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f976e59c700 1 -- 172.21.15.99:0/2916528147 learned_addr learned my addr 172.21.15.99:0/2916528147 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.555+0000 7f976d59a700 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad756510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f976e59c700 1 -- 172.21.15.99:0/2916528147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 msgr2=0x55c4ad756510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f976e59c700 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad756510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f976e59c700 1 -- 172.21.15.99:0/2916528147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 msgr2=0x55c4ad755fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f976e59c700 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad755fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f976e59c700 1 -- 172.21.15.99:0/2916528147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4ad4cbd80 con 0x55c4ad753860 2023-08-25T09:10:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad755fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:10:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f976e59c700 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad754e70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9760007f70 tx=0x7f97600114d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362c147a373955c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f97517fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:10:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f97517fa700 1 -- 172.21.15.99:0/2916528147 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976001c070 con 0x55c4ad753860 2023-08-25T09:10:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f97517fa700 1 -- 172.21.15.99:0/2916528147 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f9760004bb0 con 0x55c4ad753860 2023-08-25T09:10:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f9783a13b80 1 -- 172.21.15.99:0/2916528147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4ad740600 con 0x55c4ad753860 2023-08-25T09:10:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f97517fa700 1 -- 172.21.15.99:0/2916528147 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f976000e040 con 0x55c4ad753860 2023-08-25T09:10:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f9783a13b80 10 client.26165 fetch_fsmap learned FSMap version 37 2023-08-25T09:10:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.556+0000 7f9783a13b80 1 -- 172.21.15.99:0/2916528147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c4ad4f21f0 con 0x55c4ad753860 2023-08-25T09:10:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f97517fa700 1 -- 172.21.15.99:0/2916528147 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f976002b020 con 0x55c4ad753860 2023-08-25T09:10:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f97517fa700 1 -- 172.21.15.99:0/2916528147 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9760020410 con 0x55c4ad753860 2023-08-25T09:10:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f9783a13b80 10 client.26165 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:10:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f9783a13b80 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:10:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f9783a13b80 20 client.26165 populate_metadata read hostname 'smithi099' 2023-08-25T09:10:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c4ad7bbc80 0x55c4ad7be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f9783a13b80 4 client.26165 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:10:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f9783a13b80 1 -- 172.21.15.99:0/2916528147 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c4ad698910 con 0x55c4ad7bbc80 2023-08-25T09:10:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.557+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c4ad7bbc80 0x55c4ad7be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.558+0000 7f97517fa700 10 client.26165 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.558+0000 7f976dd9b700 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c4ad7bbc80 0x55c4ad7be0a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9a64e564456c91 server_cookie=f2d8d0dba5907d2f in_seq=0 out_seq=0 2023-08-25T09:10:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f97517fa700 1 -- 172.21.15.99:0/2916528147 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c4ad698910 con 0x55c4ad7bbc80 2023-08-25T09:10:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f97517fa700 10 client.26165 handle_command_reply: tid=0 2023-08-25T09:10:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 client.26165 shutdown 2023-08-25T09:10:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 -- 172.21.15.99:0/2916528147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 msgr2=0x55c4ad754e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad754e70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9760007f70 tx=0x7f97600114d0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 -- 172.21.15.99:0/2916528147 shutdown_connections 2023-08-25T09:10:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55c4ad7bbc80 0x55c4ad7be0a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4ad753860 0x55c4ad754e70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ad74d990 0x55c4ad755fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f97517fa700 0 client.26165 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 --2- 172.21.15.99:0/2916528147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ad752f40 0x55c4ad756510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 -- 172.21.15.99:0/2916528147 >> 172.21.15.99:0/2916528147 conn(0x55c4ad6c2080 msgr2=0x55c4ad753c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.612+0000 7f9783a13b80 1 -- 172.21.15.99:0/2916528147 shutdown_connections 2023-08-25T09:10:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.613+0000 7f9783a13b80 1 -- 172.21.15.99:0/2916528147 wait complete. 2023-08-25T09:10:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.613+0000 7f9783a13b80 20 client.26165 trim_cache size 0 max 16384 2023-08-25T09:10:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:10:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:10:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:10:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:10:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.613+0000 7f976ffff700 1 -- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f97540a00c0 msgr2=0x7f97540a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.613+0000 7f976ffff700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f97540a00c0 0x7f97540a2580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9764009be0 tx=0x7f9764009260 comp rx=0 tx=0).stop 2023-08-25T09:10:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.613+0000 7f976ffff700 1 -- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9768101260 msgr2=0x7f976819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.613+0000 7f976ffff700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9768101260 0x7f976819f0b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9758005910 tx=0x7f9758005720 comp rx=0 tx=0).stop 2023-08-25T09:10:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.614+0000 7f976ffff700 1 -- 172.21.15.99:0/3865627260 shutdown_connections 2023-08-25T09:10:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.614+0000 7f976ffff700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f97540a00c0 0x7f97540a2580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.614+0000 7f976ffff700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97681068b0 0x7f97681989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.614+0000 7f976ffff700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97681008a0 0x7f976819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.614+0000 7f976ffff700 1 --2- 172.21.15.99:0/3865627260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9768101260 0x7f976819f0b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.614+0000 7f976ffff700 1 -- 172.21.15.99:0/3865627260 >> 172.21.15.99:0/3865627260 conn(0x7f97680fc370 msgr2=0x7f97680fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.614+0000 7f976ffff700 1 -- 172.21.15.99:0/3865627260 shutdown_connections 2023-08-25T09:10:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:08.614+0000 7f976ffff700 1 -- 172.21.15.99:0/3865627260 wait complete. 2023-08-25T09:10:08.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426742 3.91 MB/sec execute 494 sec latency 2834.443 ms 2023-08-25T09:10:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:09 smithi181 ceph-mon[131296]: pgmap v341: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-08-25T09:10:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:09 smithi181 ceph-mon[131296]: osdmap e98: 12 total, 12 up, 12 in 2023-08-25T09:10:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:09 smithi099 ceph-mon[97641]: pgmap v341: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-08-25T09:10:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:09 smithi099 ceph-mon[97641]: osdmap e98: 12 total, 12 up, 12 in 2023-08-25T09:10:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:09 smithi157 ceph-mon[130879]: pgmap v341: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-08-25T09:10:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:09 smithi157 ceph-mon[130879]: osdmap e98: 12 total, 12 up, 12 in 2023-08-25T09:10:09.943 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427365 3.91 MB/sec execute 495 sec latency 3368.513 ms 2023-08-25T09:10:10.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427365 3.90 MB/sec execute 496 sec latency 1045.913 ms 2023-08-25T09:10:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:11 smithi181 ceph-mon[131296]: pgmap v343: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1001 KiB/s wr, 79 op/s 2023-08-25T09:10:11.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:11 smithi099 ceph-mon[97641]: pgmap v343: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1001 KiB/s wr, 79 op/s 2023-08-25T09:10:11.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:11 smithi157 ceph-mon[130879]: pgmap v343: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1001 KiB/s wr, 79 op/s 2023-08-25T09:10:11.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427365 3.90 MB/sec execute 497 sec latency 2046.036 ms 2023-08-25T09:10:12.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427365 3.89 MB/sec execute 498 sec latency 3046.180 ms 2023-08-25T09:10:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:13 smithi181 ceph-mon[131296]: pgmap v344: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2023-08-25T09:10:13.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:13 smithi099 ceph-mon[97641]: pgmap v344: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2023-08-25T09:10:13.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:13 smithi157 ceph-mon[130879]: pgmap v344: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2023-08-25T09:10:13.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427365 3.88 MB/sec execute 499 sec latency 4046.251 ms 2023-08-25T09:10:14.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 428281 3.88 MB/sec execute 500 sec latency 4592.374 ms 2023-08-25T09:10:15.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:15 smithi181 ceph-mon[131296]: pgmap v345: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 936 KiB/s wr, 106 op/s 2023-08-25T09:10:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:15 smithi099 ceph-mon[97641]: pgmap v345: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 936 KiB/s wr, 106 op/s 2023-08-25T09:10:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:15 smithi157 ceph-mon[130879]: pgmap v345: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 936 KiB/s wr, 106 op/s 2023-08-25T09:10:15.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 429753 3.89 MB/sec execute 501 sec latency 22.695 ms 2023-08-25T09:10:16.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 430914 3.90 MB/sec execute 502 sec latency 22.406 ms 2023-08-25T09:10:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:17 smithi181 ceph-mon[131296]: pgmap v346: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 960 KiB/s wr, 91 op/s 2023-08-25T09:10:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:17 smithi099 ceph-mon[97641]: pgmap v346: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 960 KiB/s wr, 91 op/s 2023-08-25T09:10:17.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:17 smithi157 ceph-mon[130879]: pgmap v346: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 960 KiB/s wr, 91 op/s 2023-08-25T09:10:17.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 430914 3.89 MB/sec execute 503 sec latency 1014.972 ms 2023-08-25T09:10:18.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 430914 3.88 MB/sec execute 504 sec latency 2015.117 ms 2023-08-25T09:10:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:19 smithi181 ceph-mon[131296]: pgmap v347: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 960 KiB/s wr, 91 op/s 2023-08-25T09:10:19.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:19 smithi099 ceph-mon[97641]: pgmap v347: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 960 KiB/s wr, 91 op/s 2023-08-25T09:10:19.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:19 smithi157 ceph-mon[130879]: pgmap v347: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 960 KiB/s wr, 91 op/s 2023-08-25T09:10:19.944 INFO:tasks.workunit.client.0.smithi099.stdout: 1 431981 3.88 MB/sec execute 505 sec latency 2550.211 ms 2023-08-25T09:10:20.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 433323 3.89 MB/sec execute 506 sec latency 14.731 ms 2023-08-25T09:10:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:21 smithi181 ceph-mon[131296]: pgmap v348: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 864 KiB/s wr, 82 op/s 2023-08-25T09:10:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 -- 172.21.15.99:0/3330215828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a01068b0 msgr2=0x7f26a0106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 --2- 172.21.15.99:0/3330215828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a01068b0 0x7f26a0106c90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2690009990 tx=0x7f2690009ca0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 -- 172.21.15.99:0/3330215828 shutdown_connections 2023-08-25T09:10:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 --2- 172.21.15.99:0/3330215828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a0101260 0x7f26a010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 --2- 172.21.15.99:0/3330215828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a01068b0 0x7f26a0106c90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 --2- 172.21.15.99:0/3330215828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a01008a0 0x7f26a0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 -- 172.21.15.99:0/3330215828 >> 172.21.15.99:0/3330215828 conn(0x7f26a00fc370 msgr2=0x7f26a00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 -- 172.21.15.99:0/3330215828 shutdown_connections 2023-08-25T09:10:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.766+0000 7f26a67ff700 1 -- 172.21.15.99:0/3330215828 wait complete. 2023-08-25T09:10:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.767+0000 7f26a67ff700 1 Processor -- start 2023-08-25T09:10:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f26a67ff700 1 -- start start 2023-08-25T09:10:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f26a67ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a01008a0 0x7f26a0072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f26a67ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0101260 0x7f26a0073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f26a67ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a01068b0 0x7f26a01a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f26a67ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a01a5070 con 0x7f26a01068b0 2023-08-25T09:10:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f26a67ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a01a51b0 con 0x7f26a0101260 2023-08-25T09:10:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f26a67ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a01a5320 con 0x7f26a01008a0 2023-08-25T09:10:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f269f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0101260 0x7f26a0073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f269f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0101260 0x7f26a0073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57428/0 (socket says 172.21.15.99:57428) 2023-08-25T09:10:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f26a4d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a01068b0 0x7f26a01a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f269f7fe700 1 -- 172.21.15.99:0/2970572926 learned_addr learned my addr 172.21.15.99:0/2970572926 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f269ffff700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a01008a0 0x7f26a0072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.768+0000 7f269f7fe700 1 -- 172.21.15.99:0/2970572926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a01008a0 msgr2=0x7f26a0072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f269f7fe700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a01008a0 0x7f26a0072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f269f7fe700 1 -- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a01068b0 msgr2=0x7f26a01a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f269f7fe700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a01068b0 0x7f26a01a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f269f7fe700 1 -- 172.21.15.99:0/2970572926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694009520 con 0x7f26a0101260 2023-08-25T09:10:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f269f7fe700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0101260 0x7f26a0073340 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f269400eb30 tx=0x7f269400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f269d7fa700 1 -- 172.21.15.99:0/2970572926 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694009880 con 0x7f26a0101260 2023-08-25T09:10:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f269d7fa700 1 -- 172.21.15.99:0/2970572926 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26940044e0 con 0x7f26a0101260 2023-08-25T09:10:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f26a67ff700 1 -- 172.21.15.99:0/2970572926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26900095f0 con 0x7f26a0101260 2023-08-25T09:10:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f269d7fa700 1 -- 172.21.15.99:0/2970572926 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694003d80 con 0x7f26a0101260 2023-08-25T09:10:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.769+0000 7f26a67ff700 1 -- 172.21.15.99:0/2970572926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a01a59d0 con 0x7f26a0101260 2023-08-25T09:10:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.772+0000 7f269d7fa700 1 -- 172.21.15.99:0/2970572926 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f2694003f20 con 0x7f26a0101260 2023-08-25T09:10:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.774+0000 7f26ba20db80 1 Processor -- start 2023-08-25T09:10:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.774+0000 7f26ba20db80 1 -- start start 2023-08-25T09:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb1f40 0x55b50bfb2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb2860 0x55b50bfac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfacdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f26ba20db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bf14750 con 0x55b50bfb1f40 2023-08-25T09:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f26ba20db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bf05720 con 0x55b50bfac990 2023-08-25T09:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f26ba20db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bf03040 con 0x55b50bfb2860 2023-08-25T09:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f26a4d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb1f40 0x55b50bfb2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269d7fa700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f26880a0140 0x7f26880a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb2860 0x55b50bfac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfacdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb2860 0x55b50bfac450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:60550/0 (socket says 172.21.15.99:60550) 2023-08-25T09:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269ffff700 1 -- 172.21.15.99:0/1851260634 learned_addr learned my addr 172.21.15.99:0/1851260634 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f26a4d9c700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f26880a0140 0x7f26880a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269ffff700 1 -- 172.21.15.99:0/1851260634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 msgr2=0x55b50bfacdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269ffff700 1 --2- 172.21.15.99:0/1851260634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfacdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269ffff700 1 -- 172.21.15.99:0/1851260634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb1f40 msgr2=0x55b50bfb2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269ffff700 1 --2- 172.21.15.99:0/1851260634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb1f40 0x55b50bfb2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269d7fa700 1 -- 172.21.15.99:0/2970572926 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2694014070 con 0x7f26a0101260 2023-08-25T09:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.775+0000 7f269ffff700 1 -- 172.21.15.99:0/1851260634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b50bd7e5d0 con 0x55b50bfb2860 2023-08-25T09:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.776+0000 7f269ffff700 1 --2- 172.21.15.99:0/1851260634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb2860 0x55b50bfac450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2690009990 tx=0x7f269000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c1a1354177b154 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.776+0000 7f26a4d9c700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f26880a0140 0x7f26880a2600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ec0 tx=0x7f2698004b40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.776+0000 7f26867fc700 1 -- 172.21.15.99:0/1851260634 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269001d070 con 0x55b50bfb2860 2023-08-25T09:10:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.776+0000 7f26867fc700 1 -- 172.21.15.99:0/1851260634 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f269001e040 con 0x55b50bfb2860 2023-08-25T09:10:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.776+0000 7f26867fc700 1 -- 172.21.15.99:0/1851260634 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26900049c0 con 0x55b50bfb2860 2023-08-25T09:10:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 -- 172.21.15.99:0/1851260634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb2860 msgr2=0x55b50bfac450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1851260634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb2860 0x55b50bfac450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2690009990 tx=0x7f269000b610 comp rx=0 tx=0).stop 2023-08-25T09:10:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 -- 172.21.15.99:0/1851260634 shutdown_connections 2023-08-25T09:10:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1851260634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfacdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1851260634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb1f40 0x55b50bfb2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1851260634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb2860 0x55b50bfac450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 -- 172.21.15.99:0/1851260634 >> 172.21.15.99:0/1851260634 conn(0x55b50bf21080 msgr2=0x55b50bfa6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 -- 172.21.15.99:0/1851260634 shutdown_connections 2023-08-25T09:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 -- 172.21.15.99:0/1851260634 wait complete. 2023-08-25T09:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.777+0000 7f26ba20db80 1 -- start start 2023-08-25T09:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26ba20db80 1 Processor -- start 2023-08-25T09:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfac990 0x55b50bfb86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 0x55b50bfb58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb2860 0x55b50bfb5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26ba20db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50befe930 con 0x55b50bfac990 2023-08-25T09:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26ba20db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50be9cb50 con 0x55b50bfb2860 2023-08-25T09:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26ba20db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50be9c850 con 0x55b50bfb1f40 2023-08-25T09:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26ba20db80 10 client.0 _ll_register_callbacks cb 0x55b50bde6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f269ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfac990 0x55b50bfb86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f26a4d9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb2860 0x55b50bfb5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f269ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfac990 0x55b50bfb86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34174/0 (socket says 172.21.15.99:34174) 2023-08-25T09:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f269ffff700 1 -- 172.21.15.99:0/3192173564 learned_addr learned my addr 172.21.15.99:0/3192173564 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.778+0000 7f269f7fe700 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 0x55b50bfb58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f269ffff700 1 -- 172.21.15.99:0/3192173564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 msgr2=0x55b50bfb58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f269ffff700 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 0x55b50bfb58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f269ffff700 1 -- 172.21.15.99:0/3192173564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb2860 msgr2=0x55b50bfb5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f269ffff700 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb2860 0x55b50bfb5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f269ffff700 1 -- 172.21.15.99:0/3192173564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a010e070 con 0x55b50bfac990 2023-08-25T09:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f269ffff700 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfac990 0x55b50bfb86a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f269000b2e0 tx=0x7f269000f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f26ea808b656d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f267f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f267f7fe700 1 -- 172.21.15.99:0/3192173564 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269001d070 con 0x55b50bfac990 2023-08-25T09:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f26ba20db80 1 -- 172.21.15.99:0/3192173564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b50bf7c160 con 0x55b50bfac990 2023-08-25T09:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f267f7fe700 1 -- 172.21.15.99:0/3192173564 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f269000faf0 con 0x55b50bfac990 2023-08-25T09:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f267f7fe700 1 -- 172.21.15.99:0/3192173564 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2690005bc0 con 0x55b50bfac990 2023-08-25T09:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.779+0000 7f26ba20db80 10 client.16119 fetch_fsmap learned FSMap version 37 2023-08-25T09:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f26ba20db80 1 -- 172.21.15.99:0/3192173564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55b50bd2ad80 con 0x55b50bfac990 2023-08-25T09:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f267f7fe700 1 -- 172.21.15.99:0/3192173564 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b50bf7c160 con 0x55b50bfac990 2023-08-25T09:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f267f7fe700 1 -- 172.21.15.99:0/3192173564 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f269002bc50 con 0x55b50bfac990 2023-08-25T09:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f26ba20db80 10 client.16119 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f26ba20db80 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f26ba20db80 20 client.16119 populate_metadata read hostname 'smithi099' 2023-08-25T09:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f26ba20db80 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50bfc5bf0 0x55b50bfc7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f26ba20db80 4 client.16119 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f26ba20db80 1 -- 172.21.15.99:0/3192173564 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b50bea4b40 con 0x55b50bfc5bf0 2023-08-25T09:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.780+0000 7f269f7fe700 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50bfc5bf0 0x55b50bfc7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.781+0000 7f267f7fe700 10 client.16119 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.781+0000 7f269f7fe700 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50bfc5bf0 0x55b50bfc7fe0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1088a872b5b62bc server_cookie=2e76514b79cdd548 in_seq=0 out_seq=0 2023-08-25T09:10:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.781+0000 7f267f7fe700 1 -- 172.21.15.99:0/3192173564 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b50bea4b40 con 0x55b50bfc5bf0 2023-08-25T09:10:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.781+0000 7f267f7fe700 10 client.16119 handle_command_reply: tid=0 2023-08-25T09:10:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 client.16119 shutdown 2023-08-25T09:10:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 -- 172.21.15.99:0/3192173564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfac990 msgr2=0x55b50bfb86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfac990 0x55b50bfb86a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f269000b2e0 tx=0x7f269000f6f0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 -- 172.21.15.99:0/3192173564 shutdown_connections 2023-08-25T09:10:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50bfc5bf0 0x55b50bfc7fe0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb2860 0x55b50bfb5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfac990 0x55b50bfb86a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f267f7fe700 0 client.16119 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 --2- 172.21.15.99:0/3192173564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 0x55b50bfb58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 -- 172.21.15.99:0/3192173564 >> 172.21.15.99:0/3192173564 conn(0x55b50bf21080 msgr2=0x55b50bfa1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 -- 172.21.15.99:0/3192173564 shutdown_connections 2023-08-25T09:10:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 1 -- 172.21.15.99:0/3192173564 wait complete. 2023-08-25T09:10:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.782+0000 7f26ba20db80 20 client.16119 trim_cache size 0 max 16384 2023-08-25T09:10:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.797+0000 7f26ba20db80 1 Processor -- start 2023-08-25T09:10:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.797+0000 7f26ba20db80 1 -- start start 2023-08-25T09:10:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.797+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfac990 0x55b50bfac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.797+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb1f40 0x55b50bfbd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.797+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfb81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.797+0000 7f26ba20db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bd808c0 con 0x55b50bfb2860 2023-08-25T09:10:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.797+0000 7f26ba20db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bfa72c0 con 0x55b50bfb1f40 2023-08-25T09:10:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.797+0000 7f26ba20db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bfb26b0 con 0x55b50bfac990 2023-08-25T09:10:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f26a4d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfb81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfac990 0x55b50bfac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb1f40 0x55b50bfbd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfac990 0x55b50bfac450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:60566/0 (socket says 172.21.15.99:60566) 2023-08-25T09:10:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 -- 172.21.15.99:0/1755518225 learned_addr learned my addr 172.21.15.99:0/1755518225 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 -- 172.21.15.99:0/1755518225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb1f40 msgr2=0x55b50bfbd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 --2- 172.21.15.99:0/1755518225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb1f40 0x55b50bfbd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 -- 172.21.15.99:0/1755518225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 msgr2=0x55b50bfb81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 --2- 172.21.15.99:0/1755518225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfb81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 -- 172.21.15.99:0/1755518225 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b50bd13c00 con 0x55b50bfac990 2023-08-25T09:10:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f269ffff700 1 --2- 172.21.15.99:0/1755518225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfac990 0x55b50bfac450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2690006010 tx=0x7f269000b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2992930c05d63ad server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f2686ffd700 1 -- 172.21.15.99:0/1755518225 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269001d070 con 0x55b50bfac990 2023-08-25T09:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f2686ffd700 1 -- 172.21.15.99:0/1755518225 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f269001e040 con 0x55b50bfac990 2023-08-25T09:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.798+0000 7f2686ffd700 1 -- 172.21.15.99:0/1755518225 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269000e3e0 con 0x55b50bfac990 2023-08-25T09:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 -- 172.21.15.99:0/1755518225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfac990 msgr2=0x55b50bfac450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1755518225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfac990 0x55b50bfac450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2690006010 tx=0x7f269000b690 comp rx=0 tx=0).stop 2023-08-25T09:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 -- 172.21.15.99:0/1755518225 shutdown_connections 2023-08-25T09:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1755518225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfb81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1755518225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfac990 0x55b50bfac450 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1755518225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfb1f40 0x55b50bfbd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 -- 172.21.15.99:0/1755518225 >> 172.21.15.99:0/1755518225 conn(0x55b50bf21080 msgr2=0x55b50bfaf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 -- 172.21.15.99:0/1755518225 shutdown_connections 2023-08-25T09:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 -- 172.21.15.99:0/1755518225 wait complete. 2023-08-25T09:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.799+0000 7f26ba20db80 1 -- start start 2023-08-25T09:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26ba20db80 1 Processor -- start 2023-08-25T09:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfaea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 0x55b50bfaef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26ba20db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26ba20db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bfbd7e0 con 0x55b50bfb2860 2023-08-25T09:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26ba20db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bfaddc0 con 0x55b50bfac990 2023-08-25T09:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26ba20db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50bfadf80 con 0x55b50bfb1f40 2023-08-25T09:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26ba20db80 10 client.0 _ll_register_callbacks cb 0x7f2694099000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f269ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfaea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f269ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfaea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57470/0 (socket says 172.21.15.99:57470) 2023-08-25T09:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f269ffff700 1 -- 172.21.15.99:0/1618261791 learned_addr learned my addr 172.21.15.99:0/1618261791 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f26a4d9c700 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.800+0000 7f269f7fe700 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 0x55b50bfaef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f269ffff700 1 -- 172.21.15.99:0/1618261791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 msgr2=0x55b50bfaef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f269ffff700 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 0x55b50bfaef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f269ffff700 1 -- 172.21.15.99:0/1618261791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 msgr2=0x55b50bfad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f269ffff700 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f269ffff700 1 -- 172.21.15.99:0/1618261791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b50bd2ad80 con 0x55b50bfac990 2023-08-25T09:10:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f26a4d9c700 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:10:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f269ffff700 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfaea50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26900225f0 tx=0x7f2690010d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8220f6537b963fa9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f267f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:10:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f267f7fe700 1 -- 172.21.15.99:0/1618261791 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269001d040 con 0x55b50bfac990 2023-08-25T09:10:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f26ba20db80 1 -- 172.21.15.99:0/1618261791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b50bf9f600 con 0x55b50bfac990 2023-08-25T09:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f267f7fe700 1 -- 172.21.15.99:0/1618261791 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2690000e40 con 0x55b50bfac990 2023-08-25T09:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f267f7fe700 1 -- 172.21.15.99:0/1618261791 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2690005c30 con 0x55b50bfac990 2023-08-25T09:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f26ba20db80 10 client.26183 fetch_fsmap learned FSMap version 37 2023-08-25T09:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.801+0000 7f26ba20db80 1 -- 172.21.15.99:0/1618261791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55b50bd511f0 con 0x55b50bfac990 2023-08-25T09:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f267f7fe700 1 -- 172.21.15.99:0/1618261791 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f269004f020 con 0x55b50bfac990 2023-08-25T09:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f267f7fe700 1 -- 172.21.15.99:0/1618261791 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2690004620 con 0x55b50bfac990 2023-08-25T09:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f26ba20db80 10 client.26183 fetch_fsmap finished waiting for FSMap version 37 2023-08-25T09:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f26ba20db80 10 client.26183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f26ba20db80 20 client.26183 populate_metadata read hostname 'smithi099' 2023-08-25T09:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50c01a3f0 0x55b50c01c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f26ba20db80 4 client.26183 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f26ba20db80 1 -- 172.21.15.99:0/1618261791 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b50bef7910 con 0x55b50c01a3f0 2023-08-25T09:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f269f7fe700 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50c01a3f0 0x55b50c01c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f267f7fe700 10 client.26183 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.802+0000 7f269f7fe700 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50c01a3f0 0x55b50c01c810 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60a894f6351d557 server_cookie=51b538b55a689f60 in_seq=0 out_seq=0 2023-08-25T09:10:21.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:21 smithi099 ceph-mon[97641]: pgmap v348: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 864 KiB/s wr, 82 op/s 2023-08-25T09:10:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f267f7fe700 1 -- 172.21.15.99:0/1618261791 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b50bef7910 con 0x55b50c01a3f0 2023-08-25T09:10:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f267f7fe700 10 client.26183 handle_command_reply: tid=0 2023-08-25T09:10:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 client.26183 shutdown 2023-08-25T09:10:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 -- 172.21.15.99:0/1618261791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 msgr2=0x55b50bfaea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfaea50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26900225f0 tx=0x7f2690010d30 comp rx=0 tx=0).stop 2023-08-25T09:10:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 -- 172.21.15.99:0/1618261791 shutdown_connections 2023-08-25T09:10:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50bfb2860 0x55b50bfad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b50c01a3f0 0x55b50c01c810 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b50bfac990 0x55b50bfaea50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 --2- 172.21.15.99:0/1618261791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50bfb1f40 0x55b50bfaef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f267f7fe700 0 client.26183 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 -- 172.21.15.99:0/1618261791 >> 172.21.15.99:0/1618261791 conn(0x55b50bf21080 msgr2=0x55b50bfb2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 -- 172.21.15.99:0/1618261791 shutdown_connections 2023-08-25T09:10:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 1 -- 172.21.15.99:0/1618261791 wait complete. 2023-08-25T09:10:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.844+0000 7f26ba20db80 20 client.26183 trim_cache size 0 max 16384 2023-08-25T09:10:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:10:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:10:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:10:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:10:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 -- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f26880a0140 msgr2=0x7f26880a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f26880a0140 0x7f26880a2600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ec0 tx=0x7f2698004b40 comp rx=0 tx=0).stop 2023-08-25T09:10:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 -- 172.21.15.99:0/2970572926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0101260 msgr2=0x7f26a0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0101260 0x7f26a0073340 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f269400eb30 tx=0x7f269400ee40 comp rx=0 tx=0).stop 2023-08-25T09:10:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 -- 172.21.15.99:0/2970572926 shutdown_connections 2023-08-25T09:10:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f26880a0140 0x7f26880a2600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a01068b0 0x7f26a01a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a01008a0 0x7f26a0072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 --2- 172.21.15.99:0/2970572926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0101260 0x7f26a0073340 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 -- 172.21.15.99:0/2970572926 >> 172.21.15.99:0/2970572926 conn(0x7f26a00fc370 msgr2=0x7f26a00fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 -- 172.21.15.99:0/2970572926 shutdown_connections 2023-08-25T09:10:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:21.845+0000 7f26a67ff700 1 -- 172.21.15.99:0/2970572926 wait complete. 2023-08-25T09:10:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:21 smithi157 ceph-mon[130879]: pgmap v348: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 864 KiB/s wr, 82 op/s 2023-08-25T09:10:21.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 434458 3.89 MB/sec execute 507 sec latency 249.989 ms 2023-08-25T09:10:22.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 434458 3.89 MB/sec execute 508 sec latency 1250.116 ms 2023-08-25T09:10:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:23 smithi181 ceph-mon[131296]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2023-08-25T09:10:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:23 smithi099 ceph-mon[97641]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2023-08-25T09:10:23.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:23 smithi157 ceph-mon[130879]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2023-08-25T09:10:23.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 434458 3.88 MB/sec execute 509 sec latency 2250.241 ms 2023-08-25T09:10:24.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 435369 3.88 MB/sec execute 510 sec latency 2784.456 ms 2023-08-25T09:10:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:25 smithi099 ceph-mon[97641]: pgmap v350: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2023-08-25T09:10:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:25 smithi157 ceph-mon[130879]: pgmap v350: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2023-08-25T09:10:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:25 smithi181 ceph-mon[131296]: pgmap v350: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2023-08-25T09:10:25.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 436781 3.89 MB/sec execute 511 sec latency 16.401 ms 2023-08-25T09:10:26.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:26 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:10:26.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:26 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:10:26.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:26 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:10:26.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 437382 3.89 MB/sec execute 512 sec latency 561.640 ms 2023-08-25T09:10:27.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:27 smithi099 ceph-mon[97641]: pgmap v351: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-08-25T09:10:27.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:27 smithi157 ceph-mon[130879]: pgmap v351: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-08-25T09:10:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:27 smithi181 ceph-mon[131296]: pgmap v351: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-08-25T09:10:27.946 INFO:tasks.workunit.client.0.smithi099.stdout: 1 437382 3.88 MB/sec execute 513 sec latency 1561.745 ms 2023-08-25T09:10:28.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 437382 3.87 MB/sec execute 514 sec latency 2561.837 ms 2023-08-25T09:10:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:29 smithi181 ceph-mon[131296]: pgmap v352: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 928 KiB/s wr, 106 op/s 2023-08-25T09:10:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:29 smithi099 ceph-mon[97641]: pgmap v352: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 928 KiB/s wr, 106 op/s 2023-08-25T09:10:29.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:29 smithi157 ceph-mon[130879]: pgmap v352: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 928 KiB/s wr, 106 op/s 2023-08-25T09:10:29.945 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 3.87 MB/sec execute 515 sec latency 3095.576 ms 2023-08-25T09:10:30.946 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 3.86 MB/sec execute 516 sec latency 1056.226 ms 2023-08-25T09:10:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:31 smithi181 ceph-mon[131296]: pgmap v353: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 928 KiB/s wr, 106 op/s 2023-08-25T09:10:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:31 smithi099 ceph-mon[97641]: pgmap v353: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 928 KiB/s wr, 106 op/s 2023-08-25T09:10:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:31 smithi157 ceph-mon[130879]: pgmap v353: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 928 KiB/s wr, 106 op/s 2023-08-25T09:10:31.946 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 3.86 MB/sec execute 517 sec latency 2056.656 ms 2023-08-25T09:10:32.946 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 3.85 MB/sec execute 518 sec latency 3056.813 ms 2023-08-25T09:10:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:33 smithi181 ceph-mon[131296]: pgmap v354: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2023-08-25T09:10:33.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:33 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:10:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:33 smithi099 ceph-mon[97641]: pgmap v354: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2023-08-25T09:10:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:33 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:10:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:33 smithi157 ceph-mon[130879]: pgmap v354: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2023-08-25T09:10:33.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:33 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:10:33.946 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 3.84 MB/sec execute 519 sec latency 4056.876 ms 2023-08-25T09:10:34.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:34 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:10:34.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:34 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:10:34.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:34 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:10:34.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:34 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:10:34.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:34 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:10:34.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:34 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:10:34.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:34 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:10:34.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:34 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:10:34.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:34 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:10:34.946 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438932 3.84 MB/sec execute 520 sec latency 4594.929 ms 2023-08-25T09:10:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 -- 172.21.15.99:0/377060690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01068c0 msgr2=0x7f93b0106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 --2- 172.21.15.99:0/377060690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01068c0 0x7f93b0106ca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009990 tx=0x7f93a0009ca0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 -- 172.21.15.99:0/377060690 shutdown_connections 2023-08-25T09:10:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 --2- 172.21.15.99:0/377060690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b0101270 0x7f93b010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 --2- 172.21.15.99:0/377060690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01068c0 0x7f93b0106ca0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 --2- 172.21.15.99:0/377060690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b01008b0 0x7f93b0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 -- 172.21.15.99:0/377060690 >> 172.21.15.99:0/377060690 conn(0x7f93b00fc380 msgr2=0x7f93b00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 -- 172.21.15.99:0/377060690 shutdown_connections 2023-08-25T09:10:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.000+0000 7f93b57ff700 1 -- 172.21.15.99:0/377060690 wait complete. 2023-08-25T09:10:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.001+0000 7f93b57ff700 1 Processor -- start 2023-08-25T09:10:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.001+0000 7f93b57ff700 1 -- start start 2023-08-25T09:10:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93b57ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01008b0 0x7f93b0072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93b57ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b0101270 0x7f93b0073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93b57ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b01068c0 0x7f93b01a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93b57ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b01a5110 con 0x7f93b01068c0 2023-08-25T09:10:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93b57ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b01a5250 con 0x7f93b0101270 2023-08-25T09:10:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93b57ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b01a5390 con 0x7f93b01008b0 2023-08-25T09:10:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93ae7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b0101270 0x7f93b0073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93ae7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b0101270 0x7f93b0073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52964/0 (socket says 172.21.15.99:52964) 2023-08-25T09:10:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93ae7fc700 1 -- 172.21.15.99:0/712239485 learned_addr learned my addr 172.21.15.99:0/712239485 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93aeffd700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01008b0 0x7f93b0072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93af7fe700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b01068c0 0x7f93b01a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93ae7fc700 1 -- 172.21.15.99:0/712239485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01008b0 msgr2=0x7f93b0072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01008b0 0x7f93b0072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93ae7fc700 1 -- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b01068c0 msgr2=0x7f93b01a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b01068c0 0x7f93b01a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93ae7fc700 1 -- 172.21.15.99:0/712239485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a00095f0 con 0x7f93b0101270 2023-08-25T09:10:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.002+0000 7f93aeffd700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01008b0 0x7f93b0072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:10:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.003+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b0101270 0x7f93b0073340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f93a400e9a0 tx=0x7f93a400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.003+0000 7f9397fff700 1 -- 172.21.15.99:0/712239485 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4009790 con 0x7f93b0101270 2023-08-25T09:10:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.003+0000 7f93b57ff700 1 -- 172.21.15.99:0/712239485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b01a5640 con 0x7f93b0101270 2023-08-25T09:10:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.003+0000 7f9397fff700 1 -- 172.21.15.99:0/712239485 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a4010430 con 0x7f93b0101270 2023-08-25T09:10:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.003+0000 7f9397fff700 1 -- 172.21.15.99:0/712239485 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4018680 con 0x7f93b0101270 2023-08-25T09:10:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.003+0000 7f93b57ff700 1 -- 172.21.15.99:0/712239485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b01a5b50 con 0x7f93b0101270 2023-08-25T09:10:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.007+0000 7f9397fff700 1 -- 172.21.15.99:0/712239485 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f93a40105a0 con 0x7f93b0101270 2023-08-25T09:10:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.008+0000 7f93c9034b80 1 Processor -- start 2023-08-25T09:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.008+0000 7f93c9034b80 1 -- start start 2023-08-25T09:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.008+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf24b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf245450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93c9034b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf1ad750 con 0x55bdaf24b860 2023-08-25T09:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93c9034b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf19e720 con 0x55bdaf245990 2023-08-25T09:10:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93c9034b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf19c040 con 0x55bdaf24af40 2023-08-25T09:10:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf24b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf24b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:55684/0 (socket says 172.21.15.99:55684) 2023-08-25T09:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 -- 172.21.15.99:0/311155694 learned_addr learned my addr 172.21.15.99:0/311155694 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93aeffd700 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf245450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 -- 172.21.15.99:0/311155694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 msgr2=0x55bdaf245dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 -- 172.21.15.99:0/311155694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 msgr2=0x55bdaf245450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf245450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 -- 172.21.15.99:0/311155694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdaf0175d0 con 0x55bdaf24af40 2023-08-25T09:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93aeffd700 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf245450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f9397fff700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93980a00c0 0x7f93980a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f93af7fe700 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf24b320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93a8011600 tx=0x7f93a8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f772abd1389195 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.009+0000 7f9397fff700 1 -- 172.21.15.99:0/712239485 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f93a4014070 con 0x7f93b0101270 2023-08-25T09:10:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.010+0000 7f93aeffd700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93980a00c0 0x7f93980a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.010+0000 7f93957fa700 1 -- 172.21.15.99:0/311155694 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800e040 con 0x55bdaf24af40 2023-08-25T09:10:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.010+0000 7f93957fa700 1 -- 172.21.15.99:0/311155694 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a800f040 con 0x55bdaf24af40 2023-08-25T09:10:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.010+0000 7f93957fa700 1 -- 172.21.15.99:0/311155694 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8004870 con 0x55bdaf24af40 2023-08-25T09:10:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.010+0000 7f93c9034b80 1 -- 172.21.15.99:0/311155694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 msgr2=0x55bdaf24b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.010+0000 7f93aeffd700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93980a00c0 0x7f93980a2580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93a000b2e0 tx=0x7f93a0005fb0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.010+0000 7f93c9034b80 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf24b320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93a8011600 tx=0x7f93a8009920 comp rx=0 tx=0).stop 2023-08-25T09:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 -- 172.21.15.99:0/311155694 shutdown_connections 2023-08-25T09:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf24b320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 --2- 172.21.15.99:0/311155694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf245450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 -- 172.21.15.99:0/311155694 >> 172.21.15.99:0/311155694 conn(0x55bdaf1ba080 msgr2=0x55bdaf23fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 -- 172.21.15.99:0/311155694 shutdown_connections 2023-08-25T09:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 -- 172.21.15.99:0/311155694 wait complete. 2023-08-25T09:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 -- start start 2023-08-25T09:10:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.011+0000 7f93c9034b80 1 Processor -- start 2023-08-25T09:10:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf245990 0x55bdaf251720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf24ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24b860 0x55bdaf24f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93c9034b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf197930 con 0x55bdaf245990 2023-08-25T09:10:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93c9034b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf135b50 con 0x55bdaf24af40 2023-08-25T09:10:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93c9034b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf135850 con 0x55bdaf24b860 2023-08-25T09:10:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93c9034b80 10 client.0 _ll_register_callbacks cb 0x55bdaf07fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:10:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24b860 0x55bdaf24f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93ae7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf24ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24b860 0x55bdaf24f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:55700/0 (socket says 172.21.15.99:55700) 2023-08-25T09:10:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93aeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf245990 0x55bdaf251720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.012+0000 7f93af7fe700 1 -- 172.21.15.99:0/1325515541 learned_addr learned my addr 172.21.15.99:0/1325515541 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93af7fe700 1 -- 172.21.15.99:0/1325515541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 msgr2=0x55bdaf24ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93af7fe700 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf24ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93af7fe700 1 -- 172.21.15.99:0/1325515541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf245990 msgr2=0x55bdaf251720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93af7fe700 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf245990 0x55bdaf251720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93af7fe700 1 -- 172.21.15.99:0/1325515541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a80112b0 con 0x55bdaf24b860 2023-08-25T09:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf24ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93af7fe700 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24b860 0x55bdaf24f150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93a8000f80 tx=0x7f93a8005b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb1bb3d2430a860f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f9392ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f9392ff5700 1 -- 172.21.15.99:0/1325515541 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800e070 con 0x55bdaf24b860 2023-08-25T09:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93c9034b80 1 -- 172.21.15.99:0/1325515541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdaf215160 con 0x55bdaf24b860 2023-08-25T09:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f9392ff5700 1 -- 172.21.15.99:0/1325515541 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f93a8003c10 con 0x55bdaf24b860 2023-08-25T09:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f9392ff5700 1 -- 172.21.15.99:0/1325515541 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a8003dd0 con 0x55bdaf24b860 2023-08-25T09:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93c9034b80 10 client.26179 fetch_fsmap learned FSMap version 38 2023-08-25T09:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.013+0000 7f93c9034b80 1 -- 172.21.15.99:0/1325515541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55bdaefc3d80 con 0x55bdaf24b860 2023-08-25T09:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f9392ff5700 1 -- 172.21.15.99:0/1325515541 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a8032080 con 0x55bdaf24b860 2023-08-25T09:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f9392ff5700 1 -- 172.21.15.99:0/1325515541 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93a8024020 con 0x55bdaf24b860 2023-08-25T09:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f93c9034b80 10 client.26179 fetch_fsmap finished waiting for FSMap version 38 2023-08-25T09:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f93c9034b80 10 client.26179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f93c9034b80 20 client.26179 populate_metadata read hostname 'smithi099' 2023-08-25T09:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f93c9034b80 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bdaf25f1c0 0x55bdaf2615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f93c9034b80 4 client.26179 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:10:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f93c9034b80 1 -- 172.21.15.99:0/1325515541 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdaf13db40 con 0x55bdaf25f1c0 2023-08-25T09:10:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.014+0000 7f93aeffd700 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bdaf25f1c0 0x55bdaf2615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.015+0000 7f93aeffd700 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bdaf25f1c0 0x55bdaf2615e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b19f012bb7cae4 server_cookie=dda2077bfaae11ac in_seq=0 out_seq=0 2023-08-25T09:10:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.015+0000 7f9392ff5700 10 client.26179 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.015+0000 7f9392ff5700 1 -- 172.21.15.99:0/1325515541 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bdaf13db40 con 0x55bdaf25f1c0 2023-08-25T09:10:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.015+0000 7f9392ff5700 10 client.26179 handle_command_reply: tid=0 2023-08-25T09:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.015+0000 7f93c9034b80 1 client.26179 shutdown 2023-08-25T09:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 -- 172.21.15.99:0/1325515541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24b860 msgr2=0x55bdaf24f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24b860 0x55bdaf24f150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93a8000f80 tx=0x7f93a8005b60 comp rx=0 tx=0).stop 2023-08-25T09:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 -- 172.21.15.99:0/1325515541 shutdown_connections 2023-08-25T09:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24b860 0x55bdaf24f150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf245990 0x55bdaf251720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bdaf25f1c0 0x55bdaf2615e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 --2- 172.21.15.99:0/1325515541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf24ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 -- 172.21.15.99:0/1325515541 >> 172.21.15.99:0/1325515541 conn(0x55bdaf1ba080 msgr2=0x55bdaf23a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f9392ff5700 0 client.26179 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 -- 172.21.15.99:0/1325515541 shutdown_connections 2023-08-25T09:10:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 1 -- 172.21.15.99:0/1325515541 wait complete. 2023-08-25T09:10:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.016+0000 7f93c9034b80 20 client.26179 trim_cache size 0 max 16384 2023-08-25T09:10:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93c9034b80 1 Processor -- start 2023-08-25T09:10:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93c9034b80 1 -- start start 2023-08-25T09:10:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf256240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf24e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93c9034b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf0198c0 con 0x55bdaf24b860 2023-08-25T09:10:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93c9034b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf23c960 con 0x55bdaf245990 2023-08-25T09:10:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93c9034b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf240c10 con 0x55bdaf24af40 2023-08-25T09:10:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93ae7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf256240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93ae7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf256240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:55708/0 (socket says 172.21.15.99:55708) 2023-08-25T09:10:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93aeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93ae7fc700 1 -- 172.21.15.99:0/3455214413 learned_addr learned my addr 172.21.15.99:0/3455214413 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.032+0000 7f93af7fe700 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf24e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f93ae7fc700 1 -- 172.21.15.99:0/3455214413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 msgr2=0x55bdaf245450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f93ae7fc700 1 -- 172.21.15.99:0/3455214413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 msgr2=0x55bdaf24e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf24e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f93ae7fc700 1 -- 172.21.15.99:0/3455214413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdaefacc00 con 0x55bdaf24af40 2023-08-25T09:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f93af7fe700 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf24e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f93aeffd700 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf256240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93a4098eb0 tx=0x7f93a4010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14aacf43616fcadd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f9395ffb700 1 -- 172.21.15.99:0/3455214413 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4098880 con 0x55bdaf24af40 2023-08-25T09:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f9395ffb700 1 -- 172.21.15.99:0/3455214413 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a4099620 con 0x55bdaf24af40 2023-08-25T09:10:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.033+0000 7f9395ffb700 1 -- 172.21.15.99:0/3455214413 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4098670 con 0x55bdaf24af40 2023-08-25T09:10:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 -- 172.21.15.99:0/3455214413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 msgr2=0x55bdaf256240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf256240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93a4098eb0 tx=0x7f93a4010c80 comp rx=0 tx=0).stop 2023-08-25T09:10:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 -- 172.21.15.99:0/3455214413 shutdown_connections 2023-08-25T09:10:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf24e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf245990 0x55bdaf245450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3455214413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf24af40 0x55bdaf256240 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 -- 172.21.15.99:0/3455214413 >> 172.21.15.99:0/3455214413 conn(0x55bdaf1ba080 msgr2=0x55bdaf25a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 -- 172.21.15.99:0/3455214413 shutdown_connections 2023-08-25T09:10:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 -- 172.21.15.99:0/3455214413 wait complete. 2023-08-25T09:10:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 -- start start 2023-08-25T09:10:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.034+0000 7f93c9034b80 1 Processor -- start 2023-08-25T09:10:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf245990 0x55bdaf246950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf246e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93c9034b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf247510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93c9034b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf247c20 con 0x55bdaf24b860 2023-08-25T09:10:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93c9034b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf247d90 con 0x55bdaf24af40 2023-08-25T09:10:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93c9034b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdaf247f50 con 0x55bdaf245990 2023-08-25T09:10:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93c9034b80 10 client.0 _ll_register_callbacks cb 0x7f93a80114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:10:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93aeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf245990 0x55bdaf246950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93ae7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf246e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93aeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf245990 0x55bdaf246950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:55710/0 (socket says 172.21.15.99:55710) 2023-08-25T09:10:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93aeffd700 1 -- 172.21.15.99:0/3169963031 learned_addr learned my addr 172.21.15.99:0/3169963031 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.035+0000 7f93af7fe700 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf247510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f93aeffd700 1 -- 172.21.15.99:0/3169963031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 msgr2=0x55bdaf246e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f93aeffd700 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf246e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f93aeffd700 1 -- 172.21.15.99:0/3169963031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 msgr2=0x55bdaf247510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f93aeffd700 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf247510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f93aeffd700 1 -- 172.21.15.99:0/3169963031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b010e080 con 0x55bdaf245990 2023-08-25T09:10:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f93aeffd700 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf245990 0x55bdaf246950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f93a00098e0 tx=0x7f93a0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e327e14f11f4ce server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f9392ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:10:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f9392ff5700 1 -- 172.21.15.99:0/3169963031 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a001d050 con 0x55bdaf245990 2023-08-25T09:10:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f93c9034b80 1 -- 172.21.15.99:0/3169963031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdaf238600 con 0x55bdaf245990 2023-08-25T09:10:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.036+0000 7f9392ff5700 1 -- 172.21.15.99:0/3169963031 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f93a0004500 con 0x55bdaf245990 2023-08-25T09:10:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f9392ff5700 1 -- 172.21.15.99:0/3169963031 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a0005360 con 0x55bdaf245990 2023-08-25T09:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93c9034b80 10 client.26185 fetch_fsmap learned FSMap version 38 2023-08-25T09:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93c9034b80 1 -- 172.21.15.99:0/3169963031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f93b01a5b50 con 0x55bdaf245990 2023-08-25T09:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f9392ff5700 1 -- 172.21.15.99:0/3169963031 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdaf238600 con 0x55bdaf245990 2023-08-25T09:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f9392ff5700 1 -- 172.21.15.99:0/3169963031 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93a000f6c0 con 0x55bdaf245990 2023-08-25T09:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93c9034b80 10 client.26185 fetch_fsmap finished waiting for FSMap version 38 2023-08-25T09:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93c9034b80 10 client.26185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93c9034b80 20 client.26185 populate_metadata read hostname 'smithi099' 2023-08-25T09:10:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bdaf2b3ce0 0x55bdaf2b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93c9034b80 4 client.26185 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:10:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93c9034b80 1 -- 172.21.15.99:0/3169963031 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdaf190910 con 0x55bdaf2b3ce0 2023-08-25T09:10:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.037+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bdaf2b3ce0 0x55bdaf2b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.038+0000 7f93ae7fc700 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bdaf2b3ce0 0x55bdaf2b6100 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa104dc68085ca83 server_cookie=601196e8455001d9 in_seq=0 out_seq=0 2023-08-25T09:10:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.038+0000 7f9392ff5700 10 client.26185 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f9392ff5700 1 -- 172.21.15.99:0/3169963031 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdaf190910 con 0x55bdaf2b3ce0 2023-08-25T09:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f9392ff5700 10 client.26185 handle_command_reply: tid=0 2023-08-25T09:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 client.26185 shutdown 2023-08-25T09:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 -- 172.21.15.99:0/3169963031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf245990 msgr2=0x55bdaf246950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf245990 0x55bdaf246950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f93a00098e0 tx=0x7f93a0009310 comp rx=0 tx=0).stop 2023-08-25T09:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 -- 172.21.15.99:0/3169963031 shutdown_connections 2023-08-25T09:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdaf24b860 0x55bdaf247510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdaf245990 0x55bdaf246950 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55bdaf2b3ce0 0x55bdaf2b6100 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 --2- 172.21.15.99:0/3169963031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdaf24af40 0x55bdaf246e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f9392ff5700 0 client.26185 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 -- 172.21.15.99:0/3169963031 >> 172.21.15.99:0/3169963031 conn(0x55bdaf1ba080 msgr2=0x55bdaf248b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 -- 172.21.15.99:0/3169963031 shutdown_connections 2023-08-25T09:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.127+0000 7f93c9034b80 1 -- 172.21.15.99:0/3169963031 wait complete. 2023-08-25T09:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93c9034b80 20 client.26185 trim_cache size 0 max 16384 2023-08-25T09:10:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:10:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:10:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:10:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:10:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 -- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93980a00c0 msgr2=0x7f93980a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93980a00c0 0x7f93980a2580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93a000b2e0 tx=0x7f93a0005fb0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 -- 172.21.15.99:0/712239485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b0101270 msgr2=0x7f93b0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b0101270 0x7f93b0073340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f93a400e9a0 tx=0x7f93a400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 -- 172.21.15.99:0/712239485 shutdown_connections 2023-08-25T09:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93980a00c0 0x7f93980a2580 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b01068c0 0x7f93b01a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b01008b0 0x7f93b0072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 --2- 172.21.15.99:0/712239485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b0101270 0x7f93b0073340 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 -- 172.21.15.99:0/712239485 >> 172.21.15.99:0/712239485 conn(0x7f93b00fc380 msgr2=0x7f93b00fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.128+0000 7f93b57ff700 1 -- 172.21.15.99:0/712239485 shutdown_connections 2023-08-25T09:10:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:35.129+0000 7f93b57ff700 1 -- 172.21.15.99:0/712239485 wait complete. 2023-08-25T09:10:35.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:35 smithi181 ceph-mon[131296]: pgmap v355: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 737 KiB/s wr, 81 op/s 2023-08-25T09:10:35.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:35 smithi099 ceph-mon[97641]: pgmap v355: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 737 KiB/s wr, 81 op/s 2023-08-25T09:10:35.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:35 smithi157 ceph-mon[130879]: pgmap v355: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 737 KiB/s wr, 81 op/s 2023-08-25T09:10:35.946 INFO:tasks.workunit.client.0.smithi099.stdout: 1 440305 3.85 MB/sec execute 521 sec latency 35.150 ms 2023-08-25T09:10:36.946 INFO:tasks.workunit.client.0.smithi099.stdout: 1 440924 3.85 MB/sec execute 522 sec latency 508.332 ms 2023-08-25T09:10:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:37 smithi181 ceph-mon[131296]: pgmap v356: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2023-08-25T09:10:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:37 smithi099 ceph-mon[97641]: pgmap v356: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2023-08-25T09:10:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:37 smithi157 ceph-mon[130879]: pgmap v356: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2023-08-25T09:10:37.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 440924 3.84 MB/sec execute 523 sec latency 1508.467 ms 2023-08-25T09:10:38.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 440924 3.83 MB/sec execute 524 sec latency 2508.575 ms 2023-08-25T09:10:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:39 smithi181 ceph-mon[131296]: pgmap v357: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 734 KiB/s wr, 72 op/s 2023-08-25T09:10:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:39 smithi099 ceph-mon[97641]: pgmap v357: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 734 KiB/s wr, 72 op/s 2023-08-25T09:10:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:39 smithi157 ceph-mon[130879]: pgmap v357: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 734 KiB/s wr, 72 op/s 2023-08-25T09:10:39.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 3.83 MB/sec execute 525 sec latency 3041.121 ms 2023-08-25T09:10:40.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 3.83 MB/sec execute 526 sec latency 1056.432 ms 2023-08-25T09:10:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:41 smithi181 ceph-mon[131296]: pgmap v358: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 734 KiB/s wr, 72 op/s 2023-08-25T09:10:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:41 smithi099 ceph-mon[97641]: pgmap v358: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 734 KiB/s wr, 72 op/s 2023-08-25T09:10:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:41 smithi157 ceph-mon[130879]: pgmap v358: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 734 KiB/s wr, 72 op/s 2023-08-25T09:10:41.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 3.82 MB/sec execute 527 sec latency 2056.594 ms 2023-08-25T09:10:42.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 3.81 MB/sec execute 528 sec latency 3056.702 ms 2023-08-25T09:10:43.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:43 smithi181 ceph-mon[131296]: pgmap v359: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 875 KiB/s wr, 92 op/s 2023-08-25T09:10:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:43 smithi099 ceph-mon[97641]: pgmap v359: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 875 KiB/s wr, 92 op/s 2023-08-25T09:10:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:43 smithi157 ceph-mon[130879]: pgmap v359: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 875 KiB/s wr, 92 op/s 2023-08-25T09:10:43.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 3.80 MB/sec execute 529 sec latency 4056.774 ms 2023-08-25T09:10:44.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 442615 3.81 MB/sec execute 530 sec latency 4591.310 ms 2023-08-25T09:10:45.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:45 smithi099 ceph-mon[97641]: pgmap v360: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 707 KiB/s wr, 75 op/s 2023-08-25T09:10:45.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:45 smithi157 ceph-mon[130879]: pgmap v360: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 707 KiB/s wr, 75 op/s 2023-08-25T09:10:45.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:45 smithi181 ceph-mon[131296]: pgmap v360: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 707 KiB/s wr, 75 op/s 2023-08-25T09:10:45.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 443950 3.82 MB/sec execute 531 sec latency 18.861 ms 2023-08-25T09:10:46.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 445096 3.82 MB/sec execute 532 sec latency 240.839 ms 2023-08-25T09:10:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:47 smithi181 ceph-mon[131296]: pgmap v361: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-08-25T09:10:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:47 smithi099 ceph-mon[97641]: pgmap v361: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-08-25T09:10:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:47 smithi157 ceph-mon[130879]: pgmap v361: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-08-25T09:10:47.947 INFO:tasks.workunit.client.0.smithi099.stdout: 1 445096 3.81 MB/sec execute 533 sec latency 1240.955 ms 2023-08-25T09:10:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 -- 172.21.15.99:0/2897891510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 msgr2=0x7f889410cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 --2- 172.21.15.99:0/2897891510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 0x7f889410cbb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8888009990 tx=0x7f8888009ca0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 -- 172.21.15.99:0/2897891510 shutdown_connections 2023-08-25T09:10:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 --2- 172.21.15.99:0/2897891510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f889410d0f0 0x7f889410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 --2- 172.21.15.99:0/2897891510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8894072d50 0x7f8894073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 --2- 172.21.15.99:0/2897891510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 0x7f889410cbb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 -- 172.21.15.99:0/2897891510 >> 172.21.15.99:0/2897891510 conn(0x7f88940fc380 msgr2=0x7f88940fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 -- 172.21.15.99:0/2897891510 shutdown_connections 2023-08-25T09:10:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.283+0000 7f8898fff700 1 -- 172.21.15.99:0/2897891510 wait complete. 2023-08-25T09:10:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.284+0000 7f8898fff700 1 Processor -- start 2023-08-25T09:10:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8898fff700 1 -- start start 2023-08-25T09:10:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8898fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8894072d50 0x7f889419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8898fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 0x7f889419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8898fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f889410d0f0 0x7f8894198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8898fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88941990c0 con 0x7f8894073670 2023-08-25T09:10:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8898fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894199230 con 0x7f8894072d50 2023-08-25T09:10:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8898fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88941993f0 con 0x7f889410d0f0 2023-08-25T09:10:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8891d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 0x7f889419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8891d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 0x7f889419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55354/0 (socket says 172.21.15.99:55354) 2023-08-25T09:10:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8891d9b700 1 -- 172.21.15.99:0/3840007448 learned_addr learned my addr 172.21.15.99:0/3840007448 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f8892d9d700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f889410d0f0 0x7f8894198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.285+0000 7f889259c700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8894072d50 0x7f889419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f8891d9b700 1 -- 172.21.15.99:0/3840007448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f889410d0f0 msgr2=0x7f8894198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f8891d9b700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f889410d0f0 0x7f8894198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f8891d9b700 1 -- 172.21.15.99:0/3840007448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8894072d50 msgr2=0x7f889419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f8891d9b700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8894072d50 0x7f889419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f8891d9b700 1 -- 172.21.15.99:0/3840007448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88880095f0 con 0x7f8894073670 2023-08-25T09:10:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f8891d9b700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 0x7f889419f130 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8888005910 tx=0x7f8888005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f88837fe700 1 -- 172.21.15.99:0/3840007448 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888801d070 con 0x7f8894073670 2023-08-25T09:10:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f88837fe700 1 -- 172.21.15.99:0/3840007448 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8888022470 con 0x7f8894073670 2023-08-25T09:10:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f8898fff700 1 -- 172.21.15.99:0/3840007448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8894199670 con 0x7f8894073670 2023-08-25T09:10:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.286+0000 7f88837fe700 1 -- 172.21.15.99:0/3840007448 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888017650 con 0x7f8894073670 2023-08-25T09:10:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.287+0000 7f8898fff700 1 -- 172.21.15.99:0/3840007448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88941a5be0 con 0x7f8894073670 2023-08-25T09:10:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.289+0000 7f88837fe700 1 -- 172.21.15.99:0/3840007448 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f888800fad0 con 0x7f8894073670 2023-08-25T09:10:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88ac880b80 1 Processor -- start 2023-08-25T09:10:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88ac880b80 1 -- start start 2023-08-25T09:10:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88837fe700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f887c0a00c0 0x7f887c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f5860 0x560e3f9ef450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88837fe700 1 -- 172.21.15.99:0/3840007448 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f88880ccbc0 con 0x7f8894073670 2023-08-25T09:10:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9ef990 0x560e3f9efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88ac880b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f957750 con 0x560e3f9f5860 2023-08-25T09:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88ac880b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f948720 con 0x560e3f9f4f40 2023-08-25T09:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f88ac880b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f946040 con 0x560e3f9ef990 2023-08-25T09:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.291+0000 7f8892d9d700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f887c0a00c0 0x7f887c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f8892d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9ef990 0x560e3f9efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f8891d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f5860 0x560e3f9ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f889259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f8892d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9ef990 0x560e3f9efdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:45418/0 (socket says 172.21.15.99:45418) 2023-08-25T09:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f8892d9d700 1 -- 172.21.15.99:0/2399713919 learned_addr learned my addr 172.21.15.99:0/2399713919 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f889259c700 1 -- 172.21.15.99:0/2399713919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9ef990 msgr2=0x560e3f9efdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f889259c700 1 --2- 172.21.15.99:0/2399713919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9ef990 0x560e3f9efdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f889259c700 1 -- 172.21.15.99:0/2399713919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f5860 msgr2=0x560e3f9ef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f889259c700 1 --2- 172.21.15.99:0/2399713919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f5860 0x560e3f9ef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f889259c700 1 -- 172.21.15.99:0/2399713919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e3f7c15d0 con 0x560e3f9f4f40 2023-08-25T09:10:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f8892d9d700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f887c0a00c0 0x7f887c0a2580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f888c00f910 tx=0x7f888c00fcd0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.292+0000 7f889259c700 1 --2- 172.21.15.99:0/2399713919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f5320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f888400bc80 tx=0x7f888400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86617d323b411bd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f8880ff9700 1 -- 172.21.15.99:0/2399713919 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884010c20 con 0x560e3f9f4f40 2023-08-25T09:10:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f8880ff9700 1 -- 172.21.15.99:0/2399713919 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8884015070 con 0x560e3f9f4f40 2023-08-25T09:10:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f8880ff9700 1 -- 172.21.15.99:0/2399713919 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884005020 con 0x560e3f9f4f40 2023-08-25T09:10:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f88ac880b80 1 -- 172.21.15.99:0/2399713919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 msgr2=0x560e3f9f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2399713919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f5320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f888400bc80 tx=0x7f888400bf90 comp rx=0 tx=0).stop 2023-08-25T09:10:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f88ac880b80 1 -- 172.21.15.99:0/2399713919 shutdown_connections 2023-08-25T09:10:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2399713919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9ef990 0x560e3f9efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2399713919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f5320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2399713919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f5860 0x560e3f9ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.293+0000 7f88ac880b80 1 -- 172.21.15.99:0/2399713919 >> 172.21.15.99:0/2399713919 conn(0x560e3f964080 msgr2=0x560e3f9e9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.294+0000 7f88ac880b80 1 -- 172.21.15.99:0/2399713919 shutdown_connections 2023-08-25T09:10:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.294+0000 7f88ac880b80 1 -- 172.21.15.99:0/2399713919 wait complete. 2023-08-25T09:10:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.294+0000 7f88ac880b80 1 -- start start 2023-08-25T09:10:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.294+0000 7f88ac880b80 1 Processor -- start 2023-08-25T09:10:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9ef990 0x560e3f9fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f4f40 0x560e3f9f8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f88ac880b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f941930 con 0x560e3f9f4f40 2023-08-25T09:10:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f88ac880b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f8dfb50 con 0x560e3f9ef990 2023-08-25T09:10:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f88ac880b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f8df850 con 0x560e3f9f5860 2023-08-25T09:10:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f88ac880b80 10 client.0 _ll_register_callbacks cb 0x560e3f829a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:10:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8891d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f4f40 0x560e3f9f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8891d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f4f40 0x560e3f9f8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55370/0 (socket says 172.21.15.99:55370) 2023-08-25T09:10:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f889259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9ef990 0x560e3f9fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8891d9b700 1 -- 172.21.15.99:0/1305285747 learned_addr learned my addr 172.21.15.99:0/1305285747 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8892d9d700 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8891d9b700 1 -- 172.21.15.99:0/1305285747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 msgr2=0x560e3f9f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8891d9b700 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8891d9b700 1 -- 172.21.15.99:0/1305285747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9ef990 msgr2=0x560e3f9fb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8891d9b700 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9ef990 0x560e3f9fb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.295+0000 7f8891d9b700 1 -- 172.21.15.99:0/1305285747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888400b8e0 con 0x560e3f9f4f40 2023-08-25T09:10:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f8891d9b700 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f4f40 0x560e3f9f8850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8888005f50 tx=0x7f8888099ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ebb7f6c34747e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f8879ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:10:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f8879ffb700 1 -- 172.21.15.99:0/1305285747 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888094ea0 con 0x560e3f9f4f40 2023-08-25T09:10:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f88ac880b80 1 -- 172.21.15.99:0/1305285747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e3f9bf160 con 0x560e3f9f4f40 2023-08-25T09:10:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f8879ffb700 1 -- 172.21.15.99:0/1305285747 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f888809a290 con 0x560e3f9f4f40 2023-08-25T09:10:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f8879ffb700 1 -- 172.21.15.99:0/1305285747 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888809e0c0 con 0x560e3f9f4f40 2023-08-25T09:10:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f88ac880b80 10 client.16152 fetch_fsmap learned FSMap version 38 2023-08-25T09:10:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f88ac880b80 1 -- 172.21.15.99:0/1305285747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560e3f76dd80 con 0x560e3f9f4f40 2023-08-25T09:10:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.296+0000 7f8879ffb700 1 -- 172.21.15.99:0/1305285747 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e3f9bf160 con 0x560e3f9f4f40 2023-08-25T09:10:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.297+0000 7f8879ffb700 1 -- 172.21.15.99:0/1305285747 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f88880b2220 con 0x560e3f9f4f40 2023-08-25T09:10:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.297+0000 7f88ac880b80 10 client.16152 fetch_fsmap finished waiting for FSMap version 38 2023-08-25T09:10:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.297+0000 7f88ac880b80 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.297+0000 7f88ac880b80 20 client.16152 populate_metadata read hostname 'smithi099' 2023-08-25T09:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.297+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560e3fa08dc0 0x560e3fa0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.297+0000 7f88ac880b80 4 client.16152 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.297+0000 7f88ac880b80 1 -- 172.21.15.99:0/1305285747 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e3f8e7b40 con 0x560e3fa08dc0 2023-08-25T09:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.297+0000 7f889259c700 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560e3fa08dc0 0x560e3fa0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f8879ffb700 10 client.16152 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f889259c700 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560e3fa08dc0 0x560e3fa0b1e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a776e66cc04ad9d4 server_cookie=1a8706d643881a5e in_seq=0 out_seq=0 2023-08-25T09:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f8879ffb700 1 -- 172.21.15.99:0/1305285747 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e3f8e7b40 con 0x560e3fa08dc0 2023-08-25T09:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f8879ffb700 10 client.16152 handle_command_reply: tid=0 2023-08-25T09:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 client.16152 shutdown 2023-08-25T09:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 -- 172.21.15.99:0/1305285747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f4f40 msgr2=0x560e3f9f8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f4f40 0x560e3f9f8850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8888005f50 tx=0x7f8888099ea0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 -- 172.21.15.99:0/1305285747 shutdown_connections 2023-08-25T09:10:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560e3fa08dc0 0x560e3fa0b1e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9ef990 0x560e3f9fb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f8879ffb700 0 client.16152 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1305285747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9f4f40 0x560e3f9f8850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.298+0000 7f88ac880b80 1 -- 172.21.15.99:0/1305285747 >> 172.21.15.99:0/1305285747 conn(0x560e3f964080 msgr2=0x560e3f9e4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.299+0000 7f88ac880b80 1 -- 172.21.15.99:0/1305285747 shutdown_connections 2023-08-25T09:10:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.299+0000 7f88ac880b80 1 -- 172.21.15.99:0/1305285747 wait complete. 2023-08-25T09:10:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.299+0000 7f88ac880b80 20 client.16152 trim_cache size 0 max 16384 2023-08-25T09:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f88ac880b80 1 Processor -- start 2023-08-25T09:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f88ac880b80 1 -- start start 2023-08-25T09:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9efd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3fa00240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f88ac880b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f7c38c0 con 0x560e3f9ef990 2023-08-25T09:10:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f88ac880b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f9e6700 con 0x560e3f9f4f40 2023-08-25T09:10:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f88ac880b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3fa00780 con 0x560e3f9f5860 2023-08-25T09:10:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f889259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f889259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9efd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55380/0 (socket says 172.21.15.99:55380) 2023-08-25T09:10:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f889259c700 1 -- 172.21.15.99:0/1803983154 learned_addr learned my addr 172.21.15.99:0/1803983154 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.314+0000 7f8892d9d700 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f8891d9b700 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3fa00240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f889259c700 1 -- 172.21.15.99:0/1803983154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 msgr2=0x560e3f9f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f889259c700 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f889259c700 1 -- 172.21.15.99:0/1803983154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 msgr2=0x560e3fa00240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f889259c700 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3fa00240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f889259c700 1 -- 172.21.15.99:0/1803983154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e3f756c00 con 0x560e3f9ef990 2023-08-25T09:10:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f889259c700 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9efd70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8884011010 tx=0x7f8884005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6dcf169cb9201d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f88817fa700 1 -- 172.21.15.99:0/1803983154 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884021040 con 0x560e3f9ef990 2023-08-25T09:10:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f88817fa700 1 -- 172.21.15.99:0/1803983154 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8884015040 con 0x560e3f9ef990 2023-08-25T09:10:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.315+0000 7f88817fa700 1 -- 172.21.15.99:0/1803983154 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884004040 con 0x560e3f9ef990 2023-08-25T09:10:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 -- 172.21.15.99:0/1803983154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 msgr2=0x560e3f9efd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9efd70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8884011010 tx=0x7f8884005970 comp rx=0 tx=0).stop 2023-08-25T09:10:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 -- 172.21.15.99:0/1803983154 shutdown_connections 2023-08-25T09:10:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9efd70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 --2- 172.21.15.99:0/1803983154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3fa00240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 -- 172.21.15.99:0/1803983154 >> 172.21.15.99:0/1803983154 conn(0x560e3f964080 msgr2=0x560e3f9e9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 -- 172.21.15.99:0/1803983154 shutdown_connections 2023-08-25T09:10:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 -- 172.21.15.99:0/1803983154 wait complete. 2023-08-25T09:10:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.316+0000 7f88ac880b80 1 -- start start 2023-08-25T09:10:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f88ac880b80 1 Processor -- start 2023-08-25T09:10:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f88ac880b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f88ac880b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f9f0ed0 con 0x560e3f9ef990 2023-08-25T09:10:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f88ac880b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f9f1040 con 0x560e3f9f4f40 2023-08-25T09:10:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f88ac880b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3f9f1200 con 0x560e3f9f5860 2023-08-25T09:10:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f88ac880b80 10 client.0 _ll_register_callbacks cb 0x7f888809a060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:10:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f889259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f8891d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f889259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9f1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55396/0 (socket says 172.21.15.99:55396) 2023-08-25T09:10:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f889259c700 1 -- 172.21.15.99:0/2197616631 learned_addr learned my addr 172.21.15.99:0/2197616631 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:10:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.317+0000 7f8892d9d700 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8891d9b700 1 -- 172.21.15.99:0/2197616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 msgr2=0x560e3f9f0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8891d9b700 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8891d9b700 1 -- 172.21.15.99:0/2197616631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 msgr2=0x560e3f9f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:10:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8891d9b700 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8891d9b700 1 -- 172.21.15.99:0/2197616631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e3f7c15d0 con 0x560e3f9f4f40 2023-08-25T09:10:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f889259c700 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:10:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8891d9b700 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f1ff0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8888099f90 tx=0x7f8888005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661aff0f54541850 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:10:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8879ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:10:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8879ffb700 1 -- 172.21.15.99:0/2197616631 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888094ea0 con 0x560e3f9f4f40 2023-08-25T09:10:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f88ac880b80 1 -- 172.21.15.99:0/2197616631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e3f9e2600 con 0x560e3f9f4f40 2023-08-25T09:10:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8879ffb700 1 -- 172.21.15.99:0/2197616631 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f888809a290 con 0x560e3f9f4f40 2023-08-25T09:10:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f8879ffb700 1 -- 172.21.15.99:0/2197616631 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888809e0c0 con 0x560e3f9f4f40 2023-08-25T09:10:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f88ac880b80 10 client.26219 fetch_fsmap learned FSMap version 38 2023-08-25T09:10:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.318+0000 7f88ac880b80 1 -- 172.21.15.99:0/2197616631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560e3f7941f0 con 0x560e3f9f4f40 2023-08-25T09:10:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f8879ffb700 1 -- 172.21.15.99:0/2197616631 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e3f9e2600 con 0x560e3f9f4f40 2023-08-25T09:10:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f8879ffb700 1 -- 172.21.15.99:0/2197616631 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f88880991f0 con 0x560e3f9f4f40 2023-08-25T09:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f88ac880b80 10 client.26219 fetch_fsmap finished waiting for FSMap version 38 2023-08-25T09:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f88ac880b80 10 client.26219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f88ac880b80 20 client.26219 populate_metadata read hostname 'smithi099' 2023-08-25T09:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560e3fa5d6a0 0x560e3fa5fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f88ac880b80 4 client.26219 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f88ac880b80 1 -- 172.21.15.99:0/2197616631 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e3f93a910 con 0x560e3fa5d6a0 2023-08-25T09:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f889259c700 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560e3fa5d6a0 0x560e3fa5fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f8879ffb700 10 client.26219 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.319+0000 7f889259c700 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560e3fa5d6a0 0x560e3fa5fa90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59d00351683adc server_cookie=37be33877159f825 in_seq=0 out_seq=0 2023-08-25T09:10:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.362+0000 7f8879ffb700 1 -- 172.21.15.99:0/2197616631 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e3f93a910 con 0x560e3fa5d6a0 2023-08-25T09:10:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.362+0000 7f8879ffb700 10 client.26219 handle_command_reply: tid=0 2023-08-25T09:10:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.362+0000 7f88ac880b80 1 client.26219 shutdown 2023-08-25T09:10:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 -- 172.21.15.99:0/2197616631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 msgr2=0x560e3f9f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f1ff0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8888099f90 tx=0x7f8888005fd0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 -- 172.21.15.99:0/2197616631 shutdown_connections 2023-08-25T09:10:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e3f9f5860 0x560e3f9f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e3f9ef990 0x560e3f9f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560e3fa5d6a0 0x560e3fa5fa90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 --2- 172.21.15.99:0/2197616631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e3f9f4f40 0x560e3f9f1ff0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 -- 172.21.15.99:0/2197616631 >> 172.21.15.99:0/2197616631 conn(0x560e3f964080 msgr2=0x560e3f9f5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f8879ffb700 0 client.26219 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:10:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 -- 172.21.15.99:0/2197616631 shutdown_connections 2023-08-25T09:10:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 1 -- 172.21.15.99:0/2197616631 wait complete. 2023-08-25T09:10:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.363+0000 7f88ac880b80 20 client.26219 trim_cache size 0 max 16384 2023-08-25T09:10:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:10:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:10:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:10:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:10:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 -- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f887c0a00c0 msgr2=0x7f887c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f887c0a00c0 0x7f887c0a2580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f888c00f910 tx=0x7f888c00fcd0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 -- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 msgr2=0x7f889419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:10:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 0x7f889419f130 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8888005910 tx=0x7f8888005740 comp rx=0 tx=0).stop 2023-08-25T09:10:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 -- 172.21.15.99:0/3840007448 shutdown_connections 2023-08-25T09:10:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f887c0a00c0 0x7f887c0a2580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f889410d0f0 0x7f8894198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8894072d50 0x7f889419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 --2- 172.21.15.99:0/3840007448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8894073670 0x7f889419f130 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:10:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 -- 172.21.15.99:0/3840007448 >> 172.21.15.99:0/3840007448 conn(0x7f88940fc380 msgr2=0x7f889410f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:10:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 -- 172.21.15.99:0/3840007448 shutdown_connections 2023-08-25T09:10:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:10:48.364+0000 7f8898fff700 1 -- 172.21.15.99:0/3840007448 wait complete. 2023-08-25T09:10:48.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 445096 3.80 MB/sec execute 534 sec latency 2241.107 ms 2023-08-25T09:10:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:49 smithi181 ceph-mon[131296]: pgmap v362: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 695 KiB/s wr, 57 op/s 2023-08-25T09:10:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:49 smithi099 ceph-mon[97641]: pgmap v362: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 695 KiB/s wr, 57 op/s 2023-08-25T09:10:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:49 smithi157 ceph-mon[130879]: pgmap v362: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 695 KiB/s wr, 57 op/s 2023-08-25T09:10:49.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 446172 3.81 MB/sec execute 535 sec latency 2774.839 ms 2023-08-25T09:10:50.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 447430 3.82 MB/sec execute 536 sec latency 19.756 ms 2023-08-25T09:10:51.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:51 smithi181 ceph-mon[131296]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 695 KiB/s wr, 57 op/s 2023-08-25T09:10:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:51 smithi099 ceph-mon[97641]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 695 KiB/s wr, 57 op/s 2023-08-25T09:10:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:51 smithi157 ceph-mon[130879]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 695 KiB/s wr, 57 op/s 2023-08-25T09:10:51.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448015 3.81 MB/sec execute 537 sec latency 583.184 ms 2023-08-25T09:10:52.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448015 3.81 MB/sec execute 538 sec latency 1583.332 ms 2023-08-25T09:10:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:53 smithi181 ceph-mon[131296]: pgmap v364: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-08-25T09:10:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:53 smithi099 ceph-mon[97641]: pgmap v364: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-08-25T09:10:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:53 smithi157 ceph-mon[130879]: pgmap v364: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-08-25T09:10:53.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448015 3.80 MB/sec execute 539 sec latency 2583.451 ms 2023-08-25T09:10:54.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448638 3.80 MB/sec execute 540 sec latency 3111.430 ms 2023-08-25T09:10:55.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:55 smithi099 ceph-mon[97641]: pgmap v365: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2023-08-25T09:10:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:55 smithi157 ceph-mon[130879]: pgmap v365: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2023-08-25T09:10:55.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:55 smithi181 ceph-mon[131296]: pgmap v365: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2023-08-25T09:10:55.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448638 3.79 MB/sec execute 541 sec latency 1062.191 ms 2023-08-25T09:10:56.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:56 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:10:56.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:56 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:10:56.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:56 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:10:56.948 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448638 3.78 MB/sec execute 542 sec latency 2062.326 ms 2023-08-25T09:10:57.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:57 smithi099 ceph-mon[97641]: pgmap v366: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2023-08-25T09:10:57.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:57 smithi157 ceph-mon[130879]: pgmap v366: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2023-08-25T09:10:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:57 smithi181 ceph-mon[131296]: pgmap v366: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2023-08-25T09:10:57.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448638 3.78 MB/sec execute 543 sec latency 3062.460 ms 2023-08-25T09:10:58.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448638 3.77 MB/sec execute 544 sec latency 4062.587 ms 2023-08-25T09:10:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:10:59 smithi181 ceph-mon[131296]: pgmap v367: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 936 KiB/s wr, 81 op/s 2023-08-25T09:10:59.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:10:59 smithi099 ceph-mon[97641]: pgmap v367: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 936 KiB/s wr, 81 op/s 2023-08-25T09:10:59.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:10:59 smithi157 ceph-mon[130879]: pgmap v367: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 936 KiB/s wr, 81 op/s 2023-08-25T09:10:59.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 449713 3.77 MB/sec execute 545 sec latency 4594.050 ms 2023-08-25T09:11:00.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 450767 3.78 MB/sec execute 546 sec latency 28.197 ms 2023-08-25T09:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 -- 172.21.15.99:0/305497826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 msgr2=0x7f61bc100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 --2- 172.21.15.99:0/305497826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 0x7f61bc100eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009990 tx=0x7f61b8009ca0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 -- 172.21.15.99:0/305497826 shutdown_connections 2023-08-25T09:11:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 --2- 172.21.15.99:0/305497826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc102bd0 0x7f61bc10d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 --2- 172.21.15.99:0/305497826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 0x7f61bc100eb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 --2- 172.21.15.99:0/305497826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61bc102210 0x7f61bc102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 -- 172.21.15.99:0/305497826 >> 172.21.15.99:0/305497826 conn(0x7f61bc0fc1e0 msgr2=0x7f61bc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 -- 172.21.15.99:0/305497826 shutdown_connections 2023-08-25T09:11:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.521+0000 7f61c299d700 1 -- 172.21.15.99:0/305497826 wait complete. 2023-08-25T09:11:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.522+0000 7f61c299d700 1 Processor -- start 2023-08-25T09:11:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.522+0000 7f61c299d700 1 -- start start 2023-08-25T09:11:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.522+0000 7f61c299d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 0x7f61bc072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.522+0000 7f61c299d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61bc102210 0x7f61bc073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.522+0000 7f61c299d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc102bd0 0x7f61bc1a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.522+0000 7f61c299d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc073990 con 0x7f61bc102210 2023-08-25T09:11:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.522+0000 7f61c299d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1a4ff0 con 0x7f61bc102bd0 2023-08-25T09:11:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.522+0000 7f61c299d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1a5130 con 0x7f61bc100ad0 2023-08-25T09:11:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 0x7f61bc072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 0x7f61bc072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:50056/0 (socket says 172.21.15.99:50056) 2023-08-25T09:11:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c119a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61bc102210 0x7f61bc073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 -- 172.21.15.99:0/1526818497 learned_addr learned my addr 172.21.15.99:0/1526818497 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c219c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc102bd0 0x7f61bc1a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 -- 172.21.15.99:0/1526818497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc102bd0 msgr2=0x7f61bc1a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc102bd0 0x7f61bc1a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 -- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61bc102210 msgr2=0x7f61bc073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61bc102210 0x7f61bc073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 -- 172.21.15.99:0/1526818497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4009c40 con 0x7f61bc100ad0 2023-08-25T09:11:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61c199b700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 0x7f61bc072c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61b8005910 tx=0x7f61b8004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.523+0000 7f61b2ffd700 1 -- 172.21.15.99:0/1526818497 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b801d070 con 0x7f61bc100ad0 2023-08-25T09:11:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.524+0000 7f61b2ffd700 1 -- 172.21.15.99:0/1526818497 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b8022470 con 0x7f61bc100ad0 2023-08-25T09:11:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.524+0000 7f61c299d700 1 -- 172.21.15.99:0/1526818497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b80095f0 con 0x7f61bc100ad0 2023-08-25T09:11:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.524+0000 7f61b2ffd700 1 -- 172.21.15.99:0/1526818497 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8017650 con 0x7f61bc100ad0 2023-08-25T09:11:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.524+0000 7f61c299d700 1 -- 172.21.15.99:0/1526818497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc1a5670 con 0x7f61bc100ad0 2023-08-25T09:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.527+0000 7f61b2ffd700 1 -- 172.21.15.99:0/1526818497 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f61b80225e0 con 0x7f61bc100ad0 2023-08-25T09:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.528+0000 7f61d745db80 1 Processor -- start 2023-08-25T09:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.528+0000 7f61b2ffd700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f61a80a0140 0x7f61a80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.528+0000 7f61b2ffd700 1 -- 172.21.15.99:0/1526818497 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f61b80cc900 con 0x7f61bc100ad0 2023-08-25T09:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.528+0000 7f61d745db80 1 -- start start 2023-08-25T09:11:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c219c700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f61a80a0140 0x7f61a80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61d745db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x5623325bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61d745db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc930 0x5623325b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61d745db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 0x5623325b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61d745db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233251e750 con 0x5623325b6990 2023-08-25T09:11:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61d745db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233250f720 con 0x5623325bc010 2023-08-25T09:11:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61d745db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233250d040 con 0x5623325bc930 2023-08-25T09:11:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c199b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x5623325bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c199b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x5623325bc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:60694/0 (socket says 172.21.15.99:60694) 2023-08-25T09:11:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c199b700 1 -- 172.21.15.99:0/2337188519 learned_addr learned my addr 172.21.15.99:0/2337188519 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c119a700 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc930 0x5623325b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c219c700 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 0x5623325b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c199b700 1 -- 172.21.15.99:0/2337188519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc930 msgr2=0x5623325b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c199b700 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc930 0x5623325b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c199b700 1 -- 172.21.15.99:0/2337188519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 msgr2=0x5623325b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c219c700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f61a80a0140 0x7f61a80a2600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61bc074d30 tx=0x7f61b4009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c199b700 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 0x5623325b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.529+0000 7f61c199b700 1 -- 172.21.15.99:0/2337188519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623323885d0 con 0x5623325bc010 2023-08-25T09:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.530+0000 7f61c199b700 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x5623325bc3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61b8006010 tx=0x7f61b8099da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ecfb849d36cd0c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.530+0000 7f61a7fff700 1 -- 172.21.15.99:0/2337188519 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b809a6d0 con 0x5623325bc010 2023-08-25T09:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.530+0000 7f61a7fff700 1 -- 172.21.15.99:0/2337188519 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b8099fd0 con 0x5623325bc010 2023-08-25T09:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.530+0000 7f61a7fff700 1 -- 172.21.15.99:0/2337188519 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8094e40 con 0x5623325bc010 2023-08-25T09:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.530+0000 7f61d745db80 1 -- 172.21.15.99:0/2337188519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 msgr2=0x5623325bc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.530+0000 7f61d745db80 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x5623325bc3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61b8006010 tx=0x7f61b8099da0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 -- 172.21.15.99:0/2337188519 shutdown_connections 2023-08-25T09:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 0x5623325b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x5623325bc3f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 --2- 172.21.15.99:0/2337188519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc930 0x5623325b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 -- 172.21.15.99:0/2337188519 >> 172.21.15.99:0/2337188519 conn(0x56233252b080 msgr2=0x5623325b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 -- 172.21.15.99:0/2337188519 shutdown_connections 2023-08-25T09:11:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 -- 172.21.15.99:0/2337188519 wait complete. 2023-08-25T09:11:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 -- start start 2023-08-25T09:11:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.531+0000 7f61d745db80 1 Processor -- start 2023-08-25T09:11:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61d745db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325b6990 0x5623325bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61d745db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61d745db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x5623325c0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61d745db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562332508930 con 0x5623325bc930 2023-08-25T09:11:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61d745db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623324a6b50 con 0x5623325b6990 2023-08-25T09:11:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61d745db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623324a6850 con 0x5623325bc010 2023-08-25T09:11:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61d745db80 10 client.0 _ll_register_callbacks cb 0x5623323f0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61c219c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x5623325c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61c199b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325b6990 0x5623325bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61c119a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61c199b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325b6990 0x5623325bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:60698/0 (socket says 172.21.15.99:60698) 2023-08-25T09:11:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61c219c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x5623325c0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60936/0 (socket says 172.21.15.99:60936) 2023-08-25T09:11:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61c199b700 1 -- 172.21.15.99:0/2997662997 learned_addr learned my addr 172.21.15.99:0/2997662997 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.532+0000 7f61c119a700 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:50070/0 (socket says 172.21.15.99:50070) 2023-08-25T09:11:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61c199b700 1 -- 172.21.15.99:0/2997662997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 msgr2=0x5623325c00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61c199b700 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61c199b700 1 -- 172.21.15.99:0/2997662997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 msgr2=0x5623325c0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61c199b700 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x5623325c0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61c199b700 1 -- 172.21.15.99:0/2997662997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc1a5670 con 0x5623325b6990 2023-08-25T09:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61c199b700 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325b6990 0x5623325bfb90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61b80964d0 tx=0x7f61b8022b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c123769379f0d617 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61a57fa700 1 -- 172.21.15.99:0/2997662997 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80c5020 con 0x5623325b6990 2023-08-25T09:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61a57fa700 1 -- 172.21.15.99:0/2997662997 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f61b80a1d50 con 0x5623325b6990 2023-08-25T09:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61d745db80 1 -- 172.21.15.99:0/2997662997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562332586160 con 0x5623325b6990 2023-08-25T09:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61a57fa700 1 -- 172.21.15.99:0/2997662997 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b809a6d0 con 0x5623325b6990 2023-08-25T09:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61d745db80 10 client.26231 fetch_fsmap learned FSMap version 39 2023-08-25T09:11:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61d745db80 1 -- 172.21.15.99:0/2997662997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562332334d80 con 0x5623325b6990 2023-08-25T09:11:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.533+0000 7f61a57fa700 1 -- 172.21.15.99:0/2997662997 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562332586160 con 0x5623325b6990 2023-08-25T09:11:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.534+0000 7f61a57fa700 1 -- 172.21.15.99:0/2997662997 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f61b80a18c0 con 0x5623325b6990 2023-08-25T09:11:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.534+0000 7f61d745db80 10 client.26231 fetch_fsmap finished waiting for FSMap version 39 2023-08-25T09:11:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.534+0000 7f61d745db80 10 client.26231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.534+0000 7f61d745db80 20 client.26231 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.534+0000 7f61d745db80 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5623325cfcc0 0x5623325d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.534+0000 7f61d745db80 4 client.26231 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:11:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.534+0000 7f61d745db80 1 -- 172.21.15.99:0/2997662997 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623324aeb40 con 0x5623325cfcc0 2023-08-25T09:11:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.534+0000 7f61c119a700 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5623325cfcc0 0x5623325d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.535+0000 7f61c119a700 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5623325cfcc0 0x5623325d20e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b8519d890b39ff6 server_cookie=23ff0ebdec46a88 in_seq=0 out_seq=0 2023-08-25T09:11:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.535+0000 7f61a57fa700 10 client.26231 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61a57fa700 1 -- 172.21.15.99:0/2997662997 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5623324aeb40 con 0x5623325cfcc0 2023-08-25T09:11:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61a57fa700 10 client.26231 handle_command_reply: tid=0 2023-08-25T09:11:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 client.26231 shutdown 2023-08-25T09:11:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 -- 172.21.15.99:0/2997662997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325b6990 msgr2=0x5623325bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325b6990 0x5623325bfb90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61b80964d0 tx=0x7f61b8022b30 comp rx=0 tx=0).stop 2023-08-25T09:11:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 -- 172.21.15.99:0/2997662997 shutdown_connections 2023-08-25T09:11:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x5623325c0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5623325cfcc0 0x5623325d20e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325b6990 0x5623325bfb90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 --2- 172.21.15.99:0/2997662997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61a57fa700 0 client.26231 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 -- 172.21.15.99:0/2997662997 >> 172.21.15.99:0/2997662997 conn(0x56233252b080 msgr2=0x5623325b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 -- 172.21.15.99:0/2997662997 shutdown_connections 2023-08-25T09:11:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 1 -- 172.21.15.99:0/2997662997 wait complete. 2023-08-25T09:11:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.536+0000 7f61d745db80 20 client.26231 trim_cache size 0 max 16384 2023-08-25T09:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.553+0000 7f61d745db80 1 Processor -- start 2023-08-25T09:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.553+0000 7f61d745db80 1 -- start start 2023-08-25T09:11:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.553+0000 7f61d745db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 0x5623325c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.553+0000 7f61d745db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.553+0000 7f61d745db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc930 0x5623325b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.553+0000 7f61d745db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233238a8c0 con 0x5623325b6990 2023-08-25T09:11:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.553+0000 7f61d745db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623325bc7d0 con 0x5623325bc930 2023-08-25T09:11:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.553+0000 7f61d745db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623325c2ea0 con 0x5623325bc010 2023-08-25T09:11:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c219c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc930 0x5623325b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c119a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c219c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc930 0x5623325b9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:60714/0 (socket says 172.21.15.99:60714) 2023-08-25T09:11:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c219c700 1 -- 172.21.15.99:0/218698485 learned_addr learned my addr 172.21.15.99:0/218698485 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c199b700 1 --2- 172.21.15.99:0/218698485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 0x5623325c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c119a700 1 -- 172.21.15.99:0/218698485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc930 msgr2=0x5623325b9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c119a700 1 --2- 172.21.15.99:0/218698485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc930 0x5623325b9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c119a700 1 -- 172.21.15.99:0/218698485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 msgr2=0x5623325c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c119a700 1 --2- 172.21.15.99:0/218698485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 0x5623325c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c119a700 1 -- 172.21.15.99:0/218698485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56233231dc00 con 0x5623325bc010 2023-08-25T09:11:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.554+0000 7f61c119a700 1 --2- 172.21.15.99:0/218698485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c0280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5623325d2980 tx=0x7f61ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa7c8440ffe206e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.555+0000 7f61b0bff700 1 -- 172.21.15.99:0/218698485 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac017070 con 0x5623325bc010 2023-08-25T09:11:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.555+0000 7f61b0bff700 1 -- 172.21.15.99:0/218698485 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61ac018040 con 0x5623325bc010 2023-08-25T09:11:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.555+0000 7f61b0bff700 1 -- 172.21.15.99:0/218698485 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac010690 con 0x5623325bc010 2023-08-25T09:11:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 -- 172.21.15.99:0/218698485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 msgr2=0x5623325c0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 --2- 172.21.15.99:0/218698485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c0280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5623325d2980 tx=0x7f61ac009260 comp rx=0 tx=0).stop 2023-08-25T09:11:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 -- 172.21.15.99:0/218698485 shutdown_connections 2023-08-25T09:11:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 --2- 172.21.15.99:0/218698485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc930 0x5623325b9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 --2- 172.21.15.99:0/218698485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325b6990 0x5623325c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 --2- 172.21.15.99:0/218698485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325bc010 0x5623325c0280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 -- 172.21.15.99:0/218698485 >> 172.21.15.99:0/218698485 conn(0x56233252b080 msgr2=0x5623325b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 -- 172.21.15.99:0/218698485 shutdown_connections 2023-08-25T09:11:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 -- 172.21.15.99:0/218698485 wait complete. 2023-08-25T09:11:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.556+0000 7f61d745db80 1 -- start start 2023-08-25T09:11:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61d745db80 1 Processor -- start 2023-08-25T09:11:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61d745db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325b6990 0x56233261be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61d745db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x56233261c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61d745db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x562332617a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61d745db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562332618180 con 0x5623325bc930 2023-08-25T09:11:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61d745db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623326182f0 con 0x5623325bc010 2023-08-25T09:11:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61d745db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623326184b0 con 0x5623325b6990 2023-08-25T09:11:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61d745db80 10 client.0 _ll_register_callbacks cb 0x7f61b400ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.557+0000 7f61c199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325b6990 0x56233261be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325b6990 0x56233261be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:50102/0 (socket says 172.21.15.99:50102) 2023-08-25T09:11:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c199b700 1 -- 172.21.15.99:0/1381940369 learned_addr learned my addr 172.21.15.99:0/1381940369 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c119a700 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x56233261c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c219c700 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x562332617a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c199b700 1 -- 172.21.15.99:0/1381940369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 msgr2=0x56233261c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c199b700 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x56233261c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c199b700 1 -- 172.21.15.99:0/1381940369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 msgr2=0x562332617a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c199b700 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x562332617a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c199b700 1 -- 172.21.15.99:0/1381940369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac009b30 con 0x5623325b6990 2023-08-25T09:11:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61c199b700 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325b6990 0x56233261be90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61b80228b0 tx=0x7f61b80ca060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3488b61924b42769 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:11:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61a57fa700 1 -- 172.21.15.99:0/1381940369 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80c5070 con 0x5623325b6990 2023-08-25T09:11:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61a57fa700 1 -- 172.21.15.99:0/1381940369 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f61b809a6d0 con 0x5623325b6990 2023-08-25T09:11:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.558+0000 7f61d745db80 1 -- 172.21.15.99:0/1381940369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623325a9600 con 0x5623325b6990 2023-08-25T09:11:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61a57fa700 1 -- 172.21.15.99:0/1381940369 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b8096180 con 0x5623325b6990 2023-08-25T09:11:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61d745db80 10 client.26221 fetch_fsmap learned FSMap version 39 2023-08-25T09:11:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61d745db80 1 -- 172.21.15.99:0/1381940369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56233235b1f0 con 0x5623325b6990 2023-08-25T09:11:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61a57fa700 1 -- 172.21.15.99:0/1381940369 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623325a9600 con 0x5623325b6990 2023-08-25T09:11:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61a57fa700 1 -- 172.21.15.99:0/1381940369 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f61b80a18c0 con 0x5623325b6990 2023-08-25T09:11:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61d745db80 10 client.26221 fetch_fsmap finished waiting for FSMap version 39 2023-08-25T09:11:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61d745db80 10 client.26221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61d745db80 20 client.26221 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61d745db80 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562332624900 0x562332626cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61d745db80 4 client.26221 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:11:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.559+0000 7f61d745db80 1 -- 172.21.15.99:0/1381940369 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562332501910 con 0x562332624900 2023-08-25T09:11:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.560+0000 7f61c119a700 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562332624900 0x562332626cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.560+0000 7f61a57fa700 10 client.26221 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.560+0000 7f61c119a700 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562332624900 0x562332626cf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad076dead183faf server_cookie=1878f7307975a046 in_seq=0 out_seq=0 2023-08-25T09:11:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61a57fa700 1 -- 172.21.15.99:0/1381940369 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562332501910 con 0x562332624900 2023-08-25T09:11:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61a57fa700 10 client.26221 handle_command_reply: tid=0 2023-08-25T09:11:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 client.26221 shutdown 2023-08-25T09:11:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 -- 172.21.15.99:0/1381940369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325b6990 msgr2=0x56233261be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325b6990 0x56233261be90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61b80228b0 tx=0x7f61b80ca060 comp rx=0 tx=0).stop 2023-08-25T09:11:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 -- 172.21.15.99:0/1381940369 shutdown_connections 2023-08-25T09:11:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623325bc930 0x562332617a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623325b6990 0x56233261be90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562332624900 0x562332626cf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 --2- 172.21.15.99:0/1381940369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623325bc010 0x56233261c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61a57fa700 0 client.26221 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 -- 172.21.15.99:0/1381940369 >> 172.21.15.99:0/1381940369 conn(0x56233252b080 msgr2=0x5623325b9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 -- 172.21.15.99:0/1381940369 shutdown_connections 2023-08-25T09:11:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.600+0000 7f61d745db80 1 -- 172.21.15.99:0/1381940369 wait complete. 2023-08-25T09:11:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61d745db80 20 client.26221 trim_cache size 0 max 16384 2023-08-25T09:11:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:11:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:11:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:11:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:11:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 -- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f61a80a0140 msgr2=0x7f61a80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f61a80a0140 0x7f61a80a2600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61bc074d30 tx=0x7f61b4009920 comp rx=0 tx=0).stop 2023-08-25T09:11:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 -- 172.21.15.99:0/1526818497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 msgr2=0x7f61bc072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 0x7f61bc072c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61b8005910 tx=0x7f61b8004710 comp rx=0 tx=0).stop 2023-08-25T09:11:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 -- 172.21.15.99:0/1526818497 shutdown_connections 2023-08-25T09:11:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f61a80a0140 0x7f61a80a2600 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc102bd0 0x7f61bc1a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc100ad0 0x7f61bc072c20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 --2- 172.21.15.99:0/1526818497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61bc102210 0x7f61bc073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 -- 172.21.15.99:0/1526818497 >> 172.21.15.99:0/1526818497 conn(0x7f61bc0fc1e0 msgr2=0x7f61bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 -- 172.21.15.99:0/1526818497 shutdown_connections 2023-08-25T09:11:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:01.601+0000 7f61c299d700 1 -- 172.21.15.99:0/1526818497 wait complete. 2023-08-25T09:11:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:01 smithi181 ceph-mon[131296]: pgmap v368: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 936 KiB/s wr, 81 op/s 2023-08-25T09:11:01.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:01 smithi099 ceph-mon[97641]: pgmap v368: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 936 KiB/s wr, 81 op/s 2023-08-25T09:11:01.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:01 smithi157 ceph-mon[130879]: pgmap v368: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 936 KiB/s wr, 81 op/s 2023-08-25T09:11:01.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 451943 3.78 MB/sec execute 547 sec latency 17.804 ms 2023-08-25T09:11:02.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 452187 3.78 MB/sec execute 548 sec latency 817.551 ms 2023-08-25T09:11:03.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:03 smithi181 ceph-mon[131296]: pgmap v369: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-08-25T09:11:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:03 smithi099 ceph-mon[97641]: pgmap v369: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-08-25T09:11:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:03 smithi157 ceph-mon[130879]: pgmap v369: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-08-25T09:11:03.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 452187 3.77 MB/sec execute 549 sec latency 1817.635 ms 2023-08-25T09:11:04.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:04 smithi181 ceph-mon[131296]: osdmap e99: 12 total, 12 up, 12 in 2023-08-25T09:11:04.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:04 smithi099 ceph-mon[97641]: osdmap e99: 12 total, 12 up, 12 in 2023-08-25T09:11:04.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:04 smithi157 ceph-mon[130879]: osdmap e99: 12 total, 12 up, 12 in 2023-08-25T09:11:04.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 453371 3.77 MB/sec execute 550 sec latency 2350.410 ms 2023-08-25T09:11:05.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:05 smithi181 ceph-mon[131296]: pgmap v371: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 834 KiB/s wr, 102 op/s 2023-08-25T09:11:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:05 smithi099 ceph-mon[97641]: pgmap v371: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 834 KiB/s wr, 102 op/s 2023-08-25T09:11:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:05 smithi157 ceph-mon[130879]: pgmap v371: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 834 KiB/s wr, 102 op/s 2023-08-25T09:11:05.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 454617 3.78 MB/sec execute 551 sec latency 25.496 ms 2023-08-25T09:11:06.949 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455106 3.78 MB/sec execute 552 sec latency 677.016 ms 2023-08-25T09:11:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:07 smithi181 ceph-mon[131296]: pgmap v372: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2023-08-25T09:11:07.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:07 smithi099 ceph-mon[97641]: pgmap v372: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2023-08-25T09:11:07.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:07 smithi157 ceph-mon[130879]: pgmap v372: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2023-08-25T09:11:07.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455106 3.77 MB/sec execute 553 sec latency 1677.163 ms 2023-08-25T09:11:08.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455106 3.77 MB/sec execute 554 sec latency 2677.298 ms 2023-08-25T09:11:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:09 smithi181 ceph-mon[131296]: pgmap v373: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2023-08-25T09:11:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:09 smithi181 ceph-mon[131296]: osdmap e100: 12 total, 12 up, 12 in 2023-08-25T09:11:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:09 smithi099 ceph-mon[97641]: pgmap v373: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2023-08-25T09:11:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:09 smithi099 ceph-mon[97641]: osdmap e100: 12 total, 12 up, 12 in 2023-08-25T09:11:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:09 smithi157 ceph-mon[130879]: pgmap v373: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2023-08-25T09:11:09.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:09 smithi157 ceph-mon[130879]: osdmap e100: 12 total, 12 up, 12 in 2023-08-25T09:11:09.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455730 3.76 MB/sec execute 555 sec latency 3209.110 ms 2023-08-25T09:11:10.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455748 3.76 MB/sec execute 556 sec latency 1038.893 ms 2023-08-25T09:11:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:11 smithi181 ceph-mon[131296]: pgmap v375: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-08-25T09:11:11.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:11 smithi099 ceph-mon[97641]: pgmap v375: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-08-25T09:11:11.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:11 smithi157 ceph-mon[130879]: pgmap v375: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-08-25T09:11:11.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 457862 3.78 MB/sec execute 557 sec latency 15.530 ms 2023-08-25T09:11:12.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 458583 3.79 MB/sec execute 558 sec latency 61.081 ms 2023-08-25T09:11:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:13 smithi181 ceph-mon[131296]: pgmap v376: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2023-08-25T09:11:13.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:13 smithi099 ceph-mon[97641]: pgmap v376: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2023-08-25T09:11:13.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:13 smithi157 ceph-mon[130879]: pgmap v376: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2023-08-25T09:11:13.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 459929 3.81 MB/sec execute 559 sec latency 17.543 ms 2023-08-25T09:11:14.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:14 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:11:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 -- 172.21.15.99:0/288776888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c101260 msgr2=0x7f2e1c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 --2- 172.21.15.99:0/288776888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c101260 0x7f2e1c10d770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400b0a0 tx=0x7f2e1400b3b0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 -- 172.21.15.99:0/288776888 shutdown_connections 2023-08-25T09:11:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 --2- 172.21.15.99:0/288776888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c101260 0x7f2e1c10d770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 --2- 172.21.15.99:0/288776888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c1068b0 0x7f2e1c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 --2- 172.21.15.99:0/288776888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e1c1008a0 0x7f2e1c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 -- 172.21.15.99:0/288776888 >> 172.21.15.99:0/288776888 conn(0x7f2e1c0fc370 msgr2=0x7f2e1c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 -- 172.21.15.99:0/288776888 shutdown_connections 2023-08-25T09:11:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.763+0000 7f2e211ff700 1 -- 172.21.15.99:0/288776888 wait complete. 2023-08-25T09:11:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.764+0000 7f2e211ff700 1 Processor -- start 2023-08-25T09:11:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.764+0000 7f2e211ff700 1 -- start start 2023-08-25T09:11:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e211ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c1008a0 0x7f2e1c072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e211ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c101260 0x7f2e1c073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e211ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e1c1068b0 0x7f2e1c1a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e211ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c1a4f90 con 0x7f2e1c1068b0 2023-08-25T09:11:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e211ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c1a50d0 con 0x7f2e1c1008a0 2023-08-25T09:11:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e211ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c1a5240 con 0x7f2e1c101260 2023-08-25T09:11:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e1ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c1008a0 0x7f2e1c072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e1ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c1008a0 0x7f2e1c072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:35764/0 (socket says 172.21.15.99:35764) 2023-08-25T09:11:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e1ad9d700 1 -- 172.21.15.99:0/1597972016 learned_addr learned my addr 172.21.15.99:0/1597972016 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c101260 0x7f2e1c073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.765+0000 7f2e1b59e700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e1c1068b0 0x7f2e1c1a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e1ad9d700 1 -- 172.21.15.99:0/1597972016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c101260 msgr2=0x7f2e1c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c101260 0x7f2e1c073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e1ad9d700 1 -- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e1c1068b0 msgr2=0x7f2e1c1a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e1c1068b0 0x7f2e1c1a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e1ad9d700 1 -- 172.21.15.99:0/1597972016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10009520 con 0x7f2e1c1008a0 2023-08-25T09:11:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c101260 0x7f2e1c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:11:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c1008a0 0x7f2e1c072e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00e9a0 tx=0x7f2e0c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e03fff700 1 -- 172.21.15.99:0/1597972016 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c009790 con 0x7f2e1c1008a0 2023-08-25T09:11:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e211ff700 1 -- 172.21.15.99:0/1597972016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e14009d00 con 0x7f2e1c1008a0 2023-08-25T09:11:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e03fff700 1 -- 172.21.15.99:0/1597972016 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e0c010430 con 0x7f2e1c1008a0 2023-08-25T09:11:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e03fff700 1 -- 172.21.15.99:0/1597972016 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c018680 con 0x7f2e1c1008a0 2023-08-25T09:11:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.766+0000 7f2e211ff700 1 -- 172.21.15.99:0/1597972016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c1a58a0 con 0x7f2e1c1008a0 2023-08-25T09:11:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.770+0000 7f2e03fff700 1 -- 172.21.15.99:0/1597972016 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f2e0c0105a0 con 0x7f2e1c1008a0 2023-08-25T09:11:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.772+0000 7f2e03fff700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e040a00c0 0x7f2e040a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.772+0000 7f2e34a92b80 1 Processor -- start 2023-08-25T09:11:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.772+0000 7f2e03fff700 1 -- 172.21.15.99:0/1597972016 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2e0c014070 con 0x7f2e1c1008a0 2023-08-25T09:11:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.772+0000 7f2e34a92b80 1 -- start start 2023-08-25T09:11:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.772+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e040a00c0 0x7f2e040a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 0x560cc79f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7953750 con 0x560cc79f0f40 2023-08-25T09:11:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7944720 con 0x560cc79f1860 2023-08-25T09:11:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7942040 con 0x560cc79eb990 2023-08-25T09:11:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79ebdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34158/0 (socket says 172.21.15.99:34158) 2023-08-25T09:11:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1b59e700 1 -- 172.21.15.99:0/1334659146 learned_addr learned my addr 172.21.15.99:0/1334659146 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e040a00c0 0x7f2e040a2580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009fd0 tx=0x7f2e10009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 0x560cc79f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1b59e700 1 -- 172.21.15.99:0/1334659146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 msgr2=0x560cc79eb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1b59e700 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79eb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1b59e700 1 -- 172.21.15.99:0/1334659146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 msgr2=0x560cc79f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1b59e700 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 0x560cc79f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.773+0000 7f2e1b59e700 1 -- 172.21.15.99:0/1334659146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc77bd5d0 con 0x560cc79eb990 2023-08-25T09:11:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.774+0000 7f2e1b59e700 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79ebdd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e14006130 tx=0x7f2e14009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772422a46e2df381 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.774+0000 7f2e017fa700 1 -- 172.21.15.99:0/1334659146 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400e040 con 0x560cc79eb990 2023-08-25T09:11:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.774+0000 7f2e017fa700 1 -- 172.21.15.99:0/1334659146 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e14009b80 con 0x560cc79eb990 2023-08-25T09:11:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.774+0000 7f2e017fa700 1 -- 172.21.15.99:0/1334659146 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1401c6a0 con 0x560cc79eb990 2023-08-25T09:11:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.774+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1334659146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 msgr2=0x560cc79ebdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.774+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79ebdd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e14006130 tx=0x7f2e14009450 comp rx=0 tx=0).stop 2023-08-25T09:11:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1334659146 shutdown_connections 2023-08-25T09:11:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79ebdd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 0x560cc79f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1334659146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1334659146 >> 172.21.15.99:0/1334659146 conn(0x560cc7960080 msgr2=0x560cc79e5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1334659146 shutdown_connections 2023-08-25T09:11:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1334659146 wait complete. 2023-08-25T09:11:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 -- start start 2023-08-25T09:11:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.775+0000 7f2e34a92b80 1 Processor -- start 2023-08-25T09:11:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 0x560cc79f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc793d930 con 0x560cc79f0f40 2023-08-25T09:11:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc78dbb50 con 0x560cc79f1860 2023-08-25T09:11:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc78db850 con 0x560cc79eb990 2023-08-25T09:11:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e1ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e34a92b80 10 client.0 _ll_register_callbacks cb 0x560cc7825a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e1b59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e1ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34174/0 (socket says 172.21.15.99:34174) 2023-08-25T09:11:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e1ad9d700 1 -- 172.21.15.99:0/1122419381 learned_addr learned my addr 172.21.15.99:0/1122419381 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.776+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 0x560cc79f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e1ad9d700 1 -- 172.21.15.99:0/1122419381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 msgr2=0x560cc79f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e1ad9d700 1 -- 172.21.15.99:0/1122419381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 msgr2=0x560cc79f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 0x560cc79f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e1ad9d700 1 -- 172.21.15.99:0/1122419381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c10e070 con 0x560cc79eb990 2023-08-25T09:11:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79f75f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00c2d0 tx=0x7f2e0c098b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff086730853fbee7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2dfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:11:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1122419381 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c099ea0 con 0x560cc79eb990 2023-08-25T09:11:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1122419381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cc79bb160 con 0x560cc79eb990 2023-08-25T09:11:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1122419381 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2e0c09ccc0 con 0x560cc79eb990 2023-08-25T09:11:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1122419381 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e0c0a0ca0 con 0x560cc79eb990 2023-08-25T09:11:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e34a92b80 10 client.26233 fetch_fsmap learned FSMap version 40 2023-08-25T09:11:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1122419381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560cc7769d80 con 0x560cc79eb990 2023-08-25T09:11:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1122419381 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560cc79bb160 con 0x560cc79eb990 2023-08-25T09:11:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.777+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1122419381 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e0c0a0910 con 0x560cc79eb990 2023-08-25T09:11:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2e34a92b80 10 client.26233 fetch_fsmap finished waiting for FSMap version 40 2023-08-25T09:11:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2e34a92b80 10 client.26233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2e34a92b80 20 client.26233 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560cc7a05170 0x560cc7a07590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2e34a92b80 4 client.26233 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:11:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1122419381 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cc78e3b40 con 0x560cc7a05170 2023-08-25T09:11:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560cc7a05170 0x560cc7a07590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2dfeff5700 10 client.26233 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.778+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560cc7a05170 0x560cc7a07590 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c37ca4cb4a63fdb server_cookie=eb234be4080d5f3c in_seq=0 out_seq=0 2023-08-25T09:11:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1122419381 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560cc78e3b40 con 0x560cc7a05170 2023-08-25T09:11:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2dfeff5700 10 client.26233 handle_command_reply: tid=0 2023-08-25T09:11:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 client.26233 shutdown 2023-08-25T09:11:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1122419381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 msgr2=0x560cc79f75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79f75f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00c2d0 tx=0x7f2e0c098b00 comp rx=0 tx=0).stop 2023-08-25T09:11:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1122419381 shutdown_connections 2023-08-25T09:11:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560cc7a05170 0x560cc7a07590 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2dfeff5700 0 client.26233 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79eb990 0x560cc79f75f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1122419381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f0f40 0x560cc79f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1122419381 >> 172.21.15.99:0/1122419381 conn(0x560cc7960080 msgr2=0x560cc79e0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1122419381 shutdown_connections 2023-08-25T09:11:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.779+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1122419381 wait complete. 2023-08-25T09:11:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.780+0000 7f2e34a92b80 20 client.26233 trim_cache size 0 max 16384 2023-08-25T09:11:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.795+0000 7f2e34a92b80 1 Processor -- start 2023-08-25T09:11:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.795+0000 7f2e34a92b80 1 -- start start 2023-08-25T09:11:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.795+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79eb990 0x560cc79eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.795+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79fc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.795+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f1860 0x560cc79f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.795+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc77bf8c0 con 0x560cc79f1860 2023-08-25T09:11:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.795+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc79fc750 con 0x560cc79eb990 2023-08-25T09:11:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.795+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc79ef480 con 0x560cc79f0f40 2023-08-25T09:11:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1b59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f1860 0x560cc79f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79fc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34180/0 (socket says 172.21.15.99:34180) 2023-08-25T09:11:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79eb990 0x560cc79eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 -- 172.21.15.99:0/2567500465 learned_addr learned my addr 172.21.15.99:0/2567500465 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 -- 172.21.15.99:0/2567500465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79eb990 msgr2=0x560cc79eb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/2567500465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79eb990 0x560cc79eb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 -- 172.21.15.99:0/2567500465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f1860 msgr2=0x560cc79f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/2567500465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f1860 0x560cc79f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 -- 172.21.15.99:0/2567500465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc7752c00 con 0x560cc79f0f40 2023-08-25T09:11:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/2567500465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79eb990 0x560cc79eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:11:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.796+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/2567500465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79fc210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x560cc79f5eb0 tx=0x7f2e10014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bdac2a2ded70086 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e01ffb700 1 -- 172.21.15.99:0/2567500465 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1002a4c0 con 0x560cc79f0f40 2023-08-25T09:11:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e01ffb700 1 -- 172.21.15.99:0/2567500465 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e10023040 con 0x560cc79f0f40 2023-08-25T09:11:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e01ffb700 1 -- 172.21.15.99:0/2567500465 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e10032bc0 con 0x560cc79f0f40 2023-08-25T09:11:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e34a92b80 1 -- 172.21.15.99:0/2567500465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 msgr2=0x560cc79fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/2567500465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79fc210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x560cc79f5eb0 tx=0x7f2e10014000 comp rx=0 tx=0).stop 2023-08-25T09:11:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e34a92b80 1 -- 172.21.15.99:0/2567500465 shutdown_connections 2023-08-25T09:11:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/2567500465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79f1860 0x560cc79f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/2567500465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79eb990 0x560cc79eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/2567500465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79fc210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.797+0000 7f2e34a92b80 1 -- 172.21.15.99:0/2567500465 >> 172.21.15.99:0/2567500465 conn(0x560cc7960080 msgr2=0x560cc79ee910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.798+0000 7f2e34a92b80 1 -- 172.21.15.99:0/2567500465 shutdown_connections 2023-08-25T09:11:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.798+0000 7f2e34a92b80 1 -- 172.21.15.99:0/2567500465 wait complete. 2023-08-25T09:11:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.798+0000 7f2e34a92b80 1 -- start start 2023-08-25T09:11:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.798+0000 7f2e34a92b80 1 Processor -- start 2023-08-25T09:11:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79eb990 0x560cc79f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e34a92b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc79f3a50 con 0x560cc79eb990 2023-08-25T09:11:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc79f3bc0 con 0x560cc79f1860 2023-08-25T09:11:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e34a92b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc79f3d80 con 0x560cc79f0f40 2023-08-25T09:11:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e34a92b80 10 client.0 _ll_register_callbacks cb 0x7f2e0c0987f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79f2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34190/0 (socket says 172.21.15.99:34190) 2023-08-25T09:11:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e1a59c700 1 -- 172.21.15.99:0/1151415045 learned_addr learned my addr 172.21.15.99:0/1151415045 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e1b59e700 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.799+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79eb990 0x560cc79f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2e1a59c700 1 -- 172.21.15.99:0/1151415045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 msgr2=0x560cc79f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2e1a59c700 1 -- 172.21.15.99:0/1151415045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79eb990 msgr2=0x560cc79f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79eb990 0x560cc79f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2e1a59c700 1 -- 172.21.15.99:0/1151415045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cc7769d80 con 0x560cc79f0f40 2023-08-25T09:11:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2e1a59c700 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79f2fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009fa0 tx=0x7f2e1002a550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9227d5eaa22a58e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2dfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:11:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1151415045 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000eb20 con 0x560cc79f0f40 2023-08-25T09:11:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1151415045 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2e1000ec80 con 0x560cc79f0f40 2023-08-25T09:11:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1151415045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cc79de600 con 0x560cc79f0f40 2023-08-25T09:11:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.800+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1151415045 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e10032550 con 0x560cc79f0f40 2023-08-25T09:11:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e34a92b80 10 client.26239 fetch_fsmap learned FSMap version 40 2023-08-25T09:11:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1151415045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f2e10017050 con 0x560cc79f0f40 2023-08-25T09:11:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1151415045 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560cc79de600 con 0x560cc79f0f40 2023-08-25T09:11:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1151415045 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e10032b60 con 0x560cc79f0f40 2023-08-25T09:11:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e34a92b80 10 client.26239 fetch_fsmap finished waiting for FSMap version 40 2023-08-25T09:11:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e34a92b80 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e34a92b80 20 client.26239 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560cc7a5a240 0x560cc7a5c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e34a92b80 4 client.26239 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:11:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1151415045 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560cc7936910 con 0x560cc7a5a240 2023-08-25T09:11:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.801+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560cc7a5a240 0x560cc7a5c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.802+0000 7f2e1ad9d700 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560cc7a5a240 0x560cc7a5c660 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f938a74608067b0 server_cookie=d829dda3e887e86 in_seq=0 out_seq=0 2023-08-25T09:11:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.802+0000 7f2dfeff5700 10 client.26239 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:14.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:14 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:11:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2dfeff5700 1 -- 172.21.15.99:0/1151415045 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560cc7936910 con 0x560cc7a5a240 2023-08-25T09:11:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2dfeff5700 10 client.26239 handle_command_reply: tid=0 2023-08-25T09:11:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 client.26239 shutdown 2023-08-25T09:11:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1151415045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 msgr2=0x560cc79f2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79f2fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009fa0 tx=0x7f2e1002a550 comp rx=0 tx=0).stop 2023-08-25T09:11:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1151415045 shutdown_connections 2023-08-25T09:11:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560cc7a5a240 0x560cc7a5c660 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560cc79f1860 0x560cc79f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cc79eb990 0x560cc79f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 --2- 172.21.15.99:0/1151415045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560cc79f0f40 0x560cc79f2fd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2dfeff5700 0 client.26239 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1151415045 >> 172.21.15.99:0/1151415045 conn(0x560cc7960080 msgr2=0x560cc79f1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1151415045 shutdown_connections 2023-08-25T09:11:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.854+0000 7f2e34a92b80 1 -- 172.21.15.99:0/1151415045 wait complete. 2023-08-25T09:11:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e34a92b80 20 client.26239 trim_cache size 0 max 16384 2023-08-25T09:11:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:11:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:11:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:11:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:11:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 -- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e040a00c0 msgr2=0x7f2e040a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e040a00c0 0x7f2e040a2580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009fd0 tx=0x7f2e10009260 comp rx=0 tx=0).stop 2023-08-25T09:11:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 -- 172.21.15.99:0/1597972016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c1008a0 msgr2=0x7f2e1c072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c1008a0 0x7f2e1c072e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00e9a0 tx=0x7f2e0c00ecb0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 -- 172.21.15.99:0/1597972016 shutdown_connections 2023-08-25T09:11:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f2e040a00c0 0x7f2e040a2580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e1c1068b0 0x7f2e1c1a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c1008a0 0x7f2e1c072e00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 --2- 172.21.15.99:0/1597972016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c101260 0x7f2e1c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 -- 172.21.15.99:0/1597972016 >> 172.21.15.99:0/1597972016 conn(0x7f2e1c0fc370 msgr2=0x7f2e1c0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.855+0000 7f2e211ff700 1 -- 172.21.15.99:0/1597972016 shutdown_connections 2023-08-25T09:11:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:14.856+0000 7f2e211ff700 1 -- 172.21.15.99:0/1597972016 wait complete. 2023-08-25T09:11:14.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:14 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:11:14.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 460921 3.81 MB/sec execute 560 sec latency 414.589 ms 2023-08-25T09:11:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:15 smithi181 ceph-mon[131296]: pgmap v377: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2023-08-25T09:11:15.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:15 smithi099 ceph-mon[97641]: pgmap v377: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2023-08-25T09:11:15.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:15 smithi157 ceph-mon[130879]: pgmap v377: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2023-08-25T09:11:15.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 462477 3.82 MB/sec execute 561 sec latency 23.466 ms 2023-08-25T09:11:16.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 463595 3.82 MB/sec execute 562 sec latency 189.729 ms 2023-08-25T09:11:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:17 smithi181 ceph-mon[131296]: pgmap v378: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-08-25T09:11:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:17 smithi099 ceph-mon[97641]: pgmap v378: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-08-25T09:11:17.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:17 smithi157 ceph-mon[130879]: pgmap v378: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-08-25T09:11:17.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 463595 3.82 MB/sec execute 563 sec latency 1189.809 ms 2023-08-25T09:11:18.950 INFO:tasks.workunit.client.0.smithi099.stdout: 1 463595 3.81 MB/sec execute 564 sec latency 2189.948 ms 2023-08-25T09:11:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:19 smithi181 ceph-mon[131296]: pgmap v379: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-08-25T09:11:19.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:19 smithi099 ceph-mon[97641]: pgmap v379: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-08-25T09:11:19.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:19 smithi157 ceph-mon[130879]: pgmap v379: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-08-25T09:11:19.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 464664 3.81 MB/sec execute 565 sec latency 2722.622 ms 2023-08-25T09:11:20.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 466051 3.82 MB/sec execute 566 sec latency 22.879 ms 2023-08-25T09:11:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:21 smithi181 ceph-mon[131296]: pgmap v380: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2023-08-25T09:11:21.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:21 smithi099 ceph-mon[97641]: pgmap v380: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2023-08-25T09:11:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:21 smithi157 ceph-mon[130879]: pgmap v380: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2023-08-25T09:11:21.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 467138 3.82 MB/sec execute 567 sec latency 285.559 ms 2023-08-25T09:11:22.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 467138 3.82 MB/sec execute 568 sec latency 1285.627 ms 2023-08-25T09:11:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:23 smithi181 ceph-mon[131296]: pgmap v381: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-08-25T09:11:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:23 smithi099 ceph-mon[97641]: pgmap v381: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-08-25T09:11:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:23 smithi157 ceph-mon[130879]: pgmap v381: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-08-25T09:11:23.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 467138 3.81 MB/sec execute 569 sec latency 2285.774 ms 2023-08-25T09:11:24.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 468212 3.81 MB/sec execute 570 sec latency 2819.933 ms 2023-08-25T09:11:25.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:25 smithi099 ceph-mon[97641]: pgmap v382: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 148 op/s 2023-08-25T09:11:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:25 smithi157 ceph-mon[130879]: pgmap v382: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 148 op/s 2023-08-25T09:11:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:25 smithi181 ceph-mon[131296]: pgmap v382: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 148 op/s 2023-08-25T09:11:25.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 469594 3.82 MB/sec execute 571 sec latency 19.293 ms 2023-08-25T09:11:26.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 470685 3.82 MB/sec execute 572 sec latency 287.650 ms 2023-08-25T09:11:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:27 smithi181 ceph-mon[131296]: pgmap v383: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2023-08-25T09:11:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:27 smithi099 ceph-mon[97641]: pgmap v383: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2023-08-25T09:11:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:27 smithi157 ceph-mon[130879]: pgmap v383: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2023-08-25T09:11:27.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 470685 3.82 MB/sec execute 573 sec latency 1287.746 ms 2023-08-25T09:11:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 -- 172.21.15.99:0/4221689353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01981008b0 msgr2=0x7f0198100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 --2- 172.21.15.99:0/4221689353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01981008b0 0x7f0198100d30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f01880098f0 tx=0x7f0188009c00 comp rx=0 tx=0).stop 2023-08-25T09:11:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 -- 172.21.15.99:0/4221689353 shutdown_connections 2023-08-25T09:11:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 --2- 172.21.15.99:0/4221689353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0198101270 0x7f019810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 --2- 172.21.15.99:0/4221689353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01981068c0 0x7f0198106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 --2- 172.21.15.99:0/4221689353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01981008b0 0x7f0198100d30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 -- 172.21.15.99:0/4221689353 >> 172.21.15.99:0/4221689353 conn(0x7f01980fc380 msgr2=0x7f01980fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 -- 172.21.15.99:0/4221689353 shutdown_connections 2023-08-25T09:11:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.011+0000 7f019f9ff700 1 -- 172.21.15.99:0/4221689353 wait complete. 2023-08-25T09:11:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.012+0000 7f019f9ff700 1 Processor -- start 2023-08-25T09:11:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019f9ff700 1 -- start start 2023-08-25T09:11:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019f9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01981008b0 0x7f0198072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019f9ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0198101270 0x7f0198073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019f9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01981068c0 0x7f01981a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019f9ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981a5170 con 0x7f0198101270 2023-08-25T09:11:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019f9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981a52e0 con 0x7f01981008b0 2023-08-25T09:11:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019f9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981a54a0 con 0x7f01981068c0 2023-08-25T09:11:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019cf9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0198101270 0x7f0198073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019cf9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0198101270 0x7f0198073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46638/0 (socket says 172.21.15.99:46638) 2023-08-25T09:11:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019cf9a700 1 -- 172.21.15.99:0/3776317952 learned_addr learned my addr 172.21.15.99:0/3776317952 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019d79b700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01981008b0 0x7f0198072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.013+0000 7f019df9c700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01981068c0 0x7f01981a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f019cf9a700 1 -- 172.21.15.99:0/3776317952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01981068c0 msgr2=0x7f01981a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f019cf9a700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01981068c0 0x7f01981a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f019cf9a700 1 -- 172.21.15.99:0/3776317952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01981008b0 msgr2=0x7f0198072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f019cf9a700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01981008b0 0x7f0198072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f019cf9a700 1 -- 172.21.15.99:0/3776317952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940095f0 con 0x7f0198101270 2023-08-25T09:11:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f019cf9a700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0198101270 0x7f0198073340 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0188009810 tx=0x7f018800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f018e7fc700 1 -- 172.21.15.99:0/3776317952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018801c070 con 0x7f0198101270 2023-08-25T09:11:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f019f9ff700 1 -- 172.21.15.99:0/3776317952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0188009520 con 0x7f0198101270 2023-08-25T09:11:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f018e7fc700 1 -- 172.21.15.99:0/3776317952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f018800bbd0 con 0x7f0198101270 2023-08-25T09:11:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f018e7fc700 1 -- 172.21.15.99:0/3776317952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880049c0 con 0x7f0198101270 2023-08-25T09:11:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.014+0000 7f019f9ff700 1 -- 172.21.15.99:0/3776317952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01981a5ac0 con 0x7f0198101270 2023-08-25T09:11:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.018+0000 7f018e7fc700 1 -- 172.21.15.99:0/3776317952 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f0188020070 con 0x7f0198101270 2023-08-25T09:11:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.019+0000 7f01b3277b80 1 Processor -- start 2023-08-25T09:11:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.019+0000 7f018e7fc700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01840a00c0 0x7f01840a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.019+0000 7f01b3277b80 1 -- start start 2023-08-25T09:11:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.019+0000 7f018e7fc700 1 -- 172.21.15.99:0/3776317952 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f01880cc410 con 0x7f0198101270 2023-08-25T09:11:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.019+0000 7f019df9c700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01840a00c0 0x7f01840a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af6509320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6509860 0x560af6503450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6503990 0x560af6503dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f01b3277b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af646b750 con 0x560af6508f40 2023-08-25T09:11:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f01b3277b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af645c720 con 0x560af6503990 2023-08-25T09:11:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f01b3277b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af645a040 con 0x560af6509860 2023-08-25T09:11:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019d79b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af6509320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019d79b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af6509320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46652/0 (socket says 172.21.15.99:46652) 2023-08-25T09:11:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019d79b700 1 -- 172.21.15.99:0/2697635005 learned_addr learned my addr 172.21.15.99:0/2697635005 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019cf9a700 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6509860 0x560af6503450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019df9c700 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6503990 0x560af6503dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019d79b700 1 -- 172.21.15.99:0/2697635005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6509860 msgr2=0x560af6503450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019d79b700 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6509860 0x560af6503450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019d79b700 1 -- 172.21.15.99:0/2697635005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6503990 msgr2=0x560af6503dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019d79b700 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6503990 0x560af6503dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019d79b700 1 -- 172.21.15.99:0/2697635005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af62d55d0 con 0x560af6508f40 2023-08-25T09:11:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019df9c700 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6503990 0x560af6503dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:11:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.020+0000 7f019df9c700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01840a00c0 0x7f01840a2580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01900093c0 tx=0x7f0190009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.021+0000 7f019d79b700 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af6509320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0194009f40 tx=0x7f0194011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a900fc48db64c100 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.021+0000 7f01837fe700 1 -- 172.21.15.99:0/2697635005 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194004310 con 0x560af6508f40 2023-08-25T09:11:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.021+0000 7f01837fe700 1 -- 172.21.15.99:0/2697635005 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0194004470 con 0x560af6508f40 2023-08-25T09:11:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.021+0000 7f01837fe700 1 -- 172.21.15.99:0/2697635005 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194019910 con 0x560af6508f40 2023-08-25T09:11:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.021+0000 7f01b3277b80 1 -- 172.21.15.99:0/2697635005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 msgr2=0x560af6509320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af6509320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0194009f40 tx=0x7f0194011720 comp rx=0 tx=0).stop 2023-08-25T09:11:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 -- 172.21.15.99:0/2697635005 shutdown_connections 2023-08-25T09:11:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6503990 0x560af6503dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af6509320 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 --2- 172.21.15.99:0/2697635005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6509860 0x560af6503450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 -- 172.21.15.99:0/2697635005 >> 172.21.15.99:0/2697635005 conn(0x560af6478080 msgr2=0x560af64fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:28.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 -- 172.21.15.99:0/2697635005 shutdown_connections 2023-08-25T09:11:28.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 -- 172.21.15.99:0/2697635005 wait complete. 2023-08-25T09:11:28.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 -- start start 2023-08-25T09:11:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.022+0000 7f01b3277b80 1 Processor -- start 2023-08-25T09:11:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 0x560af650c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f01b3277b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af6455930 con 0x560af6503990 2023-08-25T09:11:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f01b3277b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af63f3b50 con 0x560af6509860 2023-08-25T09:11:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f01b3277b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af63f3850 con 0x560af6508f40 2023-08-25T09:11:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f01b3277b80 10 client.0 _ll_register_callbacks cb 0x560af633da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f019d79b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f019d79b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46658/0 (socket says 172.21.15.99:46658) 2023-08-25T09:11:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f019d79b700 1 -- 172.21.15.99:0/3353450025 learned_addr learned my addr 172.21.15.99:0/3353450025 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f019df9c700 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.023+0000 7f019cf9a700 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 0x560af650c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f019df9c700 1 -- 172.21.15.99:0/3353450025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 msgr2=0x560af650c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f019df9c700 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 0x560af650c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f019df9c700 1 -- 172.21.15.99:0/3353450025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 msgr2=0x560af650f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f019df9c700 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f019df9c700 1 -- 172.21.15.99:0/3353450025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0194009ba0 con 0x560af6509860 2023-08-25T09:11:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f019d79b700 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:11:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f019df9c700 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650d030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0190007b70 tx=0x7f0190004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1a1b47e7831dd5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f0180ff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:11:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f0180ff9700 1 -- 172.21.15.99:0/3353450025 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019000e050 con 0x560af6509860 2023-08-25T09:11:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.024+0000 7f01b3277b80 1 -- 172.21.15.99:0/3353450025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560af64d3160 con 0x560af6509860 2023-08-25T09:11:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f0180ff9700 1 -- 172.21.15.99:0/3353450025 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0190018c30 con 0x560af6509860 2023-08-25T09:11:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f0180ff9700 1 -- 172.21.15.99:0/3353450025 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0190018df0 con 0x560af6509860 2023-08-25T09:11:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f01b3277b80 10 client.26267 fetch_fsmap learned FSMap version 40 2023-08-25T09:11:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f01b3277b80 1 -- 172.21.15.99:0/3353450025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560af6281d80 con 0x560af6509860 2023-08-25T09:11:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f0180ff9700 1 -- 172.21.15.99:0/3353450025 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560af64d3160 con 0x560af6509860 2023-08-25T09:11:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f0180ff9700 1 -- 172.21.15.99:0/3353450025 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 40) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f019002b3f0 con 0x560af6509860 2023-08-25T09:11:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f01b3277b80 10 client.26267 fetch_fsmap finished waiting for FSMap version 40 2023-08-25T09:11:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f01b3277b80 10 client.26267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f01b3277b80 20 client.26267 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f01b3277b80 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560af651d220 0x560af651f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f01b3277b80 4 client.26267 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:11:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.025+0000 7f01b3277b80 1 -- 172.21.15.99:0/3353450025 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560af63fbb40 con 0x560af651d220 2023-08-25T09:11:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.026+0000 7f019d79b700 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560af651d220 0x560af651f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.026+0000 7f0180ff9700 10 client.26267 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.026+0000 7f019d79b700 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560af651d220 0x560af651f640 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0296db5ff44b29 server_cookie=c72d75905650086b in_seq=0 out_seq=0 2023-08-25T09:11:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f0180ff9700 1 -- 172.21.15.99:0/3353450025 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560af63fbb40 con 0x560af651d220 2023-08-25T09:11:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f0180ff9700 10 client.26267 handle_command_reply: tid=0 2023-08-25T09:11:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 client.26267 shutdown 2023-08-25T09:11:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 -- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 msgr2=0x560af650d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650d030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0190007b70 tx=0x7f0190004b40 comp rx=0 tx=0).stop 2023-08-25T09:11:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 -- 172.21.15.99:0/3353450025 shutdown_connections 2023-08-25T09:11:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560af651d220 0x560af651f640 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650d030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f0180ff9700 0 client.26267 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 --2- 172.21.15.99:0/3353450025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 0x560af650c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 -- 172.21.15.99:0/3353450025 >> 172.21.15.99:0/3353450025 conn(0x560af6478080 msgr2=0x560af64f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 -- 172.21.15.99:0/3353450025 shutdown_connections 2023-08-25T09:11:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.027+0000 7f01b3277b80 1 -- 172.21.15.99:0/3353450025 wait complete. 2023-08-25T09:11:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.028+0000 7f01b3277b80 20 client.26267 trim_cache size 0 max 16384 2023-08-25T09:11:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.042+0000 7f01b3277b80 1 Processor -- start 2023-08-25T09:11:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.042+0000 7f01b3277b80 1 -- start start 2023-08-25T09:11:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.042+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6503990 0x560af6514240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.042+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af650f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f01b3277b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af62d78c0 con 0x560af6508f40 2023-08-25T09:11:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f01b3277b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af64f6390 con 0x560af6509860 2023-08-25T09:11:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f01b3277b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af6514780 con 0x560af6503990 2023-08-25T09:11:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af650f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af650f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46674/0 (socket says 172.21.15.99:46674) 2023-08-25T09:11:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 -- 172.21.15.99:0/203461225 learned_addr learned my addr 172.21.15.99:0/203461225 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019d79b700 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6503990 0x560af6514240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019df9c700 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 -- 172.21.15.99:0/203461225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6503990 msgr2=0x560af6514240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6503990 0x560af6514240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 -- 172.21.15.99:0/203461225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 msgr2=0x560af650c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 -- 172.21.15.99:0/203461225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af626ac00 con 0x560af6508f40 2023-08-25T09:11:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.043+0000 7f019cf9a700 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af650f210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0188096040 tx=0x7f01880b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39c7187d954e346 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f0183fff700 1 -- 172.21.15.99:0/203461225 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018809a240 con 0x560af6508f40 2023-08-25T09:11:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f0183fff700 1 -- 172.21.15.99:0/203461225 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0188099b40 con 0x560af6508f40 2023-08-25T09:11:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f0183fff700 1 -- 172.21.15.99:0/203461225 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188005260 con 0x560af6508f40 2023-08-25T09:11:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 -- 172.21.15.99:0/203461225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 msgr2=0x560af650f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af650f210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0188096040 tx=0x7f01880b9040 comp rx=0 tx=0).stop 2023-08-25T09:11:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 -- 172.21.15.99:0/203461225 shutdown_connections 2023-08-25T09:11:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6503990 0x560af6514240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 --2- 172.21.15.99:0/203461225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6508f40 0x560af650f210 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 -- 172.21.15.99:0/203461225 >> 172.21.15.99:0/203461225 conn(0x560af6478080 msgr2=0x560af64fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 -- 172.21.15.99:0/203461225 shutdown_connections 2023-08-25T09:11:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.044+0000 7f01b3277b80 1 -- 172.21.15.99:0/203461225 wait complete. 2023-08-25T09:11:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 1 -- start start 2023-08-25T09:11:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 1 Processor -- start 2023-08-25T09:11:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 0x560af650c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af650b3f0 con 0x560af6503990 2023-08-25T09:11:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af650b560 con 0x560af6509860 2023-08-25T09:11:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af650b720 con 0x560af6508f40 2023-08-25T09:11:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.045+0000 7f01b3277b80 10 client.0 _ll_register_callbacks cb 0x7f01900177e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46690/0 (socket says 172.21.15.99:46690) 2023-08-25T09:11:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 -- 172.21.15.99:0/1915226087 learned_addr learned my addr 172.21.15.99:0/1915226087 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019cf9a700 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 0x560af650c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019df9c700 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 -- 172.21.15.99:0/1915226087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 msgr2=0x560af650c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 0x560af650c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 -- 172.21.15.99:0/1915226087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 msgr2=0x560af650ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 -- 172.21.15.99:0/1915226087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01981a5ac0 con 0x560af6503990 2023-08-25T09:11:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f019d79b700 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650c300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0194009e90 tx=0x7f0194005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d1f00d1e88cf5f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f0180ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:11:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.046+0000 7f0180ff9700 1 -- 172.21.15.99:0/1915226087 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019402c070 con 0x560af6503990 2023-08-25T09:11:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 1 -- 172.21.15.99:0/1915226087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560af64f6600 con 0x560af6503990 2023-08-25T09:11:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f0180ff9700 1 -- 172.21.15.99:0/1915226087 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0194003680 con 0x560af6503990 2023-08-25T09:11:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f0180ff9700 1 -- 172.21.15.99:0/1915226087 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0194009290 con 0x560af6503990 2023-08-25T09:11:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 10 client.16212 fetch_fsmap learned FSMap version 40 2023-08-25T09:11:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 1 -- 172.21.15.99:0/1915226087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560af62a81f0 con 0x560af6503990 2023-08-25T09:11:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f0180ff9700 1 -- 172.21.15.99:0/1915226087 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560af64f6600 con 0x560af6503990 2023-08-25T09:11:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f0180ff9700 1 -- 172.21.15.99:0/1915226087 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f019401a9e0 con 0x560af6503990 2023-08-25T09:11:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 10 client.16212 fetch_fsmap finished waiting for FSMap version 40 2023-08-25T09:11:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 20 client.16212 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560af6571fb0 0x560af65743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 4 client.16212 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:11:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.047+0000 7f01b3277b80 1 -- 172.21.15.99:0/1915226087 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560af644e910 con 0x560af6571fb0 2023-08-25T09:11:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.048+0000 7f019cf9a700 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560af6571fb0 0x560af65743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.048+0000 7f0180ff9700 10 client.16212 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.048+0000 7f019cf9a700 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560af6571fb0 0x560af65743a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520f315bfea7b35d server_cookie=248baf718b287418 in_seq=0 out_seq=0 2023-08-25T09:11:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f0180ff9700 1 -- 172.21.15.99:0/1915226087 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560af644e910 con 0x560af6571fb0 2023-08-25T09:11:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f0180ff9700 10 client.16212 handle_command_reply: tid=0 2023-08-25T09:11:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 client.16212 shutdown 2023-08-25T09:11:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 -- 172.21.15.99:0/1915226087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 msgr2=0x560af650c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650c300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0194009e90 tx=0x7f0194005bc0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 -- 172.21.15.99:0/1915226087 shutdown_connections 2023-08-25T09:11:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560af6571fb0 0x560af65743a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af6509860 0x560af650ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560af6503990 0x560af650c300 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 --2- 172.21.15.99:0/1915226087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af6508f40 0x560af650c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f0180ff9700 0 client.16212 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.096+0000 7f01b3277b80 1 -- 172.21.15.99:0/1915226087 >> 172.21.15.99:0/1915226087 conn(0x560af6478080 msgr2=0x560af6509c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.097+0000 7f01b3277b80 1 -- 172.21.15.99:0/1915226087 shutdown_connections 2023-08-25T09:11:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.097+0000 7f01b3277b80 1 -- 172.21.15.99:0/1915226087 wait complete. 2023-08-25T09:11:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.097+0000 7f01b3277b80 20 client.16212 trim_cache size 0 max 16384 2023-08-25T09:11:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:11:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:11:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:11:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:11:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.097+0000 7f019f9ff700 1 -- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01840a00c0 msgr2=0x7f01840a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.097+0000 7f019f9ff700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01840a00c0 0x7f01840a2580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01900093c0 tx=0x7f0190009920 comp rx=0 tx=0).stop 2023-08-25T09:11:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.097+0000 7f019f9ff700 1 -- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0198101270 msgr2=0x7f0198073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.097+0000 7f019f9ff700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0198101270 0x7f0198073340 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0188009810 tx=0x7f018800fa00 comp rx=0 tx=0).stop 2023-08-25T09:11:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.098+0000 7f019f9ff700 1 -- 172.21.15.99:0/3776317952 shutdown_connections 2023-08-25T09:11:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.098+0000 7f019f9ff700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01840a00c0 0x7f01840a2580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.098+0000 7f019f9ff700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01981068c0 0x7f01981a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.098+0000 7f019f9ff700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01981008b0 0x7f0198072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.098+0000 7f019f9ff700 1 --2- 172.21.15.99:0/3776317952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0198101270 0x7f0198073340 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.098+0000 7f019f9ff700 1 -- 172.21.15.99:0/3776317952 >> 172.21.15.99:0/3776317952 conn(0x7f01980fc380 msgr2=0x7f01980fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.098+0000 7f019f9ff700 1 -- 172.21.15.99:0/3776317952 shutdown_connections 2023-08-25T09:11:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:28.098+0000 7f019f9ff700 1 -- 172.21.15.99:0/3776317952 wait complete. 2023-08-25T09:11:28.951 INFO:tasks.workunit.client.0.smithi099.stdout: 1 470685 3.81 MB/sec execute 574 sec latency 2287.864 ms 2023-08-25T09:11:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:29 smithi181 ceph-mon[131296]: pgmap v384: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1015 KiB/s wr, 92 op/s 2023-08-25T09:11:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:29 smithi099 ceph-mon[97641]: pgmap v384: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1015 KiB/s wr, 92 op/s 2023-08-25T09:11:29.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:29 smithi157 ceph-mon[130879]: pgmap v384: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1015 KiB/s wr, 92 op/s 2023-08-25T09:11:29.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 471759 3.81 MB/sec execute 575 sec latency 2814.794 ms 2023-08-25T09:11:30.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 473132 3.82 MB/sec execute 576 sec latency 28.017 ms 2023-08-25T09:11:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:31 smithi181 ceph-mon[131296]: pgmap v385: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1015 KiB/s wr, 92 op/s 2023-08-25T09:11:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:31 smithi099 ceph-mon[97641]: pgmap v385: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1015 KiB/s wr, 92 op/s 2023-08-25T09:11:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:31 smithi157 ceph-mon[130879]: pgmap v385: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1015 KiB/s wr, 92 op/s 2023-08-25T09:11:31.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 474231 3.82 MB/sec execute 577 sec latency 261.008 ms 2023-08-25T09:11:32.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 474231 3.82 MB/sec execute 578 sec latency 1261.133 ms 2023-08-25T09:11:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:33 smithi181 ceph-mon[131296]: pgmap v386: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2023-08-25T09:11:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:33 smithi099 ceph-mon[97641]: pgmap v386: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2023-08-25T09:11:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:33 smithi157 ceph-mon[130879]: pgmap v386: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2023-08-25T09:11:33.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 474231 3.81 MB/sec execute 579 sec latency 2261.227 ms 2023-08-25T09:11:34.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:34 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:11:34.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:34 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:11:34.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:34 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:11:34.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 475155 3.81 MB/sec execute 580 sec latency 2788.116 ms 2023-08-25T09:11:35.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:35 smithi157 ceph-mon[130879]: pgmap v387: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 708 KiB/s wr, 102 op/s 2023-08-25T09:11:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:35 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:35 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:35 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:35.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:35 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:35.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 476648 3.82 MB/sec execute 581 sec latency 12.987 ms 2023-08-25T09:11:35.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:35 smithi181 ceph-mon[131296]: pgmap v387: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 708 KiB/s wr, 102 op/s 2023-08-25T09:11:35.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:35 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:35.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:35 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:35.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:35 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:35.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:35 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:35 smithi099 ceph-mon[97641]: pgmap v387: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 708 KiB/s wr, 102 op/s 2023-08-25T09:11:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:35 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:35 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:35 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:36.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:35 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:36.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:36 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:11:36.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:36 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:11:36.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:36 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:36.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 477780 3.82 MB/sec execute 582 sec latency 227.440 ms 2023-08-25T09:11:36.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:36 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:11:36.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:36 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:11:36.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:36 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:37.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:36 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:11:37.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:36 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:11:37.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:36 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:11:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:37 smithi157 ceph-mon[130879]: pgmap v388: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-08-25T09:11:37.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 477780 3.82 MB/sec execute 583 sec latency 1227.559 ms 2023-08-25T09:11:37.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:37 smithi181 ceph-mon[131296]: pgmap v388: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-08-25T09:11:38.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:37 smithi099 ceph-mon[97641]: pgmap v388: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-08-25T09:11:38.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 477780 3.81 MB/sec execute 584 sec latency 2227.677 ms 2023-08-25T09:11:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:39 smithi157 ceph-mon[130879]: pgmap v389: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 768 KiB/s wr, 97 op/s 2023-08-25T09:11:39.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 478884 3.81 MB/sec execute 585 sec latency 2753.074 ms 2023-08-25T09:11:39.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:39 smithi181 ceph-mon[131296]: pgmap v389: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 768 KiB/s wr, 97 op/s 2023-08-25T09:11:40.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:39 smithi099 ceph-mon[97641]: pgmap v389: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 768 KiB/s wr, 97 op/s 2023-08-25T09:11:40.952 INFO:tasks.workunit.client.0.smithi099.stdout: 1 480333 3.82 MB/sec execute 586 sec latency 17.385 ms 2023-08-25T09:11:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.255+0000 7f99ef3ff700 1 -- 172.21.15.99:0/4158739246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e80686d0 msgr2=0x7f99e810d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.255+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/4158739246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e80686d0 0x7f99e810d720 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f99e000b0a0 tx=0x7f99e000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.256+0000 7f99ef3ff700 1 -- 172.21.15.99:0/4158739246 shutdown_connections 2023-08-25T09:11:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.256+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/4158739246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e80686d0 0x7f99e810d720 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.256+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/4158739246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e81068d0 0x7f99e8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.256+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/4158739246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e8067d10 0x7f99e8068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.256+0000 7f99ef3ff700 1 -- 172.21.15.99:0/4158739246 >> 172.21.15.99:0/4158739246 conn(0x7f99e8075250 msgr2=0x7f99e8075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.256+0000 7f99ef3ff700 1 -- 172.21.15.99:0/4158739246 shutdown_connections 2023-08-25T09:11:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.256+0000 7f99ef3ff700 1 -- 172.21.15.99:0/4158739246 wait complete. 2023-08-25T09:11:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.257+0000 7f99ef3ff700 1 Processor -- start 2023-08-25T09:11:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.257+0000 7f99ef3ff700 1 -- start start 2023-08-25T09:11:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.257+0000 7f99ef3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8067d10 0x7f99e819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.257+0000 7f99ef3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e80686d0 0x7f99e819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.257+0000 7f99ef3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e81068d0 0x7f99e81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.257+0000 7f99ef3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8198f10 con 0x7f99e81068d0 2023-08-25T09:11:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.257+0000 7f99ef3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8199080 con 0x7f99e80686d0 2023-08-25T09:11:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.257+0000 7f99ef3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8199240 con 0x7f99e8067d10 2023-08-25T09:11:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8067d10 0x7f99e819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8067d10 0x7f99e819ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:49598/0 (socket says 172.21.15.99:49598) 2023-08-25T09:11:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 -- 172.21.15.99:0/1261219315 learned_addr learned my addr 172.21.15.99:0/1261219315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ec99a700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e80686d0 0x7f99e819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed99c700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e81068d0 0x7f99e81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 -- 172.21.15.99:0/1261219315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e80686d0 msgr2=0x7f99e819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e80686d0 0x7f99e819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 -- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e81068d0 msgr2=0x7f99e81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e81068d0 0x7f99e81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 -- 172.21.15.99:0/1261219315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d8009520 con 0x7f99e8067d10 2023-08-25T09:11:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.258+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8067d10 0x7f99e819ea40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f99e400e9a0 tx=0x7f99e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.259+0000 7f99de7fc700 1 -- 172.21.15.99:0/1261219315 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4009790 con 0x7f99e8067d10 2023-08-25T09:11:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.259+0000 7f99de7fc700 1 -- 172.21.15.99:0/1261219315 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e4010430 con 0x7f99e8067d10 2023-08-25T09:11:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.259+0000 7f99ef3ff700 1 -- 172.21.15.99:0/1261219315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e0009d00 con 0x7f99e8067d10 2023-08-25T09:11:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.259+0000 7f99de7fc700 1 -- 172.21.15.99:0/1261219315 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4018680 con 0x7f99e8067d10 2023-08-25T09:11:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.259+0000 7f99ef3ff700 1 -- 172.21.15.99:0/1261219315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e81a5850 con 0x7f99e8067d10 2023-08-25T09:11:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.263+0000 7f99de7fc700 1 -- 172.21.15.99:0/1261219315 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f99e40105a0 con 0x7f99e8067d10 2023-08-25T09:11:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.264+0000 7f9a02d84b80 1 Processor -- start 2023-08-25T09:11:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.264+0000 7f9a02d84b80 1 -- start start 2023-08-25T09:11:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99de7fc700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99d40a00c0 0x7f99d40a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b8860 0x55d0f63b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b2990 0x55d0f63b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f631a750 con 0x55d0f63b2990 2023-08-25T09:11:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f630b720 con 0x55d0f63b8860 2023-08-25T09:11:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f6309040 con 0x55d0f63b7f40 2023-08-25T09:11:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99de7fc700 1 -- 172.21.15.99:0/1261219315 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f99e4014070 con 0x7f99e8067d10 2023-08-25T09:11:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:49612/0 (socket says 172.21.15.99:49612) 2023-08-25T09:11:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99ec99a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b8860 0x55d0f63b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99ed19b700 1 -- 172.21.15.99:0/3862464657 learned_addr learned my addr 172.21.15.99:0/3862464657 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99ed99c700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99d40a00c0 0x7f99d40a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99ed99c700 1 --2- 172.21.15.99:0/3862464657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b2990 0x55d0f63b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.265+0000 7f99ed19b700 1 -- 172.21.15.99:0/3862464657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b8860 msgr2=0x55d0f63b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99ed19b700 1 --2- 172.21.15.99:0/3862464657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b8860 0x55d0f63b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99ed19b700 1 -- 172.21.15.99:0/3862464657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b2990 msgr2=0x55d0f63b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99ed19b700 1 --2- 172.21.15.99:0/3862464657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b2990 0x55d0f63b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99ed19b700 1 -- 172.21.15.99:0/3862464657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f61845d0 con 0x55d0f63b7f40 2023-08-25T09:11:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99ed19b700 1 --2- 172.21.15.99:0/3862464657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b8320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f99e40a0770 tx=0x7f99e4098870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad60bc34a0395 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99ed99c700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99d40a00c0 0x7f99d40a2580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009cd0 tx=0x7f99e0009c20 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99d37fe700 1 -- 172.21.15.99:0/3862464657 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4076560 con 0x55d0f63b7f40 2023-08-25T09:11:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99d37fe700 1 -- 172.21.15.99:0/3862464657 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e4076380 con 0x55d0f63b7f40 2023-08-25T09:11:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.266+0000 7f99d37fe700 1 -- 172.21.15.99:0/3862464657 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40a0ca0 con 0x55d0f63b7f40 2023-08-25T09:11:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3862464657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 msgr2=0x55d0f63b8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3862464657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b8320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f99e40a0770 tx=0x7f99e4098870 comp rx=0 tx=0).stop 2023-08-25T09:11:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3862464657 shutdown_connections 2023-08-25T09:11:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3862464657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b2990 0x55d0f63b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3862464657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b8320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3862464657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b8860 0x55d0f63b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3862464657 >> 172.21.15.99:0/3862464657 conn(0x55d0f6327080 msgr2=0x55d0f63acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3862464657 shutdown_connections 2023-08-25T09:11:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.267+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3862464657 wait complete. 2023-08-25T09:11:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.268+0000 7f9a02d84b80 1 -- start start 2023-08-25T09:11:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.268+0000 7f9a02d84b80 1 Processor -- start 2023-08-25T09:11:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b7f40 0x55d0f63bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b8860 0x55d0f63bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f6304930 con 0x55d0f63b7f40 2023-08-25T09:11:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f62a2b50 con 0x55d0f63b2990 2023-08-25T09:11:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f62a2850 con 0x55d0f63b8860 2023-08-25T09:11:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f9a02d84b80 10 client.0 _ll_register_callbacks cb 0x55d0f61eca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ec99a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b7f40 0x55d0f63bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed99c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b8860 0x55d0f63bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63bb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:33346/0 (socket says 172.21.15.99:33346) 2023-08-25T09:11:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed19b700 1 -- 172.21.15.99:0/1233881675 learned_addr learned my addr 172.21.15.99:0/1233881675 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed19b700 1 -- 172.21.15.99:0/1233881675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b8860 msgr2=0x55d0f63bc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b8860 0x55d0f63bc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed19b700 1 -- 172.21.15.99:0/1233881675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b7f40 msgr2=0x55d0f63bbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b7f40 0x55d0f63bbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.269+0000 7f99ed19b700 1 -- 172.21.15.99:0/1233881675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e81a5850 con 0x55d0f63b2990 2023-08-25T09:11:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63bb990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f99e4000c00 tx=0x7f99e4098e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=838b519e510bc425 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f99d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:11:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f99d0ff9700 1 -- 172.21.15.99:0/1233881675 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4099ea0 con 0x55d0f63b2990 2023-08-25T09:11:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1233881675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0f6382160 con 0x55d0f63b2990 2023-08-25T09:11:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f99d0ff9700 1 -- 172.21.15.99:0/1233881675 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f99e4070e40 con 0x55d0f63b2990 2023-08-25T09:11:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f99d0ff9700 1 -- 172.21.15.99:0/1233881675 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e4076340 con 0x55d0f63b2990 2023-08-25T09:11:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 10 client.26285 fetch_fsmap learned FSMap version 40 2023-08-25T09:11:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1233881675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d0f6130d80 con 0x55d0f63b2990 2023-08-25T09:11:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f99d0ff9700 1 -- 172.21.15.99:0/1233881675 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0f6382160 con 0x55d0f63b2990 2023-08-25T09:11:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f99d0ff9700 1 -- 172.21.15.99:0/1233881675 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 40) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99e4073110 con 0x55d0f63b2990 2023-08-25T09:11:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 10 client.26285 fetch_fsmap finished waiting for FSMap version 40 2023-08-25T09:11:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 10 client.26285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 20 client.26285 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d0f63cbef0 0x55d0f63ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 4 client.26285 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:11:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.270+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1233881675 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0f62aab40 con 0x55d0f63cbef0 2023-08-25T09:11:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.271+0000 7f99ec99a700 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d0f63cbef0 0x55d0f63ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.271+0000 7f99ec99a700 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d0f63cbef0 0x55d0f63ce310 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d99fc285a8e0a69 server_cookie=8abf5aae86b9dbce in_seq=0 out_seq=0 2023-08-25T09:11:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.271+0000 7f99d0ff9700 10 client.26285 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f99d0ff9700 1 -- 172.21.15.99:0/1233881675 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d0f62aab40 con 0x55d0f63cbef0 2023-08-25T09:11:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f99d0ff9700 10 client.26285 handle_command_reply: tid=0 2023-08-25T09:11:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 client.26285 shutdown 2023-08-25T09:11:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1233881675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 msgr2=0x55d0f63bb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63bb990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f99e4000c00 tx=0x7f99e4098e90 comp rx=0 tx=0).stop 2023-08-25T09:11:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1233881675 shutdown_connections 2023-08-25T09:11:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b8860 0x55d0f63bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d0f63cbef0 0x55d0f63ce310 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63bb990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1233881675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b7f40 0x55d0f63bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1233881675 >> 172.21.15.99:0/1233881675 conn(0x55d0f6327080 msgr2=0x55d0f63a7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f99d0ff9700 0 client.26285 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1233881675 shutdown_connections 2023-08-25T09:11:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1233881675 wait complete. 2023-08-25T09:11:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.272+0000 7f9a02d84b80 20 client.26285 trim_cache size 0 max 16384 2023-08-25T09:11:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f9a02d84b80 1 Processor -- start 2023-08-25T09:11:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f9a02d84b80 1 -- start start 2023-08-25T09:11:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b2990 0x55d0f63b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b7f40 0x55d0f63c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f61868c0 con 0x55d0f63b8860 2023-08-25T09:11:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f63c8ba0 con 0x55d0f63b7f40 2023-08-25T09:11:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f63ae060 con 0x55d0f63b2990 2023-08-25T09:11:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b2990 0x55d0f63b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b2990 0x55d0f63b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:49620/0 (socket says 172.21.15.99:49620) 2023-08-25T09:11:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f99ed19b700 1 -- 172.21.15.99:0/1645086491 learned_addr learned my addr 172.21.15.99:0/1645086491 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f99ec99a700 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b7f40 0x55d0f63c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.289+0000 7f99ed99c700 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99ed19b700 1 -- 172.21.15.99:0/1645086491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b7f40 msgr2=0x55d0f63c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b7f40 0x55d0f63c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99ed19b700 1 -- 172.21.15.99:0/1645086491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 msgr2=0x55d0f63bc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63bc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99ed19b700 1 -- 172.21.15.99:0/1645086491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f6119c00 con 0x55d0f63b2990 2023-08-25T09:11:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99ed19b700 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b2990 0x55d0f63b2450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f99e400c2d0 tx=0x7f99e4098d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8f1d25f74911d20 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99d3fff700 1 -- 172.21.15.99:0/1645086491 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4099ea0 con 0x55d0f63b2990 2023-08-25T09:11:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99d3fff700 1 -- 172.21.15.99:0/1645086491 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e409ccc0 con 0x55d0f63b2990 2023-08-25T09:11:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.290+0000 7f99d3fff700 1 -- 172.21.15.99:0/1645086491 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40b8370 con 0x55d0f63b2990 2023-08-25T09:11:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1645086491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b2990 msgr2=0x55d0f63b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b2990 0x55d0f63b2450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f99e400c2d0 tx=0x7f99e4098d90 comp rx=0 tx=0).stop 2023-08-25T09:11:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1645086491 shutdown_connections 2023-08-25T09:11:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63bc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b2990 0x55d0f63b2450 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/1645086491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b7f40 0x55d0f63c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1645086491 >> 172.21.15.99:0/1645086491 conn(0x55d0f6327080 msgr2=0x55d0f63acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1645086491 shutdown_connections 2023-08-25T09:11:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.291+0000 7f9a02d84b80 1 -- 172.21.15.99:0/1645086491 wait complete. 2023-08-25T09:11:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 1 -- start start 2023-08-25T09:11:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 1 Processor -- start 2023-08-25T09:11:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f63b4840 con 0x55d0f63b8860 2023-08-25T09:11:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f63b49b0 con 0x55d0f63b2990 2023-08-25T09:11:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f63b4b70 con 0x55d0f63b7f40 2023-08-25T09:11:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.292+0000 7f9a02d84b80 10 client.0 _ll_register_callbacks cb 0x7f99d80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ec99a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:33360/0 (socket says 172.21.15.99:33360) 2023-08-25T09:11:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 -- 172.21.15.99:0/3729474294 learned_addr learned my addr 172.21.15.99:0/3729474294 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed99c700 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 -- 172.21.15.99:0/3729474294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 msgr2=0x55d0f63b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 -- 172.21.15.99:0/3729474294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 msgr2=0x55d0f63b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 -- 172.21.15.99:0/3729474294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0f6130d80 con 0x55d0f63b2990 2023-08-25T09:11:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed99c700 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:11:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99ed19b700 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63b3880 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99e4075810 tx=0x7f99e4077c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f177a8c80b2bbc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99d0ff9700 1 -- 172.21.15.99:0/3729474294 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4099ea0 con 0x55d0f63b2990 2023-08-25T09:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3729474294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0f63a5600 con 0x55d0f63b2990 2023-08-25T09:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.293+0000 7f99d0ff9700 1 -- 172.21.15.99:0/3729474294 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f99e4070e40 con 0x55d0f63b2990 2023-08-25T09:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f99d0ff9700 1 -- 172.21.15.99:0/3729474294 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e40990f0 con 0x55d0f63b2990 2023-08-25T09:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f9a02d84b80 10 client.26291 fetch_fsmap learned FSMap version 40 2023-08-25T09:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3729474294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d0f61571f0 con 0x55d0f63b2990 2023-08-25T09:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f99d0ff9700 1 -- 172.21.15.99:0/3729474294 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0f63a5600 con 0x55d0f63b2990 2023-08-25T09:11:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f99d0ff9700 1 -- 172.21.15.99:0/3729474294 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 40) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99e4097df0 con 0x55d0f63b2990 2023-08-25T09:11:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f9a02d84b80 10 client.26291 fetch_fsmap finished waiting for FSMap version 40 2023-08-25T09:11:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f9a02d84b80 10 client.26291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f9a02d84b80 20 client.26291 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d0f6420210 0x55d0f6422600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f9a02d84b80 4 client.26291 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:11:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3729474294 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d0f62fd910 con 0x55d0f6420210 2023-08-25T09:11:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.294+0000 7f99ec99a700 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d0f6420210 0x55d0f6422600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.295+0000 7f99ec99a700 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d0f6420210 0x55d0f6422600 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556bf5756fd172af server_cookie=ecfed8b6ea940f22 in_seq=0 out_seq=0 2023-08-25T09:11:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.295+0000 7f99d0ff9700 10 client.26291 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.343+0000 7f99d0ff9700 1 -- 172.21.15.99:0/3729474294 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d0f62fd910 con 0x55d0f6420210 2023-08-25T09:11:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.343+0000 7f99d0ff9700 10 client.26291 handle_command_reply: tid=0 2023-08-25T09:11:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.343+0000 7f9a02d84b80 1 client.26291 shutdown 2023-08-25T09:11:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3729474294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 msgr2=0x55d0f63b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63b3880 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99e4075810 tx=0x7f99e4077c10 comp rx=0 tx=0).stop 2023-08-25T09:11:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3729474294 shutdown_connections 2023-08-25T09:11:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d0f63b8860 0x55d0f63b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d0f6420210 0x55d0f6422600 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0f63b2990 0x55d0f63b3880 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 --2- 172.21.15.99:0/3729474294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f63b7f40 0x55d0f63b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f99d0ff9700 0 client.26291 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3729474294 >> 172.21.15.99:0/3729474294 conn(0x55d0f6327080 msgr2=0x55d0f63b8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3729474294 shutdown_connections 2023-08-25T09:11:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 1 -- 172.21.15.99:0/3729474294 wait complete. 2023-08-25T09:11:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.344+0000 7f9a02d84b80 20 client.26291 trim_cache size 0 max 16384 2023-08-25T09:11:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:11:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:11:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:11:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:11:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 -- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99d40a00c0 msgr2=0x7f99d40a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99d40a00c0 0x7f99d40a2580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009cd0 tx=0x7f99e0009c20 comp rx=0 tx=0).stop 2023-08-25T09:11:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 -- 172.21.15.99:0/1261219315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8067d10 msgr2=0x7f99e819ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8067d10 0x7f99e819ea40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f99e400e9a0 tx=0x7f99e400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 -- 172.21.15.99:0/1261219315 shutdown_connections 2023-08-25T09:11:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99d40a00c0 0x7f99d40a2580 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e81068d0 0x7f99e81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8067d10 0x7f99e819ea40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 --2- 172.21.15.99:0/1261219315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e80686d0 0x7f99e819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.345+0000 7f99ef3ff700 1 -- 172.21.15.99:0/1261219315 >> 172.21.15.99:0/1261219315 conn(0x7f99e8075250 msgr2=0x7f99e810ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.346+0000 7f99ef3ff700 1 -- 172.21.15.99:0/1261219315 shutdown_connections 2023-08-25T09:11:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:41.346+0000 7f99ef3ff700 1 -- 172.21.15.99:0/1261219315 wait complete. 2023-08-25T09:11:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:41 smithi157 ceph-mon[130879]: pgmap v390: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 768 KiB/s wr, 97 op/s 2023-08-25T09:11:41.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 481360 3.82 MB/sec execute 587 sec latency 290.411 ms 2023-08-25T09:11:41.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:41 smithi181 ceph-mon[131296]: pgmap v390: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 768 KiB/s wr, 97 op/s 2023-08-25T09:11:42.062 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:41 smithi099 ceph-mon[97641]: pgmap v390: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 768 KiB/s wr, 97 op/s 2023-08-25T09:11:42.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 481360 3.82 MB/sec execute 588 sec latency 1290.488 ms 2023-08-25T09:11:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:43 smithi157 ceph-mon[130879]: pgmap v391: 97 pgs: 97 active+clean; 166 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-08-25T09:11:43.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 481360 3.81 MB/sec execute 589 sec latency 2290.639 ms 2023-08-25T09:11:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:43 smithi181 ceph-mon[131296]: pgmap v391: 97 pgs: 97 active+clean; 166 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-08-25T09:11:44.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:43 smithi099 ceph-mon[97641]: pgmap v391: 97 pgs: 97 active+clean; 166 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-08-25T09:11:44.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 482436 3.81 MB/sec execute 590 sec latency 2821.029 ms 2023-08-25T09:11:45.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:45 smithi099 ceph-mon[97641]: pgmap v392: 97 pgs: 97 active+clean; 166 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 971 KiB/s wr, 87 op/s 2023-08-25T09:11:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:45 smithi157 ceph-mon[130879]: pgmap v392: 97 pgs: 97 active+clean; 166 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 971 KiB/s wr, 87 op/s 2023-08-25T09:11:45.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:45 smithi181 ceph-mon[131296]: pgmap v392: 97 pgs: 97 active+clean; 166 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 971 KiB/s wr, 87 op/s 2023-08-25T09:11:45.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 483895 3.82 MB/sec execute 591 sec latency 14.350 ms 2023-08-25T09:11:46.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484911 3.82 MB/sec execute 592 sec latency 354.994 ms 2023-08-25T09:11:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:47 smithi181 ceph-mon[131296]: pgmap v393: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2023-08-25T09:11:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:47 smithi099 ceph-mon[97641]: pgmap v393: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2023-08-25T09:11:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:47 smithi157 ceph-mon[130879]: pgmap v393: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2023-08-25T09:11:47.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484911 3.82 MB/sec execute 593 sec latency 1355.110 ms 2023-08-25T09:11:48.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484911 3.81 MB/sec execute 594 sec latency 2355.270 ms 2023-08-25T09:11:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:49 smithi181 ceph-mon[131296]: pgmap v394: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:11:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:49 smithi099 ceph-mon[97641]: pgmap v394: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:11:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:49 smithi157 ceph-mon[130879]: pgmap v394: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:11:49.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 485992 3.81 MB/sec execute 595 sec latency 2887.862 ms 2023-08-25T09:11:50.954 INFO:tasks.workunit.client.0.smithi099.stdout: 1 487428 3.82 MB/sec execute 596 sec latency 54.387 ms 2023-08-25T09:11:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:51 smithi181 ceph-mon[131296]: pgmap v395: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:11:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:51 smithi099 ceph-mon[97641]: pgmap v395: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:11:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:51 smithi157 ceph-mon[130879]: pgmap v395: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-08-25T09:11:51.953 INFO:tasks.workunit.client.0.smithi099.stdout: 1 488453 3.82 MB/sec execute 597 sec latency 388.372 ms 2023-08-25T09:11:52.954 INFO:tasks.workunit.client.0.smithi099.stdout: 1 488453 3.82 MB/sec execute 598 sec latency 1388.496 ms 2023-08-25T09:11:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:53 smithi181 ceph-mon[131296]: pgmap v396: 97 pgs: 97 active+clean; 184 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.4 MiB/s wr, 170 op/s 2023-08-25T09:11:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:53 smithi099 ceph-mon[97641]: pgmap v396: 97 pgs: 97 active+clean; 184 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.4 MiB/s wr, 170 op/s 2023-08-25T09:11:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:53 smithi157 ceph-mon[130879]: pgmap v396: 97 pgs: 97 active+clean; 184 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.4 MiB/s wr, 170 op/s 2023-08-25T09:11:53.954 INFO:tasks.workunit.client.0.smithi099.stdout: 1 488453 3.81 MB/sec execute 599 sec latency 2388.608 ms 2023-08-25T09:11:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.504+0000 7f99cc9ff700 1 -- 172.21.15.99:0/2995303429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c8101260 msgr2=0x7f99c810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.504+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/2995303429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c8101260 0x7f99c810d770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f99c000b0a0 tx=0x7f99c000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.504+0000 7f99cc9ff700 1 -- 172.21.15.99:0/2995303429 shutdown_connections 2023-08-25T09:11:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.504+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/2995303429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c8101260 0x7f99c810d770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.504+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/2995303429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c81068b0 0x7f99c8106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.504+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/2995303429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c81008a0 0x7f99c8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.504+0000 7f99cc9ff700 1 -- 172.21.15.99:0/2995303429 >> 172.21.15.99:0/2995303429 conn(0x7f99c80fc370 msgr2=0x7f99c80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.505+0000 7f99cc9ff700 1 -- 172.21.15.99:0/2995303429 shutdown_connections 2023-08-25T09:11:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.505+0000 7f99cc9ff700 1 -- 172.21.15.99:0/2995303429 wait complete. 2023-08-25T09:11:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99cc9ff700 1 Processor -- start 2023-08-25T09:11:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99cc9ff700 1 -- start start 2023-08-25T09:11:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99cc9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c81008a0 0x7f99c819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99cc9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c8101260 0x7f99c819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99cc9ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c81068b0 0x7f99c8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99cc9ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8198f90 con 0x7f99c81068b0 2023-08-25T09:11:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99cc9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8199100 con 0x7f99c8101260 2023-08-25T09:11:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99cc9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c81992c0 con 0x7f99c81008a0 2023-08-25T09:11:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99c659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c81008a0 0x7f99c819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99c5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c8101260 0x7f99c819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99c5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c8101260 0x7f99c819f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52306/0 (socket says 172.21.15.99:52306) 2023-08-25T09:11:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99c659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c81008a0 0x7f99c819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:46776/0 (socket says 172.21.15.99:46776) 2023-08-25T09:11:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99c5d9b700 1 -- 172.21.15.99:0/4184283003 learned_addr learned my addr 172.21.15.99:0/4184283003 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.506+0000 7f99c6d9d700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c81068b0 0x7f99c8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99c5d9b700 1 -- 172.21.15.99:0/4184283003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c81008a0 msgr2=0x7f99c819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c81008a0 0x7f99c819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99c5d9b700 1 -- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c81068b0 msgr2=0x7f99c8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c81068b0 0x7f99c8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99c5d9b700 1 -- 172.21.15.99:0/4184283003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0009d00 con 0x7f99c8101260 2023-08-25T09:11:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c8101260 0x7f99c819f000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e9a0 tx=0x7f99bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99b77fe700 1 -- 172.21.15.99:0/4184283003 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc009790 con 0x7f99c8101260 2023-08-25T09:11:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99b77fe700 1 -- 172.21.15.99:0/4184283003 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99bc010430 con 0x7f99c8101260 2023-08-25T09:11:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99cc9ff700 1 -- 172.21.15.99:0/4184283003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c81995a0 con 0x7f99c8101260 2023-08-25T09:11:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.507+0000 7f99b77fe700 1 -- 172.21.15.99:0/4184283003 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc018680 con 0x7f99c8101260 2023-08-25T09:11:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.508+0000 7f99cc9ff700 1 -- 172.21.15.99:0/4184283003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81a5ab0 con 0x7f99c8101260 2023-08-25T09:11:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.511+0000 7f99b77fe700 1 -- 172.21.15.99:0/4184283003 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f99bc0105a0 con 0x7f99c8101260 2023-08-25T09:11:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.512+0000 7f99e0317b80 1 Processor -- start 2023-08-25T09:11:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.512+0000 7f99b77fe700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99b00a00c0 0x7f99b00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.512+0000 7f99e0317b80 1 -- start start 2023-08-25T09:11:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.512+0000 7f99b77fe700 1 -- 172.21.15.99:0/4184283003 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f99bc014070 con 0x7f99c8101260 2023-08-25T09:11:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c6d9d700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99b00a00c0 0x7f99b00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598bf40 0x55e9b598c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5986450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b5986990 0x55e9b5986dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99e0317b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b58ee750 con 0x55e9b598bf40 2023-08-25T09:11:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99e0317b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b58df720 con 0x55e9b598c860 2023-08-25T09:11:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99e0317b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b58dd040 con 0x55e9b5986990 2023-08-25T09:11:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5986450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5986450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52312/0 (socket says 172.21.15.99:52312) 2023-08-25T09:11:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c5d9b700 1 -- 172.21.15.99:0/1579541985 learned_addr learned my addr 172.21.15.99:0/1579541985 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c659c700 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598bf40 0x55e9b598c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c6d9d700 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b5986990 0x55e9b5986dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c5d9b700 1 -- 172.21.15.99:0/1579541985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b5986990 msgr2=0x55e9b5986dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b5986990 0x55e9b5986dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c5d9b700 1 -- 172.21.15.99:0/1579541985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598bf40 msgr2=0x55e9b598c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598bf40 0x55e9b598c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c5d9b700 1 -- 172.21.15.99:0/1579541985 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9b57585d0 con 0x55e9b598c860 2023-08-25T09:11:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.513+0000 7f99c659c700 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598bf40 0x55e9b598c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:11:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.514+0000 7f99c6d9d700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99b00a00c0 0x7f99b00a2580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99c000b780 tx=0x7f99c0006230 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.514+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5986450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99bc095990 tx=0x7f99bc098b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8c4998c03a02eb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.514+0000 7f99b4ff9700 1 -- 172.21.15.99:0/1579541985 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0a0d30 con 0x55e9b598c860 2023-08-25T09:11:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.514+0000 7f99b4ff9700 1 -- 172.21.15.99:0/1579541985 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99bc010880 con 0x55e9b598c860 2023-08-25T09:11:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.514+0000 7f99b4ff9700 1 -- 172.21.15.99:0/1579541985 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0a08a0 con 0x55e9b598c860 2023-08-25T09:11:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 -- 172.21.15.99:0/1579541985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 msgr2=0x55e9b5986450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5986450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99bc095990 tx=0x7f99bc098b90 comp rx=0 tx=0).stop 2023-08-25T09:11:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 -- 172.21.15.99:0/1579541985 shutdown_connections 2023-08-25T09:11:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b5986990 0x55e9b5986dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598bf40 0x55e9b598c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 --2- 172.21.15.99:0/1579541985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5986450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 -- 172.21.15.99:0/1579541985 >> 172.21.15.99:0/1579541985 conn(0x55e9b58fb080 msgr2=0x55e9b5980d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 -- 172.21.15.99:0/1579541985 shutdown_connections 2023-08-25T09:11:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 -- 172.21.15.99:0/1579541985 wait complete. 2023-08-25T09:11:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.515+0000 7f99e0317b80 1 -- start start 2023-08-25T09:11:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99e0317b80 1 Processor -- start 2023-08-25T09:11:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b5992720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598bf40 0x55e9b598f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598c860 0x55e9b5990070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99e0317b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b58d8930 con 0x55e9b5986990 2023-08-25T09:11:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99e0317b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5876b50 con 0x55e9b598bf40 2023-08-25T09:11:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99e0317b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5876850 con 0x55e9b598c860 2023-08-25T09:11:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99e0317b80 10 client.0 _ll_register_callbacks cb 0x55e9b57c0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b5992720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99c6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598c860 0x55e9b5990070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b5992720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46880/0 (socket says 172.21.15.99:46880) 2023-08-25T09:11:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99c659c700 1 -- 172.21.15.99:0/3909210467 learned_addr learned my addr 172.21.15.99:0/3909210467 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.516+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598bf40 0x55e9b598f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99c659c700 1 -- 172.21.15.99:0/3909210467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598c860 msgr2=0x55e9b5990070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99c659c700 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598c860 0x55e9b5990070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99c659c700 1 -- 172.21.15.99:0/3909210467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598bf40 msgr2=0x55e9b598f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99c659c700 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598bf40 0x55e9b598f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99c659c700 1 -- 172.21.15.99:0/3909210467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81a5ab0 con 0x55e9b5986990 2023-08-25T09:11:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99c6d9d700 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598c860 0x55e9b5990070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:11:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99c659c700 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b5992720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f99b80116a0 tx=0x7f99b80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d866415a6103a21f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99adffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:11:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99adffb700 1 -- 172.21.15.99:0/3909210467 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8016070 con 0x55e9b5986990 2023-08-25T09:11:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99e0317b80 1 -- 172.21.15.99:0/3909210467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9b5956160 con 0x55e9b5986990 2023-08-25T09:11:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.517+0000 7f99adffb700 1 -- 172.21.15.99:0/3909210467 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f99b800f630 con 0x55e9b5986990 2023-08-25T09:11:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99adffb700 1 -- 172.21.15.99:0/3909210467 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b800f7f0 con 0x55e9b5986990 2023-08-25T09:11:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99e0317b80 10 client.16242 fetch_fsmap learned FSMap version 40 2023-08-25T09:11:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99e0317b80 1 -- 172.21.15.99:0/3909210467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e9b5704d80 con 0x55e9b5986990 2023-08-25T09:11:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99adffb700 1 -- 172.21.15.99:0/3909210467 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9b5956160 con 0x55e9b5986990 2023-08-25T09:11:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99adffb700 1 -- 172.21.15.99:0/3909210467 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99b801aa70 con 0x55e9b5986990 2023-08-25T09:11:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99e0317b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 40 2023-08-25T09:11:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99e0317b80 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99e0317b80 20 client.16242 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99e0317b80 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e9b59a0270 0x55e9b59a2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99e0317b80 4 client.16242 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:11:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99e0317b80 1 -- 172.21.15.99:0/3909210467 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9b587eb40 con 0x55e9b59a0270 2023-08-25T09:11:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.518+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e9b59a0270 0x55e9b59a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.519+0000 7f99adffb700 10 client.16242 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.519+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e9b59a0270 0x55e9b59a2690 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c65c160e768754 server_cookie=ef7714b22806d938 in_seq=0 out_seq=0 2023-08-25T09:11:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99adffb700 1 -- 172.21.15.99:0/3909210467 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9b587eb40 con 0x55e9b59a0270 2023-08-25T09:11:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99adffb700 10 client.16242 handle_command_reply: tid=0 2023-08-25T09:11:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 client.16242 shutdown 2023-08-25T09:11:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 -- 172.21.15.99:0/3909210467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 msgr2=0x55e9b5992720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b5992720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f99b80116a0 tx=0x7f99b80119b0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 -- 172.21.15.99:0/3909210467 shutdown_connections 2023-08-25T09:11:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598c860 0x55e9b5990070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b5992720 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e9b59a0270 0x55e9b59a2690 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 --2- 172.21.15.99:0/3909210467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598bf40 0x55e9b598f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99adffb700 0 client.16242 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 -- 172.21.15.99:0/3909210467 >> 172.21.15.99:0/3909210467 conn(0x55e9b58fb080 msgr2=0x55e9b597b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 -- 172.21.15.99:0/3909210467 shutdown_connections 2023-08-25T09:11:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 1 -- 172.21.15.99:0/3909210467 wait complete. 2023-08-25T09:11:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.520+0000 7f99e0317b80 20 client.16242 trim_cache size 0 max 16384 2023-08-25T09:11:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.540+0000 7f99e0317b80 1 Processor -- start 2023-08-25T09:11:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.540+0000 7f99e0317b80 1 -- start start 2023-08-25T09:11:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.540+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b598ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.540+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b59971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.540+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5980bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.540+0000 7f99e0317b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b575a8c0 con 0x55e9b5986990 2023-08-25T09:11:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.540+0000 7f99e0317b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5979460 con 0x55e9b598c860 2023-08-25T09:11:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.540+0000 7f99e0317b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5997730 con 0x55e9b598bf40 2023-08-25T09:11:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b598ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b59971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b59971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:46796/0 (socket says 172.21.15.99:46796) 2023-08-25T09:11:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c5d9b700 1 -- 172.21.15.99:0/2374872748 learned_addr learned my addr 172.21.15.99:0/2374872748 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c6d9d700 1 --2- 172.21.15.99:0/2374872748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5980bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c659c700 1 -- 172.21.15.99:0/2374872748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 msgr2=0x55e9b59971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c659c700 1 --2- 172.21.15.99:0/2374872748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b59971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c659c700 1 -- 172.21.15.99:0/2374872748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 msgr2=0x55e9b5980bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c659c700 1 --2- 172.21.15.99:0/2374872748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5980bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c659c700 1 -- 172.21.15.99:0/2374872748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9b56edc00 con 0x55e9b5986990 2023-08-25T09:11:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99c659c700 1 --2- 172.21.15.99:0/2374872748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b598ec50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f99b801ae50 tx=0x7f99b8009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2d5a92047639f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.541+0000 7f99b57fa700 1 -- 172.21.15.99:0/2374872748 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8016070 con 0x55e9b5986990 2023-08-25T09:11:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99b57fa700 1 -- 172.21.15.99:0/2374872748 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b8011db0 con 0x55e9b5986990 2023-08-25T09:11:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99b57fa700 1 -- 172.21.15.99:0/2374872748 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8005460 con 0x55e9b5986990 2023-08-25T09:11:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 -- 172.21.15.99:0/2374872748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 msgr2=0x55e9b598ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 --2- 172.21.15.99:0/2374872748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b598ec50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f99b801ae50 tx=0x7f99b8009770 comp rx=0 tx=0).stop 2023-08-25T09:11:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 -- 172.21.15.99:0/2374872748 shutdown_connections 2023-08-25T09:11:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 --2- 172.21.15.99:0/2374872748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b598c860 0x55e9b5980bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 --2- 172.21.15.99:0/2374872748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b5986990 0x55e9b598ec50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 --2- 172.21.15.99:0/2374872748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b59971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 -- 172.21.15.99:0/2374872748 >> 172.21.15.99:0/2374872748 conn(0x55e9b58fb080 msgr2=0x55e9b59a6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 -- 172.21.15.99:0/2374872748 shutdown_connections 2023-08-25T09:11:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.542+0000 7f99e0317b80 1 -- 172.21.15.99:0/2374872748 wait complete. 2023-08-25T09:11:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.543+0000 7f99e0317b80 1 -- start start 2023-08-25T09:11:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.543+0000 7f99e0317b80 1 Processor -- start 2023-08-25T09:11:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.543+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5986990 0x55e9b59951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.543+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b5995710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.543+0000 7f99e0317b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598c860 0x55e9b59f3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.543+0000 7f99e0317b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5995ff0 con 0x55e9b598c860 2023-08-25T09:11:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99e0317b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b59f35a0 con 0x55e9b5986990 2023-08-25T09:11:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99e0317b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b59f36e0 con 0x55e9b598bf40 2023-08-25T09:11:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99e0317b80 10 client.0 _ll_register_callbacks cb 0x7f99b8009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:11:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5986990 0x55e9b59951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5986990 0x55e9b59951d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:52344/0 (socket says 172.21.15.99:52344) 2023-08-25T09:11:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c659c700 1 -- 172.21.15.99:0/4210793032 learned_addr learned my addr 172.21.15.99:0/4210793032 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:11:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b5995710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c6d9d700 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598c860 0x55e9b59f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c5d9b700 1 -- 172.21.15.99:0/4210793032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5986990 msgr2=0x55e9b59951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5986990 0x55e9b59951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c5d9b700 1 -- 172.21.15.99:0/4210793032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598c860 msgr2=0x55e9b59f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:11:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598c860 0x55e9b59f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.544+0000 7f99c5d9b700 1 -- 172.21.15.99:0/4210793032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9b5704d80 con 0x55e9b598bf40 2023-08-25T09:11:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99c5d9b700 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b5995710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f99bc098ad0 tx=0x7f99bc099060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c046aa483d936687 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:11:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99adffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:11:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99adffb700 1 -- 172.21.15.99:0/4210793032 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc099180 con 0x55e9b598bf40 2023-08-25T09:11:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 1 -- 172.21.15.99:0/4210793032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9b5979600 con 0x55e9b598bf40 2023-08-25T09:11:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99adffb700 1 -- 172.21.15.99:0/4210793032 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f99bc0992e0 con 0x55e9b598bf40 2023-08-25T09:11:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99adffb700 1 -- 172.21.15.99:0/4210793032 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99bc098410 con 0x55e9b598bf40 2023-08-25T09:11:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 10 client.26290 fetch_fsmap learned FSMap version 40 2023-08-25T09:11:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 1 -- 172.21.15.99:0/4210793032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e9b572b1f0 con 0x55e9b598bf40 2023-08-25T09:11:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99adffb700 1 -- 172.21.15.99:0/4210793032 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9b5979600 con 0x55e9b598bf40 2023-08-25T09:11:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99adffb700 1 -- 172.21.15.99:0/4210793032 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99bc0b6a20 con 0x55e9b598bf40 2023-08-25T09:11:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 10 client.26290 fetch_fsmap finished waiting for FSMap version 40 2023-08-25T09:11:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 10 client.26290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:11:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 20 client.26290 populate_metadata read hostname 'smithi099' 2023-08-25T09:11:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e9b59f4b00 0x55e9b59f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:11:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 4 client.26290 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:11:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.545+0000 7f99e0317b80 1 -- 172.21.15.99:0/4210793032 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e9b58d1910 con 0x55e9b59f4b00 2023-08-25T09:11:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.546+0000 7f99c659c700 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e9b59f4b00 0x55e9b59f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:11:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.546+0000 7f99adffb700 10 client.26290 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.546+0000 7f99c659c700 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e9b59f4b00 0x55e9b59f6f20 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caeed973e254982e server_cookie=51f948b14243327e in_seq=0 out_seq=0 2023-08-25T09:11:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.587+0000 7f99adffb700 1 -- 172.21.15.99:0/4210793032 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e9b58d1910 con 0x55e9b59f4b00 2023-08-25T09:11:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.587+0000 7f99adffb700 10 client.26290 handle_command_reply: tid=0 2023-08-25T09:11:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.587+0000 7f99e0317b80 1 client.26290 shutdown 2023-08-25T09:11:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 -- 172.21.15.99:0/4210793032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 msgr2=0x55e9b5995710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b5995710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f99bc098ad0 tx=0x7f99bc099060 comp rx=0 tx=0).stop 2023-08-25T09:11:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 -- 172.21.15.99:0/4210793032 shutdown_connections 2023-08-25T09:11:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b598c860 0x55e9b59f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55e9b59f4b00 0x55e9b59f6f20 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5986990 0x55e9b59951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99adffb700 0 client.26290 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:11:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 --2- 172.21.15.99:0/4210793032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b598bf40 0x55e9b5995710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 -- 172.21.15.99:0/4210793032 >> 172.21.15.99:0/4210793032 conn(0x55e9b58fb080 msgr2=0x55e9b5980bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 -- 172.21.15.99:0/4210793032 shutdown_connections 2023-08-25T09:11:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 1 -- 172.21.15.99:0/4210793032 wait complete. 2023-08-25T09:11:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.588+0000 7f99e0317b80 20 client.26290 trim_cache size 0 max 16384 2023-08-25T09:11:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:11:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:11:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:11:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:11:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 -- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99b00a00c0 msgr2=0x7f99b00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99b00a00c0 0x7f99b00a2580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99c000b780 tx=0x7f99c0006230 comp rx=0 tx=0).stop 2023-08-25T09:11:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 -- 172.21.15.99:0/4184283003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c8101260 msgr2=0x7f99c819f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:11:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c8101260 0x7f99c819f000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e9a0 tx=0x7f99bc00ecb0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 -- 172.21.15.99:0/4184283003 shutdown_connections 2023-08-25T09:11:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99b00a00c0 0x7f99b00a2580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c81068b0 0x7f99c8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c81008a0 0x7f99c819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 --2- 172.21.15.99:0/4184283003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c8101260 0x7f99c819f000 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:11:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 -- 172.21.15.99:0/4184283003 >> 172.21.15.99:0/4184283003 conn(0x7f99c80fc370 msgr2=0x7f99c810bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:11:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 -- 172.21.15.99:0/4184283003 shutdown_connections 2023-08-25T09:11:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:11:54.589+0000 7f99cc9ff700 1 -- 172.21.15.99:0/4184283003 wait complete. 2023-08-25T09:11:54.954 INFO:tasks.workunit.client.0.smithi099.stdout: 1 cleanup 600 sec 2023-08-25T09:11:55.137 INFO:tasks.workunit.client.0.smithi099.stdout: 0 cleanup 600 sec 2023-08-25T09:11:55.137 INFO:tasks.workunit.client.0.smithi099.stdout: 2023-08-25T09:11:55.137 INFO:tasks.workunit.client.0.smithi099.stdout: Operation Count AvgLat MaxLat 2023-08-25T09:11:55.137 INFO:tasks.workunit.client.0.smithi099.stdout: ---------------------------------------- 2023-08-25T09:11:55.138 INFO:tasks.workunit.client.0.smithi099.stdout: NTCreateX 72820 3.997 16199.700 2023-08-25T09:11:55.138 INFO:tasks.workunit.client.0.smithi099.stdout: Close 53534 0.005 0.184 2023-08-25T09:11:55.138 INFO:tasks.workunit.client.0.smithi099.stdout: Rename 3084 46.570 4594.912 2023-08-25T09:11:55.138 INFO:tasks.workunit.client.0.smithi099.stdout: Unlink 14672 2.088 1933.128 2023-08-25T09:11:55.138 INFO:tasks.workunit.client.0.smithi099.stdout: Deltree 2 91.059 121.044 2023-08-25T09:11:55.138 INFO:tasks.workunit.client.0.smithi099.stdout: Mkdir 1 0.015 0.015 2023-08-25T09:11:55.138 INFO:tasks.workunit.client.0.smithi099.stdout: Qpathinfo 66014 1.580 3368.499 2023-08-25T09:11:55.138 INFO:tasks.workunit.client.0.smithi099.stdout: Qfileinfo 11580 0.002 0.066 2023-08-25T09:11:55.139 INFO:tasks.workunit.client.0.smithi099.stdout: Qfsinfo 12080 0.250 11.120 2023-08-25T09:11:55.139 INFO:tasks.workunit.client.0.smithi099.stdout: Sfileinfo 5949 0.043 4.375 2023-08-25T09:11:55.139 INFO:tasks.workunit.client.0.smithi099.stdout: Find 25508 0.249 88.775 2023-08-25T09:11:55.139 INFO:tasks.workunit.client.0.smithi099.stdout: WriteX 36356 0.109 32.050 2023-08-25T09:11:55.139 INFO:tasks.workunit.client.0.smithi099.stdout: ReadX 114016 0.016 17.693 2023-08-25T09:11:55.139 INFO:tasks.workunit.client.0.smithi099.stdout: LockX 236 0.815 10.315 2023-08-25T09:11:55.139 INFO:tasks.workunit.client.0.smithi099.stdout: UnlockX 236 0.824 7.582 2023-08-25T09:11:55.140 INFO:tasks.workunit.client.0.smithi099.stdout: Flush 5108 2.857 45.471 2023-08-25T09:11:55.140 INFO:tasks.workunit.client.0.smithi099.stdout: 2023-08-25T09:11:55.140 INFO:tasks.workunit.client.0.smithi099.stdout:Throughput 3.81057 MB/sec 1 clients 1 procs max_latency=16199.717 ms 2023-08-25T09:11:55.140 INFO:tasks.workunit.client.0.smithi099.stderr:+ dbench 10 2023-08-25T09:11:55.141 INFO:tasks.workunit.client.0.smithi099.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-08-25T09:11:55.141 INFO:tasks.workunit.client.0.smithi099.stdout: 2023-08-25T09:11:55.141 INFO:tasks.workunit.client.0.smithi099.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-08-25T09:11:55.142 INFO:tasks.workunit.client.0.smithi099.stdout:9 of 10 processes prepared for launch 0 sec 2023-08-25T09:11:55.242 INFO:tasks.workunit.client.0.smithi099.stdout:10 of 10 processes prepared for launch 0 sec 2023-08-25T09:11:55.242 INFO:tasks.workunit.client.0.smithi099.stdout:releasing clients 2023-08-25T09:11:55.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:55 smithi099 ceph-mon[97641]: pgmap v397: 97 pgs: 97 active+clean; 184 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.8 MiB/s wr, 121 op/s 2023-08-25T09:11:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:55 smithi157 ceph-mon[130879]: pgmap v397: 97 pgs: 97 active+clean; 184 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.8 MiB/s wr, 121 op/s 2023-08-25T09:11:55.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:55 smithi181 ceph-mon[131296]: pgmap v397: 97 pgs: 97 active+clean; 184 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.8 MiB/s wr, 121 op/s 2023-08-25T09:11:56.242 INFO:tasks.workunit.client.0.smithi099.stdout: 10 372 133.04 MB/sec warmup 1 sec latency 49.682 ms 2023-08-25T09:11:57.242 INFO:tasks.workunit.client.0.smithi099.stdout: 10 778 104.60 MB/sec warmup 2 sec latency 103.110 ms 2023-08-25T09:11:57.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:57 smithi181 ceph-mon[131296]: pgmap v398: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 3.3 MiB/s wr, 183 op/s 2023-08-25T09:11:57.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:57 smithi099 ceph-mon[97641]: pgmap v398: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 3.3 MiB/s wr, 183 op/s 2023-08-25T09:11:57.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:57 smithi157 ceph-mon[130879]: pgmap v398: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 3.3 MiB/s wr, 183 op/s 2023-08-25T09:11:58.242 INFO:tasks.workunit.client.0.smithi099.stdout: 10 1261 79.35 MB/sec warmup 3 sec latency 164.324 ms 2023-08-25T09:11:59.242 INFO:tasks.workunit.client.0.smithi099.stdout: 10 2204 67.90 MB/sec warmup 4 sec latency 54.103 ms 2023-08-25T09:11:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:59 smithi181 ceph-mon[131296]: pgmap v399: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2023-08-25T09:11:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:11:59 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:11:59.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:59 smithi099 ceph-mon[97641]: pgmap v399: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2023-08-25T09:11:59.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:11:59 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:11:59.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:59 smithi157 ceph-mon[130879]: pgmap v399: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2023-08-25T09:11:59.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:11:59 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:00.242 INFO:tasks.workunit.client.0.smithi099.stdout: 10 2702 63.08 MB/sec warmup 5 sec latency 94.220 ms 2023-08-25T09:12:01.242 INFO:tasks.workunit.client.0.smithi099.stdout: 10 3095 56.60 MB/sec warmup 6 sec latency 830.778 ms 2023-08-25T09:12:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:01 smithi181 ceph-mon[131296]: pgmap v400: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2023-08-25T09:12:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:01 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:01 smithi099 ceph-mon[97641]: pgmap v400: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2023-08-25T09:12:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:01 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:01.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:01 smithi157 ceph-mon[130879]: pgmap v400: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2023-08-25T09:12:01.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:01 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:02.242 INFO:tasks.workunit.client.0.smithi099.stdout: 10 3437 53.49 MB/sec warmup 7 sec latency 1830.825 ms 2023-08-25T09:12:03.242 INFO:tasks.workunit.client.0.smithi099.stdout: 10 3649 48.05 MB/sec warmup 8 sec latency 2830.874 ms 2023-08-25T09:12:03.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:03 smithi181 ceph-mon[131296]: pgmap v401: 97 pgs: 97 active+clean; 186 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 349 op/s 2023-08-25T09:12:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:03 smithi099 ceph-mon[97641]: pgmap v401: 97 pgs: 97 active+clean; 186 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 349 op/s 2023-08-25T09:12:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:03 smithi157 ceph-mon[130879]: pgmap v401: 97 pgs: 97 active+clean; 186 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 349 op/s 2023-08-25T09:12:04.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 3798 43.02 MB/sec warmup 9 sec latency 3830.972 ms 2023-08-25T09:12:04.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:04 smithi181 ceph-mon[131296]: osdmap e101: 12 total, 12 up, 12 in 2023-08-25T09:12:04.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:04 smithi099 ceph-mon[97641]: osdmap e101: 12 total, 12 up, 12 in 2023-08-25T09:12:04.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:04 smithi157 ceph-mon[130879]: osdmap e101: 12 total, 12 up, 12 in 2023-08-25T09:12:05.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 4074 39.10 MB/sec warmup 10 sec latency 4079.295 ms 2023-08-25T09:12:05.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:05 smithi181 ceph-mon[131296]: pgmap v403: 97 pgs: 97 active+clean; 186 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2023-08-25T09:12:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:05 smithi099 ceph-mon[97641]: pgmap v403: 97 pgs: 97 active+clean; 186 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2023-08-25T09:12:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:05 smithi157 ceph-mon[130879]: pgmap v403: 97 pgs: 97 active+clean; 186 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2023-08-25T09:12:06.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 4517 37.28 MB/sec warmup 11 sec latency 97.781 ms 2023-08-25T09:12:07.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 5104 36.60 MB/sec warmup 12 sec latency 41.792 ms 2023-08-25T09:12:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:07 smithi181 ceph-mon[131296]: pgmap v404: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 531 op/s 2023-08-25T09:12:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 -- 172.21.15.99:0/3496104146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 msgr2=0x7f01f810d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 --2- 172.21.15.99:0/3496104146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 0x7f01f810d6e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01f000b0a0 tx=0x7f01f000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 -- 172.21.15.99:0/3496104146 shutdown_connections 2023-08-25T09:12:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 --2- 172.21.15.99:0/3496104146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 0x7f01f810d6e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 --2- 172.21.15.99:0/3496104146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01f8100ad0 0x7f01f8100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 --2- 172.21.15.99:0/3496104146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f8102210 0x7f01f8102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 -- 172.21.15.99:0/3496104146 >> 172.21.15.99:0/3496104146 conn(0x7f01f80fc1e0 msgr2=0x7f01f80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 -- 172.21.15.99:0/3496104146 shutdown_connections 2023-08-25T09:12:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.760+0000 7f01feb9d700 1 -- 172.21.15.99:0/3496104146 wait complete. 2023-08-25T09:12:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.762+0000 7f01feb9d700 1 Processor -- start 2023-08-25T09:12:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.762+0000 7f01feb9d700 1 -- start start 2023-08-25T09:12:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.762+0000 7f01feb9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f8100ad0 0x7f01f819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.762+0000 7f01feb9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01f8102210 0x7f01f819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.762+0000 7f01feb9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 0x7f01f8196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01feb9d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f8196f70 con 0x7f01f8100ad0 2023-08-25T09:12:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01feb9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f81970e0 con 0x7f01f8102bd0 2023-08-25T09:12:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01feb9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f81972a0 con 0x7f01f8102210 2023-08-25T09:12:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fe39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 0x7f01f8196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fd39a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01f8102210 0x7f01f819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fe39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 0x7f01f8196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:60554/0 (socket says 172.21.15.99:60554) 2023-08-25T09:12:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fe39c700 1 -- 172.21.15.99:0/2334845115 learned_addr learned my addr 172.21.15.99:0/2334845115 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fdb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f8100ad0 0x7f01f819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fd39a700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01f8102210 0x7f01f819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58014/0 (socket says 172.21.15.99:58014) 2023-08-25T09:12:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fd39a700 1 -- 172.21.15.99:0/2334845115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 msgr2=0x7f01f8196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fd39a700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 0x7f01f8196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fd39a700 1 -- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f8100ad0 msgr2=0x7f01f819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fd39a700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f8100ad0 0x7f01f819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fd39a700 1 -- 172.21.15.99:0/2334845115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0009d00 con 0x7f01f8102210 2023-08-25T09:12:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.763+0000 7f01fd39a700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01f8102210 0x7f01f819cdd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01e800d950 tx=0x7f01e800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.764+0000 7f01eeffd700 1 -- 172.21.15.99:0/2334845115 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e800f890 con 0x7f01f8102210 2023-08-25T09:12:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.764+0000 7f01feb9d700 1 -- 172.21.15.99:0/2334845115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f8197580 con 0x7f01f8102210 2023-08-25T09:12:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.764+0000 7f01eeffd700 1 -- 172.21.15.99:0/2334845115 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01e8010460 con 0x7f01f8102210 2023-08-25T09:12:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.764+0000 7f01eeffd700 1 -- 172.21.15.99:0/2334845115 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e800e610 con 0x7f01f8102210 2023-08-25T09:12:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.764+0000 7f01feb9d700 1 -- 172.21.15.99:0/2334845115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f81a3880 con 0x7f01f8102210 2023-08-25T09:12:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.767+0000 7f01eeffd700 1 -- 172.21.15.99:0/2334845115 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f01e80108d0 con 0x7f01f8102210 2023-08-25T09:12:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.769+0000 7f01eeffd700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01e40a00c0 0x7f01e40a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.769+0000 7f02136fab80 1 Processor -- start 2023-08-25T09:12:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.769+0000 7f01eeffd700 1 -- 172.21.15.99:0/2334845115 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f01e80cb330 con 0x7f01f8102210 2023-08-25T09:12:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.769+0000 7f02136fab80 1 -- start start 2023-08-25T09:12:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01e40a00c0 0x7f01e40a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f02136fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x5580484863f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f02136fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486930 0x558048480450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f02136fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x558048480df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f02136fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580483e8750 con 0x558048486930 2023-08-25T09:12:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f02136fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580483d9720 con 0x558048480990 2023-08-25T09:12:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f02136fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580483d7040 con 0x558048486010 2023-08-25T09:12:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f01fdb9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x5580484863f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f01fdb9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x5580484863f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58020/0 (socket says 172.21.15.99:58020) 2023-08-25T09:12:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.770+0000 7f01fdb9b700 1 -- 172.21.15.99:0/1318572079 learned_addr learned my addr 172.21.15.99:0/1318572079 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fd39a700 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486930 0x558048480450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fe39c700 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x558048480df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fdb9b700 1 -- 172.21.15.99:0/1318572079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 msgr2=0x558048480df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x558048480df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fdb9b700 1 -- 172.21.15.99:0/1318572079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486930 msgr2=0x558048480450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486930 0x558048480450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fdb9b700 1 -- 172.21.15.99:0/1318572079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580482525d0 con 0x558048486010 2023-08-25T09:12:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01e40a00c0 0x7f01e40a2580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01f4011630 tx=0x7f01f40119f0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x5580484863f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01f4004280 tx=0x7f01f401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1428721b861564c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.771+0000 7f01e3fff700 1 -- 172.21.15.99:0/1318572079 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4003c50 con 0x558048486010 2023-08-25T09:12:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.772+0000 7f01e3fff700 1 -- 172.21.15.99:0/1318572079 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f4003db0 con 0x558048486010 2023-08-25T09:12:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.772+0000 7f01e3fff700 1 -- 172.21.15.99:0/1318572079 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4005660 con 0x558048486010 2023-08-25T09:12:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.772+0000 7f02136fab80 1 -- 172.21.15.99:0/1318572079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 msgr2=0x5580484863f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.773+0000 7f02136fab80 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x5580484863f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01f4004280 tx=0x7f01f401b000 comp rx=0 tx=0).stop 2023-08-25T09:12:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.773+0000 7f02136fab80 1 -- 172.21.15.99:0/1318572079 shutdown_connections 2023-08-25T09:12:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.773+0000 7f02136fab80 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x558048480df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.773+0000 7f02136fab80 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x5580484863f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.773+0000 7f02136fab80 1 --2- 172.21.15.99:0/1318572079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486930 0x558048480450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.773+0000 7f02136fab80 1 -- 172.21.15.99:0/1318572079 >> 172.21.15.99:0/1318572079 conn(0x5580483f5080 msgr2=0x55804847afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.773+0000 7f02136fab80 1 -- 172.21.15.99:0/1318572079 shutdown_connections 2023-08-25T09:12:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.773+0000 7f02136fab80 1 -- 172.21.15.99:0/1318572079 wait complete. 2023-08-25T09:12:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.774+0000 7f02136fab80 1 -- start start 2023-08-25T09:12:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.774+0000 7f02136fab80 1 Processor -- start 2023-08-25T09:12:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f02136fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 0x55804848c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f02136fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f02136fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048490370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f02136fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580483d2930 con 0x558048480990 2023-08-25T09:12:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f02136fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558048370b50 con 0x558048486930 2023-08-25T09:12:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f02136fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558048370850 con 0x558048486010 2023-08-25T09:12:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f02136fab80 10 client.0 _ll_register_callbacks cb 0x5580482baa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:12:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fd39a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fe39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048490370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fdb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 0x55804848c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fd39a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58030/0 (socket says 172.21.15.99:58030) 2023-08-25T09:12:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fd39a700 1 -- 172.21.15.99:0/73264808 learned_addr learned my addr 172.21.15.99:0/73264808 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fe39c700 1 -- 172.21.15.99:0/73264808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 msgr2=0x55804848fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fe39c700 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fe39c700 1 -- 172.21.15.99:0/73264808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 msgr2=0x55804848c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fe39c700 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 0x55804848c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fe39c700 1 -- 172.21.15.99:0/73264808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f4004040 con 0x558048486930 2023-08-25T09:12:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.775+0000 7f01fd39a700 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:12:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f01fe39c700 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048490370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f01f819d590 tx=0x7f01f00095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=762453ee3d750fdf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f01e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:12:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f01e17fa700 1 -- 172.21.15.99:0/73264808 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000e040 con 0x558048486930 2023-08-25T09:12:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f02136fab80 1 -- 172.21.15.99:0/73264808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558048450160 con 0x558048486930 2023-08-25T09:12:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f01e17fa700 1 -- 172.21.15.99:0/73264808 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f01f00074e0 con 0x558048486930 2023-08-25T09:12:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f01e17fa700 1 -- 172.21.15.99:0/73264808 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01f0004ea0 con 0x558048486930 2023-08-25T09:12:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f02136fab80 10 client.26318 fetch_fsmap learned FSMap version 42 2023-08-25T09:12:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f02136fab80 1 -- 172.21.15.99:0/73264808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5580481fed80 con 0x558048486930 2023-08-25T09:12:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f01e17fa700 1 -- 172.21.15.99:0/73264808 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558048450160 con 0x558048486930 2023-08-25T09:12:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f01e17fa700 1 -- 172.21.15.99:0/73264808 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f01f0019070 con 0x558048486930 2023-08-25T09:12:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f02136fab80 10 client.26318 fetch_fsmap finished waiting for FSMap version 42 2023-08-25T09:12:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f02136fab80 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:12:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.776+0000 7f02136fab80 20 client.26318 populate_metadata read hostname 'smithi099' 2023-08-25T09:12:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.777+0000 7f02136fab80 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5580484a2b90 0x5580484a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.777+0000 7f02136fab80 4 client.26318 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:12:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.777+0000 7f02136fab80 1 -- 172.21.15.99:0/73264808 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558048378b40 con 0x5580484a2b90 2023-08-25T09:12:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.777+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5580484a2b90 0x5580484a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.777+0000 7f01e17fa700 10 client.26318 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.777+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5580484a2b90 0x5580484a4fb0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74df725ea01f2af1 server_cookie=ff6851beb6003655 in_seq=0 out_seq=0 2023-08-25T09:12:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f01e17fa700 1 -- 172.21.15.99:0/73264808 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558048378b40 con 0x5580484a2b90 2023-08-25T09:12:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f01e17fa700 10 client.26318 handle_command_reply: tid=0 2023-08-25T09:12:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 client.26318 shutdown 2023-08-25T09:12:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 -- 172.21.15.99:0/73264808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 msgr2=0x558048490370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048490370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f01f819d590 tx=0x7f01f00095d0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 -- 172.21.15.99:0/73264808 shutdown_connections 2023-08-25T09:12:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5580484a2b90 0x5580484a4fb0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048490370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 0x55804848c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 --2- 172.21.15.99:0/73264808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f01e17fa700 0 client.26318 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 -- 172.21.15.99:0/73264808 >> 172.21.15.99:0/73264808 conn(0x5580483f5080 msgr2=0x55804847f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 -- 172.21.15.99:0/73264808 shutdown_connections 2023-08-25T09:12:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 1 -- 172.21.15.99:0/73264808 wait complete. 2023-08-25T09:12:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.780+0000 7f02136fab80 20 client.26318 trim_cache size 0 max 16384 2023-08-25T09:12:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f02136fab80 1 Processor -- start 2023-08-25T09:12:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f02136fab80 1 -- start start 2023-08-25T09:12:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f02136fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 0x55804848a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f02136fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f02136fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048483aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f02136fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580482548c0 con 0x558048480990 2023-08-25T09:12:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f02136fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558048490dc0 con 0x558048486930 2023-08-25T09:12:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f02136fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804847bbb0 con 0x558048486010 2023-08-25T09:12:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f01fd39a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f01fe39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048483aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f01fdb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 0x55804848a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f01fd39a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:58046/0 (socket says 172.21.15.99:58046) 2023-08-25T09:12:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.797+0000 7f01fd39a700 1 -- 172.21.15.99:0/2884471281 learned_addr learned my addr 172.21.15.99:0/2884471281 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01fd39a700 1 -- 172.21.15.99:0/2884471281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 msgr2=0x558048483aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01fd39a700 1 --2- 172.21.15.99:0/2884471281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048483aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01fd39a700 1 -- 172.21.15.99:0/2884471281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 msgr2=0x55804848a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01fd39a700 1 --2- 172.21.15.99:0/2884471281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 0x55804848a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01fd39a700 1 -- 172.21.15.99:0/2884471281 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580481e7c00 con 0x558048486010 2023-08-25T09:12:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01fd39a700 1 --2- 172.21.15.99:0/2884471281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848f5d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01e8094e40 tx=0x7f01e8098480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e051ad2ca3627a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01ecbff700 1 -- 172.21.15.99:0/2884471281 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e809c770 con 0x558048486010 2023-08-25T09:12:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01ecbff700 1 -- 172.21.15.99:0/2884471281 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01e8083040 con 0x558048486010 2023-08-25T09:12:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.798+0000 7f01ecbff700 1 -- 172.21.15.99:0/2884471281 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8097e30 con 0x558048486010 2023-08-25T09:12:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 -- 172.21.15.99:0/2884471281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 msgr2=0x55804848f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 --2- 172.21.15.99:0/2884471281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848f5d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01e8094e40 tx=0x7f01e8098480 comp rx=0 tx=0).stop 2023-08-25T09:12:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 -- 172.21.15.99:0/2884471281 shutdown_connections 2023-08-25T09:12:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 --2- 172.21.15.99:0/2884471281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048486930 0x558048483aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 --2- 172.21.15.99:0/2884471281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048480990 0x55804848a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 --2- 172.21.15.99:0/2884471281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486010 0x55804848f5d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 -- 172.21.15.99:0/2884471281 >> 172.21.15.99:0/2884471281 conn(0x5580483f5080 msgr2=0x55804848fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 -- 172.21.15.99:0/2884471281 shutdown_connections 2023-08-25T09:12:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.799+0000 7f02136fab80 1 -- 172.21.15.99:0/2884471281 wait complete. 2023-08-25T09:12:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.800+0000 7f02136fab80 1 -- start start 2023-08-25T09:12:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.800+0000 7f02136fab80 1 Processor -- start 2023-08-25T09:12:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f02136fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x55804849b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f02136fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486010 0x55804849b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f02136fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486930 0x5580484f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f02136fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804849bf20 con 0x558048486010 2023-08-25T09:12:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f02136fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804849c090 con 0x558048480990 2023-08-25T09:12:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f02136fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580484f5f20 con 0x558048486930 2023-08-25T09:12:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f02136fab80 10 client.0 _ll_register_callbacks cb 0x7f01f000bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:12:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f01fdb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x55804849b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f01fdb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x55804849b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:60592/0 (socket says 172.21.15.99:60592) 2023-08-25T09:12:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.801+0000 7f01fdb9b700 1 -- 172.21.15.99:0/1678478206 learned_addr learned my addr 172.21.15.99:0/1678478206 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01fe39c700 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486930 0x5580484f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01fdb9b700 1 -- 172.21.15.99:0/1678478206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486930 msgr2=0x5580484f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486930 0x5580484f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01fdb9b700 1 -- 172.21.15.99:0/1678478206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486010 msgr2=0x55804849b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486010 0x55804849b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01fdb9b700 1 -- 172.21.15.99:0/1678478206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f81a3880 con 0x558048480990 2023-08-25T09:12:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01fe39c700 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486930 0x5580484f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:12:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01fdb9b700 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x55804849b200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005950 tx=0x7f01f4003b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2bdad161a6a0513 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.802+0000 7f01e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:12:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.803+0000 7f01e17fa700 1 -- 172.21.15.99:0/1678478206 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f402f040 con 0x558048480990 2023-08-25T09:12:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.803+0000 7f01e17fa700 1 -- 172.21.15.99:0/1678478206 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f01f4003d20 con 0x558048480990 2023-08-25T09:12:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.803+0000 7f02136fab80 1 -- 172.21.15.99:0/1678478206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558048473600 con 0x558048480990 2023-08-25T09:12:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.803+0000 7f01e17fa700 1 -- 172.21.15.99:0/1678478206 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01f4003ee0 con 0x558048480990 2023-08-25T09:12:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.803+0000 7f02136fab80 10 client.26324 fetch_fsmap learned FSMap version 42 2023-08-25T09:12:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.803+0000 7f02136fab80 1 -- 172.21.15.99:0/1678478206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5580482251f0 con 0x558048480990 2023-08-25T09:12:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.803+0000 7f01e17fa700 1 -- 172.21.15.99:0/1678478206 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558048473600 con 0x558048480990 2023-08-25T09:12:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.803+0000 7f01e17fa700 1 -- 172.21.15.99:0/1678478206 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f01f4026020 con 0x558048480990 2023-08-25T09:12:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f02136fab80 10 client.26324 fetch_fsmap finished waiting for FSMap version 42 2023-08-25T09:12:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f02136fab80 10 client.26324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:12:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f02136fab80 20 client.26324 populate_metadata read hostname 'smithi099' 2023-08-25T09:12:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f02136fab80 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5580484f7730 0x5580484f9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f02136fab80 4 client.26324 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:12:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f02136fab80 1 -- 172.21.15.99:0/1678478206 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5580483cb910 con 0x5580484f7730 2023-08-25T09:12:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f01fd39a700 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5580484f7730 0x5580484f9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f01fd39a700 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5580484f7730 0x5580484f9b50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7717f29fae15c4a8 server_cookie=7ad192ebbec50272 in_seq=0 out_seq=0 2023-08-25T09:12:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:07.804+0000 7f01e17fa700 10 client.26324 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:07.830 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:07 smithi099 ceph-mon[97641]: pgmap v404: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 531 op/s 2023-08-25T09:12:07.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:07 smithi157 ceph-mon[130879]: pgmap v404: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 531 op/s 2023-08-25T09:12:08.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 5469 34.48 MB/sec warmup 13 sec latency 268.600 ms 2023-08-25T09:12:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f01e17fa700 1 -- 172.21.15.99:0/1678478206 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5580483cb910 con 0x5580484f7730 2023-08-25T09:12:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f01e17fa700 10 client.26324 handle_command_reply: tid=0 2023-08-25T09:12:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 client.26324 shutdown 2023-08-25T09:12:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 -- 172.21.15.99:0/1678478206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 msgr2=0x55804849b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x55804849b200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005950 tx=0x7f01f4003b00 comp rx=0 tx=0).stop 2023-08-25T09:12:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 -- 172.21.15.99:0/1678478206 shutdown_connections 2023-08-25T09:12:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558048486930 0x5580484f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5580484f7730 0x5580484f9b50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558048480990 0x55804849b200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 --2- 172.21.15.99:0/1678478206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558048486010 0x55804849b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f01e17fa700 0 client.26324 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.381+0000 7f02136fab80 1 -- 172.21.15.99:0/1678478206 >> 172.21.15.99:0/1678478206 conn(0x5580483f5080 msgr2=0x558048484c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.382+0000 7f02136fab80 1 -- 172.21.15.99:0/1678478206 shutdown_connections 2023-08-25T09:12:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.382+0000 7f02136fab80 1 -- 172.21.15.99:0/1678478206 wait complete. 2023-08-25T09:12:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.382+0000 7f02136fab80 20 client.26324 trim_cache size 0 max 16384 2023-08-25T09:12:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:12:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:12:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:12:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:12:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.382+0000 7f01feb9d700 1 -- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01e40a00c0 msgr2=0x7f01e40a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.382+0000 7f01feb9d700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01e40a00c0 0x7f01e40a2580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01f4011630 tx=0x7f01f40119f0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.382+0000 7f01feb9d700 1 -- 172.21.15.99:0/2334845115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01f8102210 msgr2=0x7f01f819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.382+0000 7f01feb9d700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01f8102210 0x7f01f819cdd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01e800d950 tx=0x7f01e800dc60 comp rx=0 tx=0).stop 2023-08-25T09:12:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.383+0000 7f01feb9d700 1 -- 172.21.15.99:0/2334845115 shutdown_connections 2023-08-25T09:12:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.383+0000 7f01feb9d700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f01e40a00c0 0x7f01e40a2580 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.383+0000 7f01feb9d700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01f8102bd0 0x7f01f8196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.383+0000 7f01feb9d700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f8100ad0 0x7f01f819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.383+0000 7f01feb9d700 1 --2- 172.21.15.99:0/2334845115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01f8102210 0x7f01f819cdd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.383+0000 7f01feb9d700 1 -- 172.21.15.99:0/2334845115 >> 172.21.15.99:0/2334845115 conn(0x7f01f80fc1e0 msgr2=0x7f01f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.383+0000 7f01feb9d700 1 -- 172.21.15.99:0/2334845115 shutdown_connections 2023-08-25T09:12:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:08.383+0000 7f01feb9d700 1 -- 172.21.15.99:0/2334845115 wait complete. 2023-08-25T09:12:09.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 5936 34.03 MB/sec warmup 14 sec latency 275.481 ms 2023-08-25T09:12:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:09 smithi181 ceph-mon[131296]: pgmap v405: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 531 op/s 2023-08-25T09:12:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:09 smithi181 ceph-mon[131296]: osdmap e102: 12 total, 12 up, 12 in 2023-08-25T09:12:09.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:09 smithi099 ceph-mon[97641]: pgmap v405: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 531 op/s 2023-08-25T09:12:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:09 smithi099 ceph-mon[97641]: osdmap e102: 12 total, 12 up, 12 in 2023-08-25T09:12:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:09 smithi157 ceph-mon[130879]: pgmap v405: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 531 op/s 2023-08-25T09:12:09.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:09 smithi157 ceph-mon[130879]: osdmap e102: 12 total, 12 up, 12 in 2023-08-25T09:12:10.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 6586 35.43 MB/sec warmup 15 sec latency 45.448 ms 2023-08-25T09:12:11.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 6978 35.50 MB/sec warmup 16 sec latency 60.822 ms 2023-08-25T09:12:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:11 smithi181 ceph-mon[131296]: pgmap v407: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 2.9 MiB/s wr, 316 op/s 2023-08-25T09:12:11.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:11 smithi099 ceph-mon[97641]: pgmap v407: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 2.9 MiB/s wr, 316 op/s 2023-08-25T09:12:11.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:11 smithi157 ceph-mon[130879]: pgmap v407: 97 pgs: 97 active+clean; 203 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 2.9 MiB/s wr, 316 op/s 2023-08-25T09:12:12.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7132 33.65 MB/sec warmup 17 sec latency 73.104 ms 2023-08-25T09:12:13.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7160 31.82 MB/sec warmup 18 sec latency 728.977 ms 2023-08-25T09:12:13.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:13 smithi099 ceph-mon[97641]: pgmap v408: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 827 op/s 2023-08-25T09:12:13.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:13 smithi157 ceph-mon[130879]: pgmap v408: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 827 op/s 2023-08-25T09:12:13.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:13 smithi181 ceph-mon[131296]: pgmap v408: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 827 op/s 2023-08-25T09:12:14.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7246 30.16 MB/sec warmup 19 sec latency 1067.895 ms 2023-08-25T09:12:15.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7454 28.79 MB/sec warmup 20 sec latency 207.513 ms 2023-08-25T09:12:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:15 smithi099 ceph-mon[97641]: pgmap v409: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 738 op/s 2023-08-25T09:12:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:15 smithi157 ceph-mon[130879]: pgmap v409: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 738 op/s 2023-08-25T09:12:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:15 smithi181 ceph-mon[131296]: pgmap v409: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 738 op/s 2023-08-25T09:12:16.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7766 27.87 MB/sec warmup 21 sec latency 74.582 ms 2023-08-25T09:12:17.243 INFO:tasks.workunit.client.0.smithi099.stdout: 10 8284 28.04 MB/sec warmup 22 sec latency 66.066 ms 2023-08-25T09:12:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:17 smithi181 ceph-mon[131296]: pgmap v410: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 744 op/s 2023-08-25T09:12:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:17 smithi099 ceph-mon[97641]: pgmap v410: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 744 op/s 2023-08-25T09:12:17.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:17 smithi157 ceph-mon[130879]: pgmap v410: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 744 op/s 2023-08-25T09:12:18.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 8808 27.56 MB/sec warmup 23 sec latency 497.842 ms 2023-08-25T09:12:19.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 9375 27.89 MB/sec warmup 24 sec latency 1497.920 ms 2023-08-25T09:12:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:19 smithi181 ceph-mon[131296]: pgmap v411: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 744 op/s 2023-08-25T09:12:19.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:19 smithi099 ceph-mon[97641]: pgmap v411: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 744 op/s 2023-08-25T09:12:19.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:19 smithi157 ceph-mon[130879]: pgmap v411: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 744 op/s 2023-08-25T09:12:20.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 10009 28.67 MB/sec warmup 25 sec latency 1732.834 ms 2023-08-25T09:12:21.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 10370 28.68 MB/sec warmup 26 sec latency 107.533 ms 2023-08-25T09:12:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 -- 172.21.15.99:0/1688490445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c101260 msgr2=0x7f5d8c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 --2- 172.21.15.99:0/1688490445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c101260 0x7f5d8c10d770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b0a0 tx=0x7f5d8400b3b0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 -- 172.21.15.99:0/1688490445 shutdown_connections 2023-08-25T09:12:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 --2- 172.21.15.99:0/1688490445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c101260 0x7f5d8c10d770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 --2- 172.21.15.99:0/1688490445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1068b0 0x7f5d8c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 --2- 172.21.15.99:0/1688490445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c1008a0 0x7f5d8c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 -- 172.21.15.99:0/1688490445 >> 172.21.15.99:0/1688490445 conn(0x7f5d8c0fc370 msgr2=0x7f5d8c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 -- 172.21.15.99:0/1688490445 shutdown_connections 2023-08-25T09:12:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.544+0000 7f5d92fff700 1 -- 172.21.15.99:0/1688490445 wait complete. 2023-08-25T09:12:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.545+0000 7f5d92fff700 1 Processor -- start 2023-08-25T09:12:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d92fff700 1 -- start start 2023-08-25T09:12:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d92fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c1008a0 0x7f5d8c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d92fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c101260 0x7f5d8c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d92fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1068b0 0x7f5d8c194500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d92fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c194b70 con 0x7f5d8c1068b0 2023-08-25T09:12:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d92fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c194ce0 con 0x7f5d8c101260 2023-08-25T09:12:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d92fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c194ea0 con 0x7f5d8c1008a0 2023-08-25T09:12:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c101260 0x7f5d8c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c101260 0x7f5d8c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40478/0 (socket says 172.21.15.99:40478) 2023-08-25T09:12:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d8bfff700 1 -- 172.21.15.99:0/3018935713 learned_addr learned my addr 172.21.15.99:0/3018935713 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c1008a0 0x7f5d8c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.546+0000 7f5d9159c700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1068b0 0x7f5d8c194500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d8bfff700 1 -- 172.21.15.99:0/3018935713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c1008a0 msgr2=0x7f5d8c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c1008a0 0x7f5d8c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d8bfff700 1 -- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1068b0 msgr2=0x7f5d8c194500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1068b0 0x7f5d8c194500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d8bfff700 1 -- 172.21.15.99:0/3018935713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84009d00 con 0x7f5d8c101260 2023-08-25T09:12:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d9159c700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1068b0 0x7f5d8c194500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:12:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c1008a0 0x7f5d8c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:12:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c101260 0x7f5d8c19abe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5d78012fc0 tx=0x7f5d780100c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d89ffb700 1 -- 172.21.15.99:0/3018935713 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7800dba0 con 0x7f5d8c101260 2023-08-25T09:12:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d92fff700 1 -- 172.21.15.99:0/3018935713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c195120 con 0x7f5d8c101260 2023-08-25T09:12:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d89ffb700 1 -- 172.21.15.99:0/3018935713 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d78013040 con 0x7f5d8c101260 2023-08-25T09:12:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d89ffb700 1 -- 172.21.15.99:0/3018935713 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7800bd90 con 0x7f5d8c101260 2023-08-25T09:12:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.547+0000 7f5d92fff700 1 -- 172.21.15.99:0/3018935713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c1a1690 con 0x7f5d8c101260 2023-08-25T09:12:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.551+0000 7f5d89ffb700 1 -- 172.21.15.99:0/3018935713 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f5d780107f0 con 0x7f5d8c101260 2023-08-25T09:12:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5da6931b80 1 Processor -- start 2023-08-25T09:12:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5da6931b80 1 -- start start 2023-08-25T09:12:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5d89ffb700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5d7c0a00c0 0x7f5d7c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0f860 0x55fdecc09450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc09990 0x55fdecc09dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5da6931b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecb71750 con 0x55fdecc0ef40 2023-08-25T09:12:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5da6931b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecb62720 con 0x55fdecc0f860 2023-08-25T09:12:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5da6931b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecb60040 con 0x55fdecc09990 2023-08-25T09:12:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.553+0000 7f5d89ffb700 1 -- 172.21.15.99:0/3018935713 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5d780cf790 con 0x7f5d8c101260 2023-08-25T09:12:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc09990 0x55fdecc09dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc09990 0x55fdecc09dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:44624/0 (socket says 172.21.15.99:44624) 2023-08-25T09:12:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 -- 172.21.15.99:0/1386791691 learned_addr learned my addr 172.21.15.99:0/1386791691 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0f860 0x55fdecc09450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 -- 172.21.15.99:0/1386791691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0f860 msgr2=0x55fdecc09450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0f860 0x55fdecc09450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 -- 172.21.15.99:0/1386791691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 msgr2=0x55fdecc0f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 -- 172.21.15.99:0/1386791691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdec9db5d0 con 0x55fdecc09990 2023-08-25T09:12:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5d7c0a00c0 0x7f5d7c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.554+0000 7f5d9159c700 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc09990 0x55fdecc09dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b0a0 tx=0x7f5d840095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=253e594cabb89f1d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.555+0000 7f5d76ffd700 1 -- 172.21.15.99:0/1386791691 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400e040 con 0x55fdecc09990 2023-08-25T09:12:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.555+0000 7f5d76ffd700 1 -- 172.21.15.99:0/1386791691 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d840079a0 con 0x55fdecc09990 2023-08-25T09:12:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.555+0000 7f5d76ffd700 1 -- 172.21.15.99:0/1386791691 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840055e0 con 0x55fdecc09990 2023-08-25T09:12:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 -- 172.21.15.99:0/1386791691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc09990 msgr2=0x55fdecc09dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc09990 0x55fdecc09dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b0a0 tx=0x7f5d840095b0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5d9159c700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5d7c0a00c0 0x7f5d7c0a2580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d84029000 tx=0x7f5d8402c000 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 -- 172.21.15.99:0/1386791691 shutdown_connections 2023-08-25T09:12:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc09990 0x55fdecc09dd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1386791691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0f860 0x55fdecc09450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 -- 172.21.15.99:0/1386791691 >> 172.21.15.99:0/1386791691 conn(0x55fdecb7e080 msgr2=0x55fdecc03d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 -- 172.21.15.99:0/1386791691 shutdown_connections 2023-08-25T09:12:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 -- 172.21.15.99:0/1386791691 wait complete. 2023-08-25T09:12:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 -- start start 2023-08-25T09:12:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.556+0000 7f5da6931b80 1 Processor -- start 2023-08-25T09:12:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc12f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc13460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5da6931b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecb5b930 con 0x55fdecc09990 2023-08-25T09:12:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5da6931b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecaf9b50 con 0x55fdecc0ef40 2023-08-25T09:12:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5da6931b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecaf9850 con 0x55fdecc0f860 2023-08-25T09:12:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5da6931b80 10 client.0 _ll_register_callbacks cb 0x55fdeca43a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:12:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc12f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5d90d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc12f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40486/0 (socket says 172.21.15.99:40486) 2023-08-25T09:12:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5d8bfff700 1 -- 172.21.15.99:0/1969189795 learned_addr learned my addr 172.21.15.99:0/1969189795 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.557+0000 7f5d9159c700 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc13460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d8bfff700 1 -- 172.21.15.99:0/1969189795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 msgr2=0x55fdecc13460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc13460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d8bfff700 1 -- 172.21.15.99:0/1969189795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 msgr2=0x55fdecc129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc129e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d8bfff700 1 -- 172.21.15.99:0/1969189795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c10e070 con 0x55fdecc0ef40 2023-08-25T09:12:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:12:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc12f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5d7809cb10 tx=0x7f5d7809c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef87f752d0e0bed4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:12:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1969189795 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7809d510 con 0x55fdecc0ef40 2023-08-25T09:12:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1969189795 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5d780bc280 con 0x55fdecc0ef40 2023-08-25T09:12:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5da6931b80 1 -- 172.21.15.99:0/1969189795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdecbd9160 con 0x55fdecc0ef40 2023-08-25T09:12:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1969189795 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7809ce10 con 0x55fdecc0ef40 2023-08-25T09:12:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5da6931b80 10 client.26336 fetch_fsmap learned FSMap version 42 2023-08-25T09:12:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5da6931b80 1 -- 172.21.15.99:0/1969189795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55fdec987d80 con 0x55fdecc0ef40 2023-08-25T09:12:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.558+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1969189795 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fdecbd9160 con 0x55fdecc0ef40 2023-08-25T09:12:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1969189795 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5d7809c300 con 0x55fdecc0ef40 2023-08-25T09:12:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5da6931b80 10 client.26336 fetch_fsmap finished waiting for FSMap version 42 2023-08-25T09:12:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5da6931b80 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:12:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5da6931b80 20 client.26336 populate_metadata read hostname 'smithi099' 2023-08-25T09:12:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fdecc23190 0x55fdecc255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5da6931b80 4 client.26336 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:12:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5da6931b80 1 -- 172.21.15.99:0/1969189795 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdecb01b40 con 0x55fdecc23190 2023-08-25T09:12:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fdecc23190 0x55fdecc255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fdecc23190 0x55fdecc255b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fafc1ab58e982b3 server_cookie=bfa8b5d23bd21822 in_seq=0 out_seq=0 2023-08-25T09:12:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.559+0000 7f5d6bfff700 10 client.26336 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.565+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1969189795 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fdecb01b40 con 0x55fdecc23190 2023-08-25T09:12:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.565+0000 7f5d6bfff700 10 client.26336 handle_command_reply: tid=0 2023-08-25T09:12:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 client.26336 shutdown 2023-08-25T09:12:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 -- 172.21.15.99:0/1969189795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 msgr2=0x55fdecc12f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc12f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5d7809cb10 tx=0x7f5d7809c950 comp rx=0 tx=0).stop 2023-08-25T09:12:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 -- 172.21.15.99:0/1969189795 shutdown_connections 2023-08-25T09:12:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc13460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fdecc23190 0x55fdecc255b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1969189795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc12f20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5d6bfff700 0 client.26336 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 -- 172.21.15.99:0/1969189795 >> 172.21.15.99:0/1969189795 conn(0x55fdecb7e080 msgr2=0x55fdecbfe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 -- 172.21.15.99:0/1969189795 shutdown_connections 2023-08-25T09:12:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 1 -- 172.21.15.99:0/1969189795 wait complete. 2023-08-25T09:12:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.566+0000 7f5da6931b80 20 client.26336 trim_cache size 0 max 16384 2023-08-25T09:12:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5da6931b80 1 Processor -- start 2023-08-25T09:12:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5da6931b80 1 -- start start 2023-08-25T09:12:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc09450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc1a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5da6931b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdec9dd8c0 con 0x55fdecc09990 2023-08-25T09:12:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5da6931b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecc1a7b0 con 0x55fdecc0ef40 2023-08-25T09:12:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5da6931b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecbfc3d0 con 0x55fdecc0f860 2023-08-25T09:12:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5d90d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc09450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5d90d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc09450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38974/0 (socket says 172.21.15.99:38974) 2023-08-25T09:12:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5d90d9b700 1 -- 172.21.15.99:0/434839427 learned_addr learned my addr 172.21.15.99:0/434839427 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc1a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.582+0000 7f5d9159c700 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d90d9b700 1 -- 172.21.15.99:0/434839427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 msgr2=0x55fdecc0da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d90d9b700 1 -- 172.21.15.99:0/434839427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 msgr2=0x55fdecc1a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc1a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d90d9b700 1 -- 172.21.15.99:0/434839427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdec970c00 con 0x55fdecc09990 2023-08-25T09:12:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc1a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:12:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d9159c700 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:12:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc09450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55fdecc25eb0 tx=0x7f5d80009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbbc52cda2ecd2c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d777fe700 1 -- 172.21.15.99:0/434839427 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80016030 con 0x55fdecc09990 2023-08-25T09:12:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d777fe700 1 -- 172.21.15.99:0/434839427 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d80017040 con 0x55fdecc09990 2023-08-25T09:12:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.583+0000 7f5d777fe700 1 -- 172.21.15.99:0/434839427 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80004830 con 0x55fdecc09990 2023-08-25T09:12:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 -- 172.21.15.99:0/434839427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 msgr2=0x55fdecc09450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc09450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55fdecc25eb0 tx=0x7f5d80009260 comp rx=0 tx=0).stop 2023-08-25T09:12:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 -- 172.21.15.99:0/434839427 shutdown_connections 2023-08-25T09:12:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc09990 0x55fdecc09450 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 --2- 172.21.15.99:0/434839427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc0ef40 0x55fdecc1a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 -- 172.21.15.99:0/434839427 >> 172.21.15.99:0/434839427 conn(0x55fdecb7e080 msgr2=0x55fdecc0caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 -- 172.21.15.99:0/434839427 shutdown_connections 2023-08-25T09:12:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 -- 172.21.15.99:0/434839427 wait complete. 2023-08-25T09:12:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.584+0000 7f5da6931b80 1 -- start start 2023-08-25T09:12:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.585+0000 7f5da6931b80 1 Processor -- start 2023-08-25T09:12:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.585+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc09990 0x55fdecc0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.585+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.585+0000 7f5da6931b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.585+0000 7f5da6931b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecc0adc0 con 0x55fdecc0ef40 2023-08-25T09:12:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.585+0000 7f5da6931b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecc0af30 con 0x55fdecc09990 2023-08-25T09:12:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.585+0000 7f5da6931b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdecc0b0f0 con 0x55fdecc0f860 2023-08-25T09:12:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.585+0000 7f5da6931b80 10 client.0 _ll_register_callbacks cb 0x7f5d78099150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:12:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d90d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc09990 0x55fdecc0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38982/0 (socket says 172.21.15.99:38982) 2023-08-25T09:12:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d8bfff700 1 -- 172.21.15.99:0/1782901031 learned_addr learned my addr 172.21.15.99:0/1782901031 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d90d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc09990 0x55fdecc0b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40506/0 (socket says 172.21.15.99:40506) 2023-08-25T09:12:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d9159c700 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d9159c700 1 -- 172.21.15.99:0/1782901031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc09990 msgr2=0x55fdecc0b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d9159c700 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc09990 0x55fdecc0b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d9159c700 1 -- 172.21.15.99:0/1782901031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 msgr2=0x55fdecc0be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d9159c700 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d9159c700 1 -- 172.21.15.99:0/1782901031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdec987d80 con 0x55fdecc0f860 2023-08-25T09:12:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d8bfff700 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:12:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d9159c700 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0a880 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b070 tx=0x7f5d840094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=327c7b4fcb164c36 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.586+0000 7f5d6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:12:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.587+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1782901031 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400e040 con 0x55fdecc0f860 2023-08-25T09:12:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.587+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1782901031 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f5d84032640 con 0x55fdecc0f860 2023-08-25T09:12:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.587+0000 7f5da6931b80 1 -- 172.21.15.99:0/1782901031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdecbfc600 con 0x55fdecc0f860 2023-08-25T09:12:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.587+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1782901031 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d840079a0 con 0x55fdecc0f860 2023-08-25T09:12:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.587+0000 7f5da6931b80 10 client.26326 fetch_fsmap learned FSMap version 42 2023-08-25T09:12:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.587+0000 7f5da6931b80 1 -- 172.21.15.99:0/1782901031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f5d80011a10 con 0x55fdecc0f860 2023-08-25T09:12:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1782901031 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fdecbfc600 con 0x55fdecc0f860 2023-08-25T09:12:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1782901031 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5d8401d870 con 0x55fdecc0f860 2023-08-25T09:12:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5da6931b80 10 client.26326 fetch_fsmap finished waiting for FSMap version 42 2023-08-25T09:12:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5da6931b80 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:12:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5da6931b80 20 client.26326 populate_metadata read hostname 'smithi099' 2023-08-25T09:12:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fdecc77aa0 0x55fdecc79ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5da6931b80 4 client.26326 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:12:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5da6931b80 1 -- 172.21.15.99:0/1782901031 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fdecb54910 con 0x55fdecc77aa0 2023-08-25T09:12:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fdecc77aa0 0x55fdecc79ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5d90d9b700 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fdecc77aa0 0x55fdecc79ec0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad18f0ac4c60961c server_cookie=cd554f3182329beb in_seq=0 out_seq=0 2023-08-25T09:12:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.588+0000 7f5d6bfff700 10 client.26326 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:21.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:21 smithi181 ceph-mon[131296]: pgmap v412: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 23 MiB/s wr, 670 op/s 2023-08-25T09:12:21.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:21 smithi099 ceph-mon[97641]: pgmap v412: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 23 MiB/s wr, 670 op/s 2023-08-25T09:12:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:21 smithi157 ceph-mon[130879]: pgmap v412: 97 pgs: 97 active+clean; 406 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 23 MiB/s wr, 670 op/s 2023-08-25T09:12:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.989+0000 7f5d6bfff700 1 -- 172.21.15.99:0/1782901031 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fdecb54910 con 0x55fdecc77aa0 2023-08-25T09:12:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.989+0000 7f5d6bfff700 10 client.26326 handle_command_reply: tid=0 2023-08-25T09:12:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.989+0000 7f5da6931b80 1 client.26326 shutdown 2023-08-25T09:12:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 -- 172.21.15.99:0/1782901031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 msgr2=0x55fdecc0a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0a880 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b070 tx=0x7f5d840094d0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 -- 172.21.15.99:0/1782901031 shutdown_connections 2023-08-25T09:12:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fdecc0f860 0x55fdecc0a880 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55fdecc77aa0 0x55fdecc79ec0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdecc09990 0x55fdecc0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 --2- 172.21.15.99:0/1782901031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fdecc0ef40 0x55fdecc0be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 -- 172.21.15.99:0/1782901031 >> 172.21.15.99:0/1782901031 conn(0x55fdecb7e080 msgr2=0x55fdecc0da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5d6bfff700 0 client.26326 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 -- 172.21.15.99:0/1782901031 shutdown_connections 2023-08-25T09:12:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.990+0000 7f5da6931b80 1 -- 172.21.15.99:0/1782901031 wait complete. 2023-08-25T09:12:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.994+0000 7f5da6931b80 20 client.26326 trim_cache size 0 max 16384 2023-08-25T09:12:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:12:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:12:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:12:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:12:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 -- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5d7c0a00c0 msgr2=0x7f5d7c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5d7c0a00c0 0x7f5d7c0a2580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d84029000 tx=0x7f5d8402c000 comp rx=0 tx=0).stop 2023-08-25T09:12:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 -- 172.21.15.99:0/3018935713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c101260 msgr2=0x7f5d8c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c101260 0x7f5d8c19abe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5d78012fc0 tx=0x7f5d780100c0 comp rx=0 tx=0).stop 2023-08-25T09:12:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 -- 172.21.15.99:0/3018935713 shutdown_connections 2023-08-25T09:12:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5d7c0a00c0 0x7f5d7c0a2580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1068b0 0x7f5d8c194500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8c1008a0 0x7f5d8c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 --2- 172.21.15.99:0/3018935713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c101260 0x7f5d8c19abe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 -- 172.21.15.99:0/3018935713 >> 172.21.15.99:0/3018935713 conn(0x7f5d8c0fc370 msgr2=0x7f5d8c0fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 -- 172.21.15.99:0/3018935713 shutdown_connections 2023-08-25T09:12:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:21.995+0000 7f5d92fff700 1 -- 172.21.15.99:0/3018935713 wait complete. 2023-08-25T09:12:22.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 10521 27.95 MB/sec warmup 27 sec latency 240.131 ms 2023-08-25T09:12:23.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 10687 27.06 MB/sec warmup 28 sec latency 201.682 ms 2023-08-25T09:12:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:23 smithi181 ceph-mon[131296]: pgmap v413: 97 pgs: 97 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 22 MiB/s wr, 745 op/s 2023-08-25T09:12:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:23 smithi099 ceph-mon[97641]: pgmap v413: 97 pgs: 97 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 22 MiB/s wr, 745 op/s 2023-08-25T09:12:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:23 smithi157 ceph-mon[130879]: pgmap v413: 97 pgs: 97 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 22 MiB/s wr, 745 op/s 2023-08-25T09:12:24.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 10953 26.41 MB/sec warmup 29 sec latency 804.166 ms 2023-08-25T09:12:24.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:24 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:24.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:24 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:24.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:24 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:25.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 11258 25.90 MB/sec warmup 30 sec latency 1804.221 ms 2023-08-25T09:12:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:25 smithi181 ceph-mon[131296]: pgmap v414: 97 pgs: 97 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 341 op/s 2023-08-25T09:12:25.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:25 smithi099 ceph-mon[97641]: pgmap v414: 97 pgs: 97 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 341 op/s 2023-08-25T09:12:25.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:25 smithi157 ceph-mon[130879]: pgmap v414: 97 pgs: 97 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 341 op/s 2023-08-25T09:12:26.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 11791 25.98 MB/sec warmup 31 sec latency 2804.262 ms 2023-08-25T09:12:27.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 12422 26.05 MB/sec warmup 32 sec latency 3804.310 ms 2023-08-25T09:12:27.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:27 smithi181 ceph-mon[131296]: pgmap v415: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 651 op/s 2023-08-25T09:12:27.820 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:27 smithi099 ceph-mon[97641]: pgmap v415: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 651 op/s 2023-08-25T09:12:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:27 smithi157 ceph-mon[130879]: pgmap v415: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 651 op/s 2023-08-25T09:12:28.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 12844 26.32 MB/sec warmup 33 sec latency 4804.447 ms 2023-08-25T09:12:29.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 13233 26.48 MB/sec warmup 34 sec latency 5804.532 ms 2023-08-25T09:12:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:29 smithi181 ceph-mon[131296]: pgmap v416: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 434 op/s 2023-08-25T09:12:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:29 smithi099 ceph-mon[97641]: pgmap v416: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 434 op/s 2023-08-25T09:12:29.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:29 smithi157 ceph-mon[130879]: pgmap v416: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 434 op/s 2023-08-25T09:12:30.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 13399 25.81 MB/sec warmup 35 sec latency 6804.598 ms 2023-08-25T09:12:31.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 13637 25.24 MB/sec warmup 36 sec latency 7804.638 ms 2023-08-25T09:12:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:31 smithi181 ceph-mon[131296]: pgmap v417: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 434 op/s 2023-08-25T09:12:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:31 smithi099 ceph-mon[97641]: pgmap v417: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 434 op/s 2023-08-25T09:12:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:31 smithi157 ceph-mon[130879]: pgmap v417: 97 pgs: 97 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 434 op/s 2023-08-25T09:12:32.244 INFO:tasks.workunit.client.0.smithi099.stdout: 10 14106 25.22 MB/sec warmup 37 sec latency 8804.727 ms 2023-08-25T09:12:33.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 14737 25.25 MB/sec warmup 38 sec latency 9578.214 ms 2023-08-25T09:12:33.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:33 smithi181 ceph-mon[131296]: pgmap v418: 97 pgs: 97 active+clean; 408 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 550 op/s 2023-08-25T09:12:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:33 smithi099 ceph-mon[97641]: pgmap v418: 97 pgs: 97 active+clean; 408 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 550 op/s 2023-08-25T09:12:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:33 smithi157 ceph-mon[130879]: pgmap v418: 97 pgs: 97 active+clean; 408 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 550 op/s 2023-08-25T09:12:34.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 15650 26.11 MB/sec warmup 39 sec latency 59.590 ms 2023-08-25T09:12:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.206+0000 7f1e6bfff700 1 -- 172.21.15.99:0/893714661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e64072cc0 msgr2=0x7f1e640730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.206+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/893714661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e64072cc0 0x7f1e640730a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009990 tx=0x7f1e60009ca0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.207+0000 7f1e6bfff700 1 -- 172.21.15.99:0/893714661 shutdown_connections 2023-08-25T09:12:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.207+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/893714661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e641118c0 0x7f1e64113cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.207+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/893714661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e64072cc0 0x7f1e640730a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.207+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/893714661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e640735e0 0x7f1e64111380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.207+0000 7f1e6bfff700 1 -- 172.21.15.99:0/893714661 >> 172.21.15.99:0/893714661 conn(0x7f1e640fc380 msgr2=0x7f1e640fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.207+0000 7f1e6bfff700 1 -- 172.21.15.99:0/893714661 shutdown_connections 2023-08-25T09:12:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.207+0000 7f1e6bfff700 1 -- 172.21.15.99:0/893714661 wait complete. 2023-08-25T09:12:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.209+0000 7f1e6bfff700 1 Processor -- start 2023-08-25T09:12:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.209+0000 7f1e6bfff700 1 -- start start 2023-08-25T09:12:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e6bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e64072cc0 0x7f1e6419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e6bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e640735e0 0x7f1e6419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e6bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e641118c0 0x7f1e641a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e6bfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e641a2230 con 0x7f1e641118c0 2023-08-25T09:12:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e6bfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e641a23a0 con 0x7f1e64072cc0 2023-08-25T09:12:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e6bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e641a2560 con 0x7f1e640735e0 2023-08-25T09:12:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e69d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e64072cc0 0x7f1e6419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e69d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e64072cc0 0x7f1e6419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:44344/0 (socket says 172.21.15.99:44344) 2023-08-25T09:12:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e69d9b700 1 -- 172.21.15.99:0/3906866911 learned_addr learned my addr 172.21.15.99:0/3906866911 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.210+0000 7f1e6959a700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e640735e0 0x7f1e6419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.211+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e641118c0 0x7f1e641a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.211+0000 7f1e69d9b700 1 -- 172.21.15.99:0/3906866911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e640735e0 msgr2=0x7f1e6419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.211+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e640735e0 0x7f1e6419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.211+0000 7f1e69d9b700 1 -- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e641118c0 msgr2=0x7f1e641a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.211+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e641118c0 0x7f1e641a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.211+0000 7f1e69d9b700 1 -- 172.21.15.99:0/3906866911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e600095f0 con 0x7f1e64072cc0 2023-08-25T09:12:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.212+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e64072cc0 0x7f1e6419d040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006010 tx=0x7f1e60003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.212+0000 7f1e5affd700 1 -- 172.21.15.99:0/3906866911 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6001d070 con 0x7f1e64072cc0 2023-08-25T09:12:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.212+0000 7f1e5affd700 1 -- 172.21.15.99:0/3906866911 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e60003e70 con 0x7f1e64072cc0 2023-08-25T09:12:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.212+0000 7f1e5affd700 1 -- 172.21.15.99:0/3906866911 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000fdd0 con 0x7f1e64072cc0 2023-08-25T09:12:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.212+0000 7f1e6bfff700 1 -- 172.21.15.99:0/3906866911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e641a27e0 con 0x7f1e64072cc0 2023-08-25T09:12:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.212+0000 7f1e6bfff700 1 -- 172.21.15.99:0/3906866911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e64104d30 con 0x7f1e64072cc0 2023-08-25T09:12:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.216+0000 7f1e5affd700 1 -- 172.21.15.99:0/3906866911 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f1e6000f4f0 con 0x7f1e64072cc0 2023-08-25T09:12:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.218+0000 7f1e7f899b80 1 Processor -- start 2023-08-25T09:12:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.218+0000 7f1e7f899b80 1 -- start start 2023-08-25T09:12:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.218+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e40f40 0x55b047e41320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.218+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 0x55b047e3b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.218+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e3b990 0x55b047e3bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.218+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047da3750 con 0x55b047e41860 2023-08-25T09:12:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.218+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047d94720 con 0x55b047e40f40 2023-08-25T09:12:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.218+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047d92040 con 0x55b047e3b990 2023-08-25T09:12:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.219+0000 7f1e6a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e40f40 0x55b047e41320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.219+0000 7f1e6a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e40f40 0x55b047e41320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:44358/0 (socket says 172.21.15.99:44358) 2023-08-25T09:12:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.219+0000 7f1e6a59c700 1 -- 172.21.15.99:0/694346936 learned_addr learned my addr 172.21.15.99:0/694346936 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.219+0000 7f1e6959a700 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e3b990 0x55b047e3bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.219+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 0x55b047e3b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.219+0000 7f1e5affd700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f1e500a0210 0x7f1e500a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.219+0000 7f1e6a59c700 1 -- 172.21.15.99:0/694346936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e3b990 msgr2=0x55b047e3bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.219+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e3b990 0x55b047e3bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.220+0000 7f1e6a59c700 1 -- 172.21.15.99:0/694346936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 msgr2=0x55b047e3b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.220+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 0x55b047e3b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.220+0000 7f1e6a59c700 1 -- 172.21.15.99:0/694346936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b047c0d5d0 con 0x55b047e40f40 2023-08-25T09:12:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.220+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f1e500a0210 0x7f1e500a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.220+0000 7f1e5affd700 1 -- 172.21.15.99:0/3906866911 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f1e600cc8b0 con 0x7f1e64072cc0 2023-08-25T09:12:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.220+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e40f40 0x55b047e41320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00b480 tx=0x7f1e5c009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4a9cb46fc310fe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.221+0000 7f1e4ffff700 1 -- 172.21.15.99:0/694346936 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c010040 con 0x55b047e40f40 2023-08-25T09:12:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.221+0000 7f1e4ffff700 1 -- 172.21.15.99:0/694346936 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e5c01b070 con 0x55b047e40f40 2023-08-25T09:12:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.221+0000 7f1e4ffff700 1 -- 172.21.15.99:0/694346936 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0205f0 con 0x55b047e40f40 2023-08-25T09:12:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.221+0000 7f1e7f899b80 1 -- 172.21.15.99:0/694346936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e40f40 msgr2=0x55b047e41320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.221+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e40f40 0x55b047e41320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00b480 tx=0x7f1e5c009630 comp rx=0 tx=0).stop 2023-08-25T09:12:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.222+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f1e500a0210 0x7f1e500a26d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c01e000 tx=0x7f1e5c014f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.222+0000 7f1e7f899b80 1 -- 172.21.15.99:0/694346936 shutdown_connections 2023-08-25T09:12:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.222+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e3b990 0x55b047e3bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.222+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e40f40 0x55b047e41320 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.222+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/694346936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 0x55b047e3b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.222+0000 7f1e7f899b80 1 -- 172.21.15.99:0/694346936 >> 172.21.15.99:0/694346936 conn(0x55b047db0080 msgr2=0x55b047e35d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.223+0000 7f1e7f899b80 1 -- 172.21.15.99:0/694346936 shutdown_connections 2023-08-25T09:12:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.223+0000 7f1e7f899b80 1 -- 172.21.15.99:0/694346936 wait complete. 2023-08-25T09:12:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.224+0000 7f1e7f899b80 1 -- start start 2023-08-25T09:12:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.224+0000 7f1e7f899b80 1 Processor -- start 2023-08-25T09:12:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.225+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.225+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.225+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 0x55b047e44ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.225+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047d8d930 con 0x55b047e41860 2023-08-25T09:12:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.225+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047d2bb50 con 0x55b047e3b990 2023-08-25T09:12:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.225+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047d2b850 con 0x55b047e40f40 2023-08-25T09:12:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.225+0000 7f1e7f899b80 10 client.0 _ll_register_callbacks cb 0x55b047c75a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:12:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6959a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6959a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:42366/0 (socket says 172.21.15.99:42366) 2023-08-25T09:12:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6959a700 1 -- 172.21.15.99:0/1627324855 learned_addr learned my addr 172.21.15.99:0/1627324855 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 0x55b047e44ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6959a700 1 -- 172.21.15.99:0/1627324855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 msgr2=0x55b047e476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6959a700 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6959a700 1 -- 172.21.15.99:0/1627324855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 msgr2=0x55b047e44ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6959a700 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 0x55b047e44ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.226+0000 7f1e6959a700 1 -- 172.21.15.99:0/1627324855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c009c60 con 0x55b047e40f40 2023-08-25T09:12:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.227+0000 7f1e6959a700 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e448c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e54010840 tx=0x7f1e54010b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909795315d2bd4ab server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.227+0000 7f1e4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:12:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.227+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/1627324855 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54004220 con 0x55b047e40f40 2023-08-25T09:12:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.227+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1627324855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b047e0b160 con 0x55b047e40f40 2023-08-25T09:12:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.227+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/1627324855 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f1e54017030 con 0x55b047e40f40 2023-08-25T09:12:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.227+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/1627324855 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e54004380 con 0x55b047e40f40 2023-08-25T09:12:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.227+0000 7f1e7f899b80 10 client.26338 fetch_fsmap learned FSMap version 43 2023-08-25T09:12:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.227+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1627324855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b047bb9d80 con 0x55b047e40f40 2023-08-25T09:12:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.228+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/1627324855 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b047e0b160 con 0x55b047e40f40 2023-08-25T09:12:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.229+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/1627324855 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1e54011770 con 0x55b047e40f40 2023-08-25T09:12:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.229+0000 7f1e7f899b80 10 client.26338 fetch_fsmap finished waiting for FSMap version 43 2023-08-25T09:12:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.229+0000 7f1e7f899b80 10 client.26338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:12:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.229+0000 7f1e7f899b80 20 client.26338 populate_metadata read hostname 'smithi099' 2023-08-25T09:12:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.229+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b047e55210 0x55b047e57630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.229+0000 7f1e7f899b80 4 client.26338 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:12:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.229+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1627324855 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b047d33b40 con 0x55b047e55210 2023-08-25T09:12:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.229+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b047e55210 0x55b047e57630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.230+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b047e55210 0x55b047e57630 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9a3a723c7de9f3 server_cookie=89a4e3d62a42b697 in_seq=0 out_seq=0 2023-08-25T09:12:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.230+0000 7f1e4d7fa700 10 client.26338 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.232+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/1627324855 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b047d33b40 con 0x55b047e55210 2023-08-25T09:12:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.232+0000 7f1e4d7fa700 10 client.26338 handle_command_reply: tid=0 2023-08-25T09:12:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.232+0000 7f1e7f899b80 1 client.26338 shutdown 2023-08-25T09:12:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1627324855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 msgr2=0x55b047e448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e448c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e54010840 tx=0x7f1e54010b50 comp rx=0 tx=0).stop 2023-08-25T09:12:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1627324855 shutdown_connections 2023-08-25T09:12:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e41860 0x55b047e44ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b047e55210 0x55b047e57630 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1627324855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e448c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1627324855 >> 172.21.15.99:0/1627324855 conn(0x55b047db0080 msgr2=0x55b047e30820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e4d7fa700 0 client.26338 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1627324855 shutdown_connections 2023-08-25T09:12:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.233+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1627324855 wait complete. 2023-08-25T09:12:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.234+0000 7f1e7f899b80 20 client.26338 trim_cache size 0 max 16384 2023-08-25T09:12:35.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 16155 26.07 MB/sec warmup 40 sec latency 380.892 ms 2023-08-25T09:12:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.253+0000 7f1e7f899b80 1 Processor -- start 2023-08-25T09:12:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.253+0000 7f1e7f899b80 1 -- start start 2023-08-25T09:12:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e3b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e40f40 0x55b047e4c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e41860 0x55b047e35bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047c0f8c0 con 0x55b047e40f40 2023-08-25T09:12:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047e37f30 con 0x55b047e3b990 2023-08-25T09:12:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047e4c7b0 con 0x55b047e41860 2023-08-25T09:12:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e69d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e3b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e69d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e3b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:44368/0 (socket says 172.21.15.99:44368) 2023-08-25T09:12:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e69d9b700 1 -- 172.21.15.99:0/1765677180 learned_addr learned my addr 172.21.15.99:0/1765677180 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e41860 0x55b047e35bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e6959a700 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e40f40 0x55b047e4c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e69d9b700 1 -- 172.21.15.99:0/1765677180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e41860 msgr2=0x55b047e35bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e41860 0x55b047e35bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e69d9b700 1 -- 172.21.15.99:0/1765677180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e40f40 msgr2=0x55b047e4c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e40f40 0x55b047e4c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.254+0000 7f1e69d9b700 1 -- 172.21.15.99:0/1765677180 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b047ba2c00 con 0x55b047e3b990 2023-08-25T09:12:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.255+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e3b450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1e60003760 tx=0x7f1e6000f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78dee39bfee86352 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.255+0000 7f1e58ff9700 1 -- 172.21.15.99:0/1765677180 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600998e0 con 0x55b047e3b990 2023-08-25T09:12:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.255+0000 7f1e58ff9700 1 -- 172.21.15.99:0/1765677180 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e6009a680 con 0x55b047e3b990 2023-08-25T09:12:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.255+0000 7f1e58ff9700 1 -- 172.21.15.99:0/1765677180 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600a1d90 con 0x55b047e3b990 2023-08-25T09:12:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1765677180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 msgr2=0x55b047e3b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e3b450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1e60003760 tx=0x7f1e6000f890 comp rx=0 tx=0).stop 2023-08-25T09:12:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1765677180 shutdown_connections 2023-08-25T09:12:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e41860 0x55b047e35bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e3b990 0x55b047e3b450 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/1765677180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e40f40 0x55b047e4c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1765677180 >> 172.21.15.99:0/1765677180 conn(0x55b047db0080 msgr2=0x55b047e5bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1765677180 shutdown_connections 2023-08-25T09:12:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 -- 172.21.15.99:0/1765677180 wait complete. 2023-08-25T09:12:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 -- start start 2023-08-25T09:12:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.256+0000 7f1e7f899b80 1 Processor -- start 2023-08-25T09:12:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e3b990 0x55b047e3c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e3cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e7f899b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e41860 0x55b047e3d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047e3d840 con 0x55b047e3b990 2023-08-25T09:12:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047e3d9b0 con 0x55b047e41860 2023-08-25T09:12:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e7f899b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b047e3db70 con 0x55b047e40f40 2023-08-25T09:12:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e7f899b80 10 client.0 _ll_register_callbacks cb 0x7f1e5400ec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:12:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e6959a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e3cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e6959a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e3cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:42392/0 (socket says 172.21.15.99:42392) 2023-08-25T09:12:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e6959a700 1 -- 172.21.15.99:0/2045877061 learned_addr learned my addr 172.21.15.99:0/2045877061 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.257+0000 7f1e6a59c700 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e41860 0x55b047e3d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e6959a700 1 -- 172.21.15.99:0/2045877061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e41860 msgr2=0x55b047e3d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e6959a700 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e41860 0x55b047e3d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e6959a700 1 -- 172.21.15.99:0/2045877061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e3b990 msgr2=0x55b047e3c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e6959a700 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e3b990 0x55b047e3c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e6959a700 1 -- 172.21.15.99:0/2045877061 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e64104d30 con 0x55b047e40f40 2023-08-25T09:12:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e6959a700 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e3cdc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400e3e0 tx=0x7f1e54010f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89999cff61c0790b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:12:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/2045877061 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54005450 con 0x55b047e40f40 2023-08-25T09:12:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.258+0000 7f1e7f899b80 1 -- 172.21.15.99:0/2045877061 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b047e2e600 con 0x55b047e40f40 2023-08-25T09:12:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.259+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/2045877061 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f1e54017040 con 0x55b047e40f40 2023-08-25T09:12:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.259+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/2045877061 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e540055b0 con 0x55b047e40f40 2023-08-25T09:12:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.259+0000 7f1e7f899b80 10 client.26344 fetch_fsmap learned FSMap version 43 2023-08-25T09:12:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.259+0000 7f1e7f899b80 1 -- 172.21.15.99:0/2045877061 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b047be01f0 con 0x55b047e40f40 2023-08-25T09:12:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.259+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/2045877061 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e5403a020 con 0x55b047e40f40 2023-08-25T09:12:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.259+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/2045877061 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1e5402d070 con 0x55b047e40f40 2023-08-25T09:12:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e7f899b80 10 client.26344 fetch_fsmap finished waiting for FSMap version 43 2023-08-25T09:12:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e7f899b80 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:12:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e7f899b80 20 client.26344 populate_metadata read hostname 'smithi099' 2023-08-25T09:12:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b047ea9ca0 0x55b047eac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e7f899b80 4 client.26344 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:12:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e7f899b80 1 -- 172.21.15.99:0/2045877061 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b047d86910 con 0x55b047ea9ca0 2023-08-25T09:12:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b047ea9ca0 0x55b047eac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e69d9b700 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b047ea9ca0 0x55b047eac0c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=377f8bf24f7c47d3 server_cookie=4e56f3dba3ed26e7 in_seq=0 out_seq=0 2023-08-25T09:12:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.260+0000 7f1e4d7fa700 10 client.26344 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:35.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:35 smithi099 ceph-mon[97641]: pgmap v419: 97 pgs: 97 active+clean; 408 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2023-08-25T09:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.614+0000 7f1e4d7fa700 1 -- 172.21.15.99:0/2045877061 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b047d86910 con 0x55b047ea9ca0 2023-08-25T09:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.614+0000 7f1e4d7fa700 10 client.26344 handle_command_reply: tid=0 2023-08-25T09:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.614+0000 7f1e7f899b80 1 client.26344 shutdown 2023-08-25T09:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 -- 172.21.15.99:0/2045877061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 msgr2=0x55b047e3cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e3cdc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400e3e0 tx=0x7f1e54010f10 comp rx=0 tx=0).stop 2023-08-25T09:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 -- 172.21.15.99:0/2045877061 shutdown_connections 2023-08-25T09:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55b047ea9ca0 0x55b047eac0c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b047e41860 0x55b047e3d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b047e3b990 0x55b047e3c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 --2- 172.21.15.99:0/2045877061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b047e40f40 0x55b047e3cdc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 -- 172.21.15.99:0/2045877061 >> 172.21.15.99:0/2045877061 conn(0x55b047db0080 msgr2=0x55b047e35bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e4d7fa700 0 client.26344 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 -- 172.21.15.99:0/2045877061 shutdown_connections 2023-08-25T09:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.615+0000 7f1e7f899b80 1 -- 172.21.15.99:0/2045877061 wait complete. 2023-08-25T09:12:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.620+0000 7f1e7f899b80 20 client.26344 trim_cache size 0 max 16384 2023-08-25T09:12:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:12:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:12:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:12:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:12:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.623+0000 7f1e6bfff700 1 -- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f1e500a0210 msgr2=0x7f1e500a26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.623+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f1e500a0210 0x7f1e500a26d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c01e000 tx=0x7f1e5c014f90 comp rx=0 tx=0).stop 2023-08-25T09:12:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.623+0000 7f1e6bfff700 1 -- 172.21.15.99:0/3906866911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e64072cc0 msgr2=0x7f1e6419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.623+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e64072cc0 0x7f1e6419d040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006010 tx=0x7f1e60003c10 comp rx=0 tx=0).stop 2023-08-25T09:12:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:35 smithi157 ceph-mon[130879]: pgmap v419: 97 pgs: 97 active+clean; 408 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2023-08-25T09:12:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.631+0000 7f1e6bfff700 1 -- 172.21.15.99:0/3906866911 shutdown_connections 2023-08-25T09:12:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.631+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f1e500a0210 0x7f1e500a26d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.631+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e641118c0 0x7f1e641a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.631+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e64072cc0 0x7f1e6419d040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.631+0000 7f1e6bfff700 1 --2- 172.21.15.99:0/3906866911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e640735e0 0x7f1e6419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.631+0000 7f1e6bfff700 1 -- 172.21.15.99:0/3906866911 >> 172.21.15.99:0/3906866911 conn(0x7f1e640fc380 msgr2=0x7f1e640fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.635+0000 7f1e6bfff700 1 -- 172.21.15.99:0/3906866911 shutdown_connections 2023-08-25T09:12:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:35.635+0000 7f1e6bfff700 1 -- 172.21.15.99:0/3906866911 wait complete. 2023-08-25T09:12:35.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:35 smithi181 ceph-mon[131296]: pgmap v419: 97 pgs: 97 active+clean; 408 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2023-08-25T09:12:36.129 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:12:35 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ignoring --setuser ceph since I am not root 2023-08-25T09:12:36.130 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:12:35 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ignoring --setgroup ceph since I am not root 2023-08-25T09:12:36.130 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:12:35 smithi157 ceph-mds[155590]: -- 172.21.15.157:0/1302647364 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f734151e0 con 0x561f733ea800 2023-08-25T09:12:36.130 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:12:35 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: starting mds.h at 2023-08-25T09:12:36.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 16827 26.40 MB/sec warmup 41 sec latency 159.243 ms 2023-08-25T09:12:36.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:36 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:12:36.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:36 smithi099 ceph-mon[97641]: daemon mds.h restarted 2023-08-25T09:12:36.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:36 smithi099 ceph-mon[97641]: osdmap e103: 12 total, 12 up, 12 in 2023-08-25T09:12:36.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:36 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:36.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:36 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:36.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:36 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:12:36.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:36 smithi157 ceph-mon[130879]: daemon mds.h restarted 2023-08-25T09:12:36.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:36 smithi157 ceph-mon[130879]: osdmap e103: 12 total, 12 up, 12 in 2023-08-25T09:12:36.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:36 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:36.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:36 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:36.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:36 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:12:36.763 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:36 smithi181 ceph-mon[131296]: daemon mds.h restarted 2023-08-25T09:12:36.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:36 smithi181 ceph-mon[131296]: osdmap e103: 12 total, 12 up, 12 in 2023-08-25T09:12:36.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:36 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:36.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:36 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:37.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 17569 26.74 MB/sec warmup 42 sec latency 79.804 ms 2023-08-25T09:12:37.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:37 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6836/3442966510,v1:172.21.15.157:6837/3442966510] up:boot 2023-08-25T09:12:37.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:37 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T09:12:37.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:37 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:12:37.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:37 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:37.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:37 smithi099 ceph-mon[97641]: pgmap v421: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:37.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:37 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:12:37.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:37 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:12:37.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:37 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:37.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:37 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6836/3442966510,v1:172.21.15.157:6837/3442966510] up:boot 2023-08-25T09:12:37.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:37 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T09:12:37.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:37 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:12:37.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:37 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:37.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:37 smithi157 ceph-mon[130879]: pgmap v421: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:37.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:37 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:12:37.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:37 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:12:37.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:37 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:37 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6836/3442966510,v1:172.21.15.157:6837/3442966510] up:boot 2023-08-25T09:12:37.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:37 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T09:12:37.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:37 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:12:37.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:37 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:37.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:37 smithi181 ceph-mon[131296]: pgmap v421: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:37.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:37 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:12:37.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:37 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:12:37.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:37 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:12:38.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 18361 27.16 MB/sec warmup 43 sec latency 38.003 ms 2023-08-25T09:12:39.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 18879 27.03 MB/sec warmup 44 sec latency 222.283 ms 2023-08-25T09:12:39.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:39 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:39.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:39 smithi157 ceph-mon[130879]: pgmap v422: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:39 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:39 smithi181 ceph-mon[131296]: pgmap v422: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:39 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:12:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:39 smithi099 ceph-mon[97641]: pgmap v422: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:40.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 19590 27.31 MB/sec warmup 45 sec latency 52.973 ms 2023-08-25T09:12:41.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 20545 27.84 MB/sec warmup 46 sec latency 82.702 ms 2023-08-25T09:12:41.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:41 smithi181 ceph-mon[131296]: pgmap v423: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:41 smithi099 ceph-mon[97641]: pgmap v423: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:41 smithi157 ceph-mon[130879]: pgmap v423: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-08-25T09:12:42.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 21360 28.37 MB/sec warmup 47 sec latency 161.237 ms 2023-08-25T09:12:43.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 21952 28.22 MB/sec warmup 48 sec latency 69.028 ms 2023-08-25T09:12:43.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:43 smithi181 ceph-mon[131296]: pgmap v424: 97 pgs: 97 active+clean; 424 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 783 op/s 2023-08-25T09:12:43.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:43 smithi099 ceph-mon[97641]: pgmap v424: 97 pgs: 97 active+clean; 424 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 783 op/s 2023-08-25T09:12:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:43 smithi157 ceph-mon[130879]: pgmap v424: 97 pgs: 97 active+clean; 424 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 783 op/s 2023-08-25T09:12:44.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 22311 28.15 MB/sec warmup 49 sec latency 369.687 ms 2023-08-25T09:12:45.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 23063 28.26 MB/sec warmup 50 sec latency 381.337 ms 2023-08-25T09:12:45.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:45 smithi099 ceph-mon[97641]: pgmap v425: 97 pgs: 97 active+clean; 424 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 783 op/s 2023-08-25T09:12:45.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:45 smithi157 ceph-mon[130879]: pgmap v425: 97 pgs: 97 active+clean; 424 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 783 op/s 2023-08-25T09:12:45.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:45 smithi181 ceph-mon[131296]: pgmap v425: 97 pgs: 97 active+clean; 424 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 783 op/s 2023-08-25T09:12:46.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 24044 28.77 MB/sec warmup 51 sec latency 32.944 ms 2023-08-25T09:12:47.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 24842 29.02 MB/sec warmup 52 sec latency 51.699 ms 2023-08-25T09:12:47.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:47 smithi181 ceph-mon[131296]: pgmap v426: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 845 op/s 2023-08-25T09:12:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:47 smithi099 ceph-mon[97641]: pgmap v426: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 845 op/s 2023-08-25T09:12:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:47 smithi157 ceph-mon[130879]: pgmap v426: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 845 op/s 2023-08-25T09:12:48.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 25606 29.47 MB/sec warmup 53 sec latency 87.841 ms 2023-08-25T09:12:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 -- 172.21.15.99:0/4211851984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b001068c0 msgr2=0x7f9b00106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 --2- 172.21.15.99:0/4211851984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b001068c0 0x7f9b00106ca0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009990 tx=0x7f9af0009ca0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 -- 172.21.15.99:0/4211851984 shutdown_connections 2023-08-25T09:12:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 --2- 172.21.15.99:0/4211851984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b00101270 0x7f9b0010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 --2- 172.21.15.99:0/4211851984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b001068c0 0x7f9b00106ca0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 --2- 172.21.15.99:0/4211851984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b001008b0 0x7f9b00100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 -- 172.21.15.99:0/4211851984 >> 172.21.15.99:0/4211851984 conn(0x7f9b000fc380 msgr2=0x7f9b000fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 -- 172.21.15.99:0/4211851984 shutdown_connections 2023-08-25T09:12:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.822+0000 7f9b049ff700 1 -- 172.21.15.99:0/4211851984 wait complete. 2023-08-25T09:12:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.824+0000 7f9b049ff700 1 Processor -- start 2023-08-25T09:12:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.824+0000 7f9b049ff700 1 -- start start 2023-08-25T09:12:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.824+0000 7f9b049ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b001008b0 0x7f9b00074330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.824+0000 7f9b049ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b00101270 0x7f9b0006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.824+0000 7f9b049ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b001068c0 0x7f9b0006f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.824+0000 7f9b049ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0006fb90 con 0x7f9b001068c0 2023-08-25T09:12:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.824+0000 7f9b049ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0006fcd0 con 0x7f9b00101270 2023-08-25T09:12:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.824+0000 7f9b049ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0006fe90 con 0x7f9b001008b0 2023-08-25T09:12:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b001008b0 0x7f9b00074330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b001008b0 0x7f9b00074330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:54372/0 (socket says 172.21.15.99:54372) 2023-08-25T09:12:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afe59c700 1 -- 172.21.15.99:0/2402760780 learned_addr learned my addr 172.21.15.99:0/2402760780 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b001068c0 0x7f9b0006f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b00101270 0x7f9b0006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afe59c700 1 -- 172.21.15.99:0/2402760780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b00101270 msgr2=0x7f9b0006f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afe59c700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b00101270 0x7f9b0006f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afe59c700 1 -- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b001068c0 msgr2=0x7f9b0006f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afe59c700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b001068c0 0x7f9b0006f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afe59c700 1 -- 172.21.15.99:0/2402760780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8009c40 con 0x7f9b001008b0 2023-08-25T09:12:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.825+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b00101270 0x7f9b0006f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:12:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.826+0000 7f9afe59c700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b001008b0 0x7f9b00074330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9af0005910 tx=0x7f9af0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.826+0000 7f9aef7fe700 1 -- 172.21.15.99:0/2402760780 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af001d070 con 0x7f9b001008b0 2023-08-25T09:12:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.826+0000 7f9aef7fe700 1 -- 172.21.15.99:0/2402760780 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9af0022470 con 0x7f9b001008b0 2023-08-25T09:12:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.826+0000 7f9b049ff700 1 -- 172.21.15.99:0/2402760780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af00095f0 con 0x7f9b001008b0 2023-08-25T09:12:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.826+0000 7f9aef7fe700 1 -- 172.21.15.99:0/2402760780 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0017650 con 0x7f9b001008b0 2023-08-25T09:12:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.826+0000 7f9b049ff700 1 -- 172.21.15.99:0/2402760780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b001ae1c0 con 0x7f9b001008b0 2023-08-25T09:12:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.831+0000 7f9aef7fe700 1 -- 172.21.15.99:0/2402760780 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9af000fad0 con 0x7f9b001008b0 2023-08-25T09:12:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.832+0000 7f9b1832eb80 1 Processor -- start 2023-08-25T09:12:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.832+0000 7f9b1832eb80 1 -- start start 2023-08-25T09:12:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.832+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7d1f40 0x562d0b7d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7cc990 0x562d0b7ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b734750 con 0x562d0b7cc990 2023-08-25T09:12:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b725720 con 0x562d0b7d1f40 2023-08-25T09:12:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b723040 con 0x562d0b7d2860 2023-08-25T09:12:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7d1f40 0x562d0b7d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7d1f40 0x562d0b7d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:35204/0 (socket says 172.21.15.99:35204) 2023-08-25T09:12:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afed9d700 1 -- 172.21.15.99:0/3352296460 learned_addr learned my addr 172.21.15.99:0/3352296460 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afe59c700 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afed9d700 1 -- 172.21.15.99:0/3352296460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 msgr2=0x562d0b7cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7cc990 0x562d0b7ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afed9d700 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9aef7fe700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ae80a0140 0x7f9ae80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afed9d700 1 -- 172.21.15.99:0/3352296460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7cc990 msgr2=0x562d0b7ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afed9d700 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7cc990 0x562d0b7ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9afed9d700 1 -- 172.21.15.99:0/3352296460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d0b59e5d0 con 0x562d0b7d1f40 2023-08-25T09:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.833+0000 7f9aef7fe700 1 -- 172.21.15.99:0/2402760780 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9af00cca50 con 0x7f9b001008b0 2023-08-25T09:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.834+0000 7f9afe59c700 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.834+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7cc990 0x562d0b7ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.834+0000 7f9afed9d700 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7d1f40 0x562d0b7d2320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e730 tx=0x7f9af8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abdb57c8eb55f893 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.834+0000 7f9aecff9700 1 -- 172.21.15.99:0/3352296460 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800e040 con 0x562d0b7d1f40 2023-08-25T09:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.834+0000 7f9afed9d700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ae80a0140 0x7f9ae80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.834+0000 7f9aecff9700 1 -- 172.21.15.99:0/3352296460 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9af800f040 con 0x562d0b7d1f40 2023-08-25T09:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.834+0000 7f9aecff9700 1 -- 172.21.15.99:0/3352296460 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8015910 con 0x562d0b7d1f40 2023-08-25T09:12:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3352296460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7d1f40 msgr2=0x562d0b7d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7d1f40 0x562d0b7d2320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e730 tx=0x7f9af8009920 comp rx=0 tx=0).stop 2023-08-25T09:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9afed9d700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ae80a0140 0x7f9ae80a2600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9af8015fd0 tx=0x7f9af8015bc0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3352296460 shutdown_connections 2023-08-25T09:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7cc990 0x562d0b7ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7d1f40 0x562d0b7d2320 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/3352296460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3352296460 >> 172.21.15.99:0/3352296460 conn(0x562d0b741080 msgr2=0x562d0b7c6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3352296460 shutdown_connections 2023-08-25T09:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.835+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3352296460 wait complete. 2023-08-25T09:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.836+0000 7f9b1832eb80 1 -- start start 2023-08-25T09:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.836+0000 7f9b1832eb80 1 Processor -- start 2023-08-25T09:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.837+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.837+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d1f40 0x562d0b7d5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.837+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.837+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b71e930 con 0x562d0b7d1f40 2023-08-25T09:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.837+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b6bcb50 con 0x562d0b7cc990 2023-08-25T09:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.837+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b6bc850 con 0x562d0b7d2860 2023-08-25T09:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.837+0000 7f9b1832eb80 10 client.0 _ll_register_callbacks cb 0x562d0b606a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7d8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:35210/0 (socket says 172.21.15.99:35210) 2023-08-25T09:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afe59c700 1 -- 172.21.15.99:0/944701198 learned_addr learned my addr 172.21.15.99:0/944701198 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afed9d700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afe59c700 1 -- 172.21.15.99:0/944701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 msgr2=0x562d0b7d5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d1f40 0x562d0b7d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afe59c700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7d5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afe59c700 1 -- 172.21.15.99:0/944701198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d1f40 msgr2=0x562d0b7d5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afe59c700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d1f40 0x562d0b7d5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afe59c700 1 -- 172.21.15.99:0/944701198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af800e410 con 0x562d0b7cc990 2023-08-25T09:12:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afed9d700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:12:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.838+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d1f40 0x562d0b7d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:12:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.839+0000 7f9afe59c700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7d8650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9af0000c00 tx=0x7f9af00964f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea8a1f1c79dc5674 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.839+0000 7f9ae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:12:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.839+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/944701198 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0099cf0 con 0x562d0b7cc990 2023-08-25T09:12:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.839+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/944701198 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9af009a8b0 con 0x562d0b7cc990 2023-08-25T09:12:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.839+0000 7f9b1832eb80 1 -- 172.21.15.99:0/944701198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d0b79c160 con 0x562d0b7cc990 2023-08-25T09:12:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.839+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/944701198 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9af0099b10 con 0x562d0b7cc990 2023-08-25T09:12:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.839+0000 7f9b1832eb80 10 client.26375 fetch_fsmap learned FSMap version 46 2023-08-25T09:12:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.839+0000 7f9b1832eb80 1 -- 172.21.15.99:0/944701198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x562d0b54ad80 con 0x562d0b7cc990 2023-08-25T09:12:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/944701198 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d0b79c160 con 0x562d0b7cc990 2023-08-25T09:12:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/944701198 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9af00996a0 con 0x562d0b7cc990 2023-08-25T09:12:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9b1832eb80 10 client.26375 fetch_fsmap finished waiting for FSMap version 46 2023-08-25T09:12:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9b1832eb80 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:12:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9b1832eb80 20 client.26375 populate_metadata read hostname 'smithi099' 2023-08-25T09:12:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562d0b7dcd20 0x562d0b7df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9b1832eb80 4 client.26375 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:12:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9b1832eb80 1 -- 172.21.15.99:0/944701198 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d0b6c4b40 con 0x562d0b7dcd20 2023-08-25T09:12:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.840+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562d0b7dcd20 0x562d0b7df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.841+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562d0b7dcd20 0x562d0b7df140 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e22155626c6d5a9 server_cookie=f215a9bb10218b82 in_seq=0 out_seq=0 2023-08-25T09:12:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.841+0000 7f9ae5ffb700 10 client.26375 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/944701198 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562d0b6c4b40 con 0x562d0b7dcd20 2023-08-25T09:12:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9ae5ffb700 10 client.26375 handle_command_reply: tid=0 2023-08-25T09:12:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 client.26375 shutdown 2023-08-25T09:12:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 -- 172.21.15.99:0/944701198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 msgr2=0x562d0b7d8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7d8650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9af0000c00 tx=0x7f9af00964f0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 -- 172.21.15.99:0/944701198 shutdown_connections 2023-08-25T09:12:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d2860 0x562d0b7d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562d0b7dcd20 0x562d0b7df140 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7d8650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/944701198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d1f40 0x562d0b7d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9ae5ffb700 0 client.26375 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 -- 172.21.15.99:0/944701198 >> 172.21.15.99:0/944701198 conn(0x562d0b741080 msgr2=0x562d0b7c1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.842+0000 7f9b1832eb80 1 -- 172.21.15.99:0/944701198 shutdown_connections 2023-08-25T09:12:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.843+0000 7f9b1832eb80 1 -- 172.21.15.99:0/944701198 wait complete. 2023-08-25T09:12:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.843+0000 7f9b1832eb80 20 client.26375 trim_cache size 0 max 16384 2023-08-25T09:12:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.862+0000 7f9b1832eb80 1 Processor -- start 2023-08-25T09:12:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.862+0000 7f9b1832eb80 1 -- start start 2023-08-25T09:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 0x562d0b7df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b5a08c0 con 0x562d0b7d2860 2023-08-25T09:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b7c3700 con 0x562d0b7cc990 2023-08-25T09:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b7d6830 con 0x562d0b7d1f40 2023-08-25T09:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:35212/0 (socket says 172.21.15.99:35212) 2023-08-25T09:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9afe59c700 1 -- 172.21.15.99:0/3873172792 learned_addr learned my addr 172.21.15.99:0/3873172792 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9afed9d700 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.863+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 0x562d0b7df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9afe59c700 1 -- 172.21.15.99:0/3873172792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 msgr2=0x562d0b7df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9afe59c700 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 0x562d0b7df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9afe59c700 1 -- 172.21.15.99:0/3873172792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 msgr2=0x562d0b7e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9afe59c700 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9afe59c700 1 -- 172.21.15.99:0/3873172792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d0b533c00 con 0x562d0b7cc990 2023-08-25T09:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9afed9d700 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9afe59c700 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cc450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9af0096c20 tx=0x7f9af00ccdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40920cb8e6bb5fa9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9aed7fa700 1 -- 172.21.15.99:0/3873172792 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b22e0 con 0x562d0b7cc990 2023-08-25T09:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9aed7fa700 1 -- 172.21.15.99:0/3873172792 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9af0099cf0 con 0x562d0b7cc990 2023-08-25T09:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.864+0000 7f9aed7fa700 1 -- 172.21.15.99:0/3873172792 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00a1f30 con 0x562d0b7cc990 2023-08-25T09:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.865+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3873172792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 msgr2=0x562d0b7cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.865+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cc450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9af0096c20 tx=0x7f9af00ccdc0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.865+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3873172792 shutdown_connections 2023-08-25T09:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.865+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.865+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cc450 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.865+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/3873172792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 0x562d0b7df110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.865+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3873172792 >> 172.21.15.99:0/3873172792 conn(0x562d0b741080 msgr2=0x562d0b7c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.865+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3873172792 shutdown_connections 2023-08-25T09:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.866+0000 7f9b1832eb80 1 -- 172.21.15.99:0/3873172792 wait complete. 2023-08-25T09:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.866+0000 7f9b1832eb80 1 -- start start 2023-08-25T09:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.866+0000 7f9b1832eb80 1 Processor -- start 2023-08-25T09:12:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 0x562d0b7cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9b1832eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b7ce8e0 con 0x562d0b7d2860 2023-08-25T09:12:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b7cea50 con 0x562d0b7cc990 2023-08-25T09:12:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9b1832eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0b7cec10 con 0x562d0b7d1f40 2023-08-25T09:12:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9b1832eb80 10 client.0 _ll_register_callbacks cb 0x562d0b606a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:12:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:35226/0 (socket says 172.21.15.99:35226) 2023-08-25T09:12:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afe59c700 1 -- 172.21.15.99:0/224493377 learned_addr learned my addr 172.21.15.99:0/224493377 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:12:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 0x562d0b7cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afed9d700 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afe59c700 1 -- 172.21.15.99:0/224493377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 msgr2=0x562d0b7cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afe59c700 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 0x562d0b7cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afe59c700 1 -- 172.21.15.99:0/224493377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 msgr2=0x562d0b7ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:12:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afe59c700 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.867+0000 7f9afe59c700 1 -- 172.21.15.99:0/224493377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af00cce70 con 0x562d0b7cc990 2023-08-25T09:12:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9afe59c700 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cd880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9af0006010 tx=0x7f9af009a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9055a6898d25645 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:12:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9ae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:12:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/224493377 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00cce70 con 0x562d0b7cc990 2023-08-25T09:12:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/224493377 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9af009a1b0 con 0x562d0b7cc990 2023-08-25T09:12:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9b1832eb80 1 -- 172.21.15.99:0/224493377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d0b7bf600 con 0x562d0b7cc990 2023-08-25T09:12:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/224493377 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9af0099b10 con 0x562d0b7cc990 2023-08-25T09:12:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9b1832eb80 10 client.26381 fetch_fsmap learned FSMap version 46 2023-08-25T09:12:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9b1832eb80 1 -- 172.21.15.99:0/224493377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x562d0b5711f0 con 0x562d0b7cc990 2023-08-25T09:12:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/224493377 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d0b7bf600 con 0x562d0b7cc990 2023-08-25T09:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.868+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/224493377 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9af0099230 con 0x562d0b7cc990 2023-08-25T09:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9b1832eb80 10 client.26381 fetch_fsmap finished waiting for FSMap version 46 2023-08-25T09:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9b1832eb80 10 client.26381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9b1832eb80 20 client.26381 populate_metadata read hostname 'smithi099' 2023-08-25T09:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562d0b84b240 0x562d0b84d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9b1832eb80 4 client.26381 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9b1832eb80 1 -- 172.21.15.99:0/224493377 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562d0b717910 con 0x562d0b84b240 2023-08-25T09:12:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562d0b84b240 0x562d0b84d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:12:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9afdd9b700 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562d0b84b240 0x562d0b84d630 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1de2cb81b1fe61 server_cookie=81f15b5a1185c6d9 in_seq=0 out_seq=0 2023-08-25T09:12:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:48.869+0000 7f9ae5ffb700 10 client.26381 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.005+0000 7f9ae5ffb700 1 -- 172.21.15.99:0/224493377 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562d0b717910 con 0x562d0b84b240 2023-08-25T09:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.005+0000 7f9ae5ffb700 10 client.26381 handle_command_reply: tid=0 2023-08-25T09:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.005+0000 7f9b1832eb80 1 client.26381 shutdown 2023-08-25T09:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.005+0000 7f9b1832eb80 1 -- 172.21.15.99:0/224493377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 msgr2=0x562d0b7cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.005+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cd880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9af0006010 tx=0x7f9af009a6f0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9b1832eb80 1 -- 172.21.15.99:0/224493377 shutdown_connections 2023-08-25T09:12:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562d0b7d2860 0x562d0b7ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x562d0b84b240 0x562d0b84d630 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d0b7cc990 0x562d0b7cd880 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9b1832eb80 1 --2- 172.21.15.99:0/224493377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d0b7d1f40 0x562d0b7cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9b1832eb80 1 -- 172.21.15.99:0/224493377 >> 172.21.15.99:0/224493377 conn(0x562d0b741080 msgr2=0x562d0b7cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9ae5ffb700 0 client.26381 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:12:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9b1832eb80 1 -- 172.21.15.99:0/224493377 shutdown_connections 2023-08-25T09:12:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.010+0000 7f9b1832eb80 1 -- 172.21.15.99:0/224493377 wait complete. 2023-08-25T09:12:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.011+0000 7f9b1832eb80 20 client.26381 trim_cache size 0 max 16384 2023-08-25T09:12:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:12:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:12:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:12:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:12:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.012+0000 7f9b049ff700 1 -- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ae80a0140 msgr2=0x7f9ae80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.012+0000 7f9b049ff700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ae80a0140 0x7f9ae80a2600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9af8015fd0 tx=0x7f9af8015bc0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.012+0000 7f9b049ff700 1 -- 172.21.15.99:0/2402760780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b001008b0 msgr2=0x7f9b00074330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:12:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.012+0000 7f9b049ff700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b001008b0 0x7f9b00074330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9af0005910 tx=0x7f9af0005740 comp rx=0 tx=0).stop 2023-08-25T09:12:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.013+0000 7f9b049ff700 1 -- 172.21.15.99:0/2402760780 shutdown_connections 2023-08-25T09:12:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.013+0000 7f9b049ff700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ae80a0140 0x7f9ae80a2600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.013+0000 7f9b049ff700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b001068c0 0x7f9b0006f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.013+0000 7f9b049ff700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b001008b0 0x7f9b00074330 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.013+0000 7f9b049ff700 1 --2- 172.21.15.99:0/2402760780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b00101270 0x7f9b0006f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:12:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.013+0000 7f9b049ff700 1 -- 172.21.15.99:0/2402760780 >> 172.21.15.99:0/2402760780 conn(0x7f9b000fc380 msgr2=0x7f9b000fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:12:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.013+0000 7f9b049ff700 1 -- 172.21.15.99:0/2402760780 shutdown_connections 2023-08-25T09:12:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:12:49.013+0000 7f9b049ff700 1 -- 172.21.15.99:0/2402760780 wait complete. 2023-08-25T09:12:49.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 26302 29.42 MB/sec warmup 54 sec latency 262.142 ms 2023-08-25T09:12:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:49 smithi181 ceph-mon[131296]: pgmap v427: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-08-25T09:12:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:49 smithi099 ceph-mon[97641]: pgmap v427: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-08-25T09:12:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:49 smithi157 ceph-mon[130879]: pgmap v427: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-08-25T09:12:50.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 27031 29.59 MB/sec warmup 55 sec latency 112.803 ms 2023-08-25T09:12:51.245 INFO:tasks.workunit.client.0.smithi099.stdout: 10 27888 29.95 MB/sec warmup 56 sec latency 76.947 ms 2023-08-25T09:12:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:51 smithi181 ceph-mon[131296]: pgmap v428: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-08-25T09:12:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:51 smithi099 ceph-mon[97641]: pgmap v428: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-08-25T09:12:51.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:51 smithi157 ceph-mon[130879]: pgmap v428: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-08-25T09:12:52.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 28852 30.13 MB/sec warmup 57 sec latency 39.791 ms 2023-08-25T09:12:53.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 29840 30.80 MB/sec warmup 58 sec latency 50.336 ms 2023-08-25T09:12:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:53 smithi181 ceph-mon[131296]: pgmap v429: 97 pgs: 97 active+clean; 452 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2023-08-25T09:12:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:53 smithi099 ceph-mon[97641]: pgmap v429: 97 pgs: 97 active+clean; 452 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2023-08-25T09:12:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:53 smithi157 ceph-mon[130879]: pgmap v429: 97 pgs: 97 active+clean; 452 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2023-08-25T09:12:54.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 30604 30.89 MB/sec warmup 59 sec latency 66.380 ms 2023-08-25T09:12:55.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 31352 31.07 MB/sec warmup 60 sec latency 143.937 ms 2023-08-25T09:12:55.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:55 smithi099 ceph-mon[97641]: pgmap v430: 97 pgs: 97 active+clean; 452 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 694 op/s 2023-08-25T09:12:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:55 smithi157 ceph-mon[130879]: pgmap v430: 97 pgs: 97 active+clean; 452 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 694 op/s 2023-08-25T09:12:55.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:55 smithi181 ceph-mon[131296]: pgmap v430: 97 pgs: 97 active+clean; 452 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 694 op/s 2023-08-25T09:12:56.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 32221 31.27 MB/sec warmup 61 sec latency 75.549 ms 2023-08-25T09:12:57.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 33097 31.58 MB/sec warmup 62 sec latency 38.971 ms 2023-08-25T09:12:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:57 smithi181 ceph-mon[131296]: pgmap v431: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 924 op/s 2023-08-25T09:12:57.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:57 smithi099 ceph-mon[97641]: pgmap v431: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 924 op/s 2023-08-25T09:12:57.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:57 smithi157 ceph-mon[130879]: pgmap v431: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 924 op/s 2023-08-25T09:12:58.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 33869 31.64 MB/sec warmup 63 sec latency 61.078 ms 2023-08-25T09:12:59.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 34847 32.04 MB/sec warmup 64 sec latency 83.970 ms 2023-08-25T09:12:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:12:59 smithi181 ceph-mon[131296]: pgmap v432: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2023-08-25T09:12:59.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:12:59 smithi099 ceph-mon[97641]: pgmap v432: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2023-08-25T09:12:59.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:12:59 smithi157 ceph-mon[130879]: pgmap v432: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2023-08-25T09:13:00.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 35258 32.00 MB/sec warmup 65 sec latency 446.342 ms 2023-08-25T09:13:01.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 35358 31.59 MB/sec warmup 66 sec latency 849.538 ms 2023-08-25T09:13:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:01 smithi181 ceph-mon[131296]: pgmap v433: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2023-08-25T09:13:01.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:01 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:13:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:01 smithi099 ceph-mon[97641]: pgmap v433: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2023-08-25T09:13:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:01 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:13:01.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:01 smithi157 ceph-mon[130879]: pgmap v433: 97 pgs: 97 active+clean; 479 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2023-08-25T09:13:01.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:01 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:13:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 -- 172.21.15.99:0/1246042925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 msgr2=0x7fe348100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 --2- 172.21.15.99:0/1246042925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 0x7fe348100d20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe3340098f0 tx=0x7fe334009c00 comp rx=0 tx=0).stop 2023-08-25T09:13:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 -- 172.21.15.99:0/1246042925 shutdown_connections 2023-08-25T09:13:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 --2- 172.21.15.99:0/1246042925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe348101260 0x7fe34810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 --2- 172.21.15.99:0/1246042925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3481068b0 0x7fe348106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 --2- 172.21.15.99:0/1246042925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 0x7fe348100d20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 -- 172.21.15.99:0/1246042925 >> 172.21.15.99:0/1246042925 conn(0x7fe3480fc370 msgr2=0x7fe3480fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 -- 172.21.15.99:0/1246042925 shutdown_connections 2023-08-25T09:13:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.180+0000 7fe34ebff700 1 -- 172.21.15.99:0/1246042925 wait complete. 2023-08-25T09:13:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.181+0000 7fe34ebff700 1 Processor -- start 2023-08-25T09:13:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.181+0000 7fe34ebff700 1 -- start start 2023-08-25T09:13:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34ebff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 0x7fe34819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34ebff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe348101260 0x7fe34819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34ebff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3481068b0 0x7fe348196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34ebff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348196e00 con 0x7fe3481068b0 2023-08-25T09:13:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34ebff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348196f70 con 0x7fe3481008a0 2023-08-25T09:13:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34ebff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348197130 con 0x7fe348101260 2023-08-25T09:13:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 0x7fe34819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34d19c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3481068b0 0x7fe348196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 0x7fe34819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47410/0 (socket says 172.21.15.99:47410) 2023-08-25T09:13:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34c99b700 1 -- 172.21.15.99:0/3225108294 learned_addr learned my addr 172.21.15.99:0/3225108294 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe347fff700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe348101260 0x7fe34819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34c99b700 1 -- 172.21.15.99:0/3225108294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe348101260 msgr2=0x7fe34819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34c99b700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe348101260 0x7fe34819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.182+0000 7fe34c99b700 1 -- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3481068b0 msgr2=0x7fe348196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe34c99b700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3481068b0 0x7fe348196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe34c99b700 1 -- 172.21.15.99:0/3225108294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c0095f0 con 0x7fe3481008a0 2023-08-25T09:13:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe347fff700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe348101260 0x7fe34819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:13:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe34c99b700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 0x7fe34819c930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00ec10 tx=0x7fe33c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe345ffb700 1 -- 172.21.15.99:0/3225108294 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c00cc90 con 0x7fe3481008a0 2023-08-25T09:13:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe34ebff700 1 -- 172.21.15.99:0/3225108294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe334009520 con 0x7fe3481008a0 2023-08-25T09:13:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe345ffb700 1 -- 172.21.15.99:0/3225108294 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe33c0044e0 con 0x7fe3481008a0 2023-08-25T09:13:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe345ffb700 1 -- 172.21.15.99:0/3225108294 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c010430 con 0x7fe3481008a0 2023-08-25T09:13:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.183+0000 7fe34ebff700 1 -- 172.21.15.99:0/3225108294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3481975d0 con 0x7fe3481008a0 2023-08-25T09:13:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.187+0000 7fe345ffb700 1 -- 172.21.15.99:0/3225108294 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fe33c003fb0 con 0x7fe3481008a0 2023-08-25T09:13:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe36256bb80 1 Processor -- start 2023-08-25T09:13:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe36256bb80 1 -- start start 2023-08-25T09:13:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe345ffb700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe3380a00c0 0x7fe3380a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54378f40 0x560c54379320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe345ffb700 1 -- 172.21.15.99:0/3225108294 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe33c014070 con 0x7fe3481008a0 2023-08-25T09:13:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54379860 0x560c54373450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c54373dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe36256bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c542db750 con 0x560c54379860 2023-08-25T09:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.188+0000 7fe36256bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c542cc720 con 0x560c54373990 2023-08-25T09:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe36256bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c542ca040 con 0x560c54378f40 2023-08-25T09:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34d19c700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe3380a00c0 0x7fe3380a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54378f40 0x560c54379320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54378f40 0x560c54379320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:53428/0 (socket says 172.21.15.99:53428) 2023-08-25T09:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 -- 172.21.15.99:0/3485002003 learned_addr learned my addr 172.21.15.99:0/3485002003 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe347fff700 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54379860 0x560c54373450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34d19c700 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c54373dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 -- 172.21.15.99:0/3485002003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 msgr2=0x560c54373dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c54373dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 -- 172.21.15.99:0/3485002003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54379860 msgr2=0x560c54373450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54379860 0x560c54373450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 -- 172.21.15.99:0/3485002003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c541455d0 con 0x560c54378f40 2023-08-25T09:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe347fff700 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54379860 0x560c54373450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:13:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34d19c700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe3380a00c0 0x7fe3380a2580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe340009ec0 tx=0x7fe34000fc40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.189+0000 7fe34c99b700 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54378f40 0x560c54379320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0a0af0 tx=0x7fe33c095680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c2eb513ec2bef1e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe332ffd700 1 -- 172.21.15.99:0/3485002003 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0b0070 con 0x560c54378f40 2023-08-25T09:13:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe332ffd700 1 -- 172.21.15.99:0/3485002003 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe33c099910 con 0x560c54378f40 2023-08-25T09:13:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe332ffd700 1 -- 172.21.15.99:0/3485002003 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c004260 con 0x560c54378f40 2023-08-25T09:13:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 -- 172.21.15.99:0/3485002003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54378f40 msgr2=0x560c54379320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54378f40 0x560c54379320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0a0af0 tx=0x7fe33c095680 comp rx=0 tx=0).stop 2023-08-25T09:13:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 -- 172.21.15.99:0/3485002003 shutdown_connections 2023-08-25T09:13:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c54373dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54378f40 0x560c54379320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 --2- 172.21.15.99:0/3485002003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54379860 0x560c54373450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 -- 172.21.15.99:0/3485002003 >> 172.21.15.99:0/3485002003 conn(0x560c542e8080 msgr2=0x560c5436dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 -- 172.21.15.99:0/3485002003 shutdown_connections 2023-08-25T09:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.190+0000 7fe36256bb80 1 -- 172.21.15.99:0/3485002003 wait complete. 2023-08-25T09:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.191+0000 7fe36256bb80 1 -- start start 2023-08-25T09:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.191+0000 7fe36256bb80 1 Processor -- start 2023-08-25T09:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54373990 0x560c5437f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54379860 0x560c5437d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe36256bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c542c5930 con 0x560c54378f40 2023-08-25T09:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe36256bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c54263b50 con 0x560c54379860 2023-08-25T09:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe36256bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c54263850 con 0x560c54373990 2023-08-25T09:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe36256bb80 10 client.0 _ll_register_callbacks cb 0x560c541ada20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54373990 0x560c5437f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe347fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54373990 0x560c5437f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:53436/0 (socket says 172.21.15.99:53436) 2023-08-25T09:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.192+0000 7fe34c99b700 1 -- 172.21.15.99:0/2180753778 learned_addr learned my addr 172.21.15.99:0/2180753778 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe34d19c700 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54379860 0x560c5437d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe34c99b700 1 -- 172.21.15.99:0/2180753778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54379860 msgr2=0x560c5437d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe34c99b700 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54379860 0x560c5437d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe34c99b700 1 -- 172.21.15.99:0/2180753778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 msgr2=0x560c5437c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe34c99b700 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe34c99b700 1 -- 172.21.15.99:0/2180753778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3481975d0 con 0x560c54373990 2023-08-25T09:13:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe34c99b700 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54373990 0x560c5437f730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe33c095c80 tx=0x7fe33c09d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce538fb666cf38e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe327fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:13:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe327fff700 1 -- 172.21.15.99:0/2180753778 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0b0040 con 0x560c54373990 2023-08-25T09:13:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe36256bb80 1 -- 172.21.15.99:0/2180753778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c54343160 con 0x560c54373990 2023-08-25T09:13:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.193+0000 7fe327fff700 1 -- 172.21.15.99:0/2180753778 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe33c099210 con 0x560c54373990 2023-08-25T09:13:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe327fff700 1 -- 172.21.15.99:0/2180753778 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe33c0a1020 con 0x560c54373990 2023-08-25T09:13:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe36256bb80 10 client.26380 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe36256bb80 1 -- 172.21.15.99:0/2180753778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560c540f1d80 con 0x560c54373990 2023-08-25T09:13:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe327fff700 1 -- 172.21.15.99:0/2180753778 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe33c0c7080 con 0x560c54373990 2023-08-25T09:13:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe327fff700 1 -- 172.21.15.99:0/2180753778 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe33c098700 con 0x560c54373990 2023-08-25T09:13:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe36256bb80 10 client.26380 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe36256bb80 10 client.26380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe36256bb80 20 client.26380 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560c5438ce20 0x560c5438f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe36256bb80 4 client.26380 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe36256bb80 1 -- 172.21.15.99:0/2180753778 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c5426bb40 con 0x560c5438ce20 2023-08-25T09:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.194+0000 7fe347fff700 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560c5438ce20 0x560c5438f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.195+0000 7fe347fff700 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560c5438ce20 0x560c5438f210 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20ea1de55121797 server_cookie=10ff3f51072f1380 in_seq=0 out_seq=0 2023-08-25T09:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.195+0000 7fe327fff700 10 client.26380 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe327fff700 1 -- 172.21.15.99:0/2180753778 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c5426bb40 con 0x560c5438ce20 2023-08-25T09:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe327fff700 10 client.26380 handle_command_reply: tid=0 2023-08-25T09:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 client.26380 shutdown 2023-08-25T09:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 -- 172.21.15.99:0/2180753778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54373990 msgr2=0x560c5437f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54373990 0x560c5437f730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe33c095c80 tx=0x7fe33c09d040 comp rx=0 tx=0).stop 2023-08-25T09:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 -- 172.21.15.99:0/2180753778 shutdown_connections 2023-08-25T09:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560c5438ce20 0x560c5438f210 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54379860 0x560c5437d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54373990 0x560c5437f730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe327fff700 0 client.26380 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2180753778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 -- 172.21.15.99:0/2180753778 >> 172.21.15.99:0/2180753778 conn(0x560c542e8080 msgr2=0x560c54368820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 -- 172.21.15.99:0/2180753778 shutdown_connections 2023-08-25T09:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.196+0000 7fe36256bb80 1 -- 172.21.15.99:0/2180753778 wait complete. 2023-08-25T09:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.197+0000 7fe36256bb80 20 client.26380 trim_cache size 0 max 16384 2023-08-25T09:13:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.211+0000 7fe36256bb80 1 Processor -- start 2023-08-25T09:13:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe36256bb80 1 -- start start 2023-08-25T09:13:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c54373450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c54384240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 0x560c5437f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe36256bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c541478c0 con 0x560c54378f40 2023-08-25T09:13:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe36256bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c54366460 con 0x560c54373990 2023-08-25T09:13:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe36256bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5436fa10 con 0x560c54379860 2023-08-25T09:13:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe347fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c54384240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c54373450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe347fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c54384240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37882/0 (socket says 172.21.15.99:37882) 2023-08-25T09:13:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe347fff700 1 -- 172.21.15.99:0/1255435153 learned_addr learned my addr 172.21.15.99:0/1255435153 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe34d19c700 1 --2- 172.21.15.99:0/1255435153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 0x560c5437f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe347fff700 1 -- 172.21.15.99:0/1255435153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 msgr2=0x560c5437f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe347fff700 1 --2- 172.21.15.99:0/1255435153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 0x560c5437f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe347fff700 1 -- 172.21.15.99:0/1255435153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 msgr2=0x560c54373450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.212+0000 7fe347fff700 1 --2- 172.21.15.99:0/1255435153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c54373450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.213+0000 7fe347fff700 1 -- 172.21.15.99:0/1255435153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c54374040 con 0x560c54378f40 2023-08-25T09:13:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.213+0000 7fe347fff700 1 --2- 172.21.15.99:0/1255435153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c54384240 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe3340098f0 tx=0x7fe33400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b810cda1ce1820d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.213+0000 7fe3337fe700 1 -- 172.21.15.99:0/1255435153 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33401c040 con 0x560c54378f40 2023-08-25T09:13:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.213+0000 7fe3337fe700 1 -- 172.21.15.99:0/1255435153 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe334027150 con 0x560c54378f40 2023-08-25T09:13:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.213+0000 7fe3337fe700 1 -- 172.21.15.99:0/1255435153 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33400bae0 con 0x560c54378f40 2023-08-25T09:13:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 -- 172.21.15.99:0/1255435153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 msgr2=0x560c54384240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 --2- 172.21.15.99:0/1255435153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c54384240 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe3340098f0 tx=0x7fe33400bef0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 -- 172.21.15.99:0/1255435153 shutdown_connections 2023-08-25T09:13:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 --2- 172.21.15.99:0/1255435153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 0x560c5437f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 --2- 172.21.15.99:0/1255435153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c54373450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 --2- 172.21.15.99:0/1255435153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c54384240 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 -- 172.21.15.99:0/1255435153 >> 172.21.15.99:0/1255435153 conn(0x560c542e8080 msgr2=0x560c543894c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 -- 172.21.15.99:0/1255435153 shutdown_connections 2023-08-25T09:13:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 -- 172.21.15.99:0/1255435153 wait complete. 2023-08-25T09:13:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 -- start start 2023-08-25T09:13:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.214+0000 7fe36256bb80 1 Processor -- start 2023-08-25T09:13:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c5437c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe36256bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 0x560c5437b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe36256bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5437b990 con 0x560c54378f40 2023-08-25T09:13:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe36256bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5437bb00 con 0x560c54373990 2023-08-25T09:13:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe36256bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5437bcc0 con 0x560c54379860 2023-08-25T09:13:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe36256bb80 10 client.0 _ll_register_callbacks cb 0x7fe33c098fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe347fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c5437c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe34c99b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c5437c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47446/0 (socket says 172.21.15.99:47446) 2023-08-25T09:13:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe34c99b700 1 -- 172.21.15.99:0/2122355054 learned_addr learned my addr 172.21.15.99:0/2122355054 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe347fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37888/0 (socket says 172.21.15.99:37888) 2023-08-25T09:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.215+0000 7fe34d19c700 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 0x560c5437b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe34c99b700 1 -- 172.21.15.99:0/2122355054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 msgr2=0x560c5437b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe34c99b700 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 0x560c5437b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe34c99b700 1 -- 172.21.15.99:0/2122355054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 msgr2=0x560c5437af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe34c99b700 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe34c99b700 1 -- 172.21.15.99:0/2122355054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34810e070 con 0x560c54373990 2023-08-25T09:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe347fff700 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe34c99b700 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c5437c730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe33c098a60 tx=0x7fe33c095550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8765d808c0ff2262 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe327fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe327fff700 1 -- 172.21.15.99:0/2122355054 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c07a060 con 0x560c54373990 2023-08-25T09:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe36256bb80 1 -- 172.21.15.99:0/2122355054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c54366600 con 0x560c54373990 2023-08-25T09:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe327fff700 1 -- 172.21.15.99:0/2122355054 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe33c099210 con 0x560c54373990 2023-08-25T09:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe327fff700 1 -- 172.21.15.99:0/2122355054 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe33c0b0040 con 0x560c54373990 2023-08-25T09:13:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe36256bb80 10 client.26399 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.216+0000 7fe36256bb80 1 -- 172.21.15.99:0/2122355054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560c540dac00 con 0x560c54373990 2023-08-25T09:13:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe327fff700 1 -- 172.21.15.99:0/2122355054 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe33c082080 con 0x560c54373990 2023-08-25T09:13:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe327fff700 1 -- 172.21.15.99:0/2122355054 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe33c095120 con 0x560c54373990 2023-08-25T09:13:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe36256bb80 10 client.26399 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe36256bb80 10 client.26399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe36256bb80 20 client.26399 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560c543e1ae0 0x560c543e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe36256bb80 4 client.26399 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe36256bb80 1 -- 172.21.15.99:0/2122355054 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c542be910 con 0x560c543e1ae0 2023-08-25T09:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.217+0000 7fe347fff700 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560c543e1ae0 0x560c543e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.218+0000 7fe327fff700 10 client.26399 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.218+0000 7fe347fff700 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560c543e1ae0 0x560c543e3ed0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e59874bb61e77c2 server_cookie=8701c16e6c7fcb78 in_seq=0 out_seq=0 2023-08-25T09:13:02.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 35358 31.12 MB/sec warmup 67 sec latency 1849.676 ms 2023-08-25T09:13:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.393+0000 7fe327fff700 1 -- 172.21.15.99:0/2122355054 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c542be910 con 0x560c543e1ae0 2023-08-25T09:13:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.393+0000 7fe327fff700 10 client.26399 handle_command_reply: tid=0 2023-08-25T09:13:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.393+0000 7fe36256bb80 1 client.26399 shutdown 2023-08-25T09:13:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe36256bb80 1 -- 172.21.15.99:0/2122355054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 msgr2=0x560c5437c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c5437c730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe33c098a60 tx=0x7fe33c095550 comp rx=0 tx=0).stop 2023-08-25T09:13:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe36256bb80 1 -- 172.21.15.99:0/2122355054 shutdown_connections 2023-08-25T09:13:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560c54379860 0x560c5437b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560c543e1ae0 0x560c543e3ed0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c54373990 0x560c5437c730 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe36256bb80 1 --2- 172.21.15.99:0/2122355054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c54378f40 0x560c5437af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe36256bb80 1 -- 172.21.15.99:0/2122355054 >> 172.21.15.99:0/2122355054 conn(0x560c542e8080 msgr2=0x560c54377a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.397+0000 7fe327fff700 0 client.26399 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.399+0000 7fe36256bb80 1 -- 172.21.15.99:0/2122355054 shutdown_connections 2023-08-25T09:13:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.401+0000 7fe36256bb80 1 -- 172.21.15.99:0/2122355054 wait complete. 2023-08-25T09:13:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.401+0000 7fe36256bb80 20 client.26399 trim_cache size 0 max 16384 2023-08-25T09:13:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:13:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:13:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:13:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:13:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 -- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe3380a00c0 msgr2=0x7fe3380a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe3380a00c0 0x7fe3380a2580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe340009ec0 tx=0x7fe34000fc40 comp rx=0 tx=0).stop 2023-08-25T09:13:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 -- 172.21.15.99:0/3225108294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 msgr2=0x7fe34819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 0x7fe34819c930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00ec10 tx=0x7fe33c00ef20 comp rx=0 tx=0).stop 2023-08-25T09:13:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 -- 172.21.15.99:0/3225108294 shutdown_connections 2023-08-25T09:13:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe3380a00c0 0x7fe3380a2580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3481068b0 0x7fe348196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3481008a0 0x7fe34819c930 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00ec10 tx=0x7fe33c00ef20 comp rx=0 tx=0).stop 2023-08-25T09:13:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 --2- 172.21.15.99:0/3225108294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe348101260 0x7fe34819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.402+0000 7fe34ebff700 1 -- 172.21.15.99:0/3225108294 >> 172.21.15.99:0/3225108294 conn(0x7fe3480fc370 msgr2=0x7fe3481043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.403+0000 7fe34ebff700 1 -- 172.21.15.99:0/3225108294 shutdown_connections 2023-08-25T09:13:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:02.403+0000 7fe34ebff700 1 -- 172.21.15.99:0/3225108294 wait complete. 2023-08-25T09:13:03.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 35882 31.02 MB/sec warmup 68 sec latency 1986.857 ms 2023-08-25T09:13:03.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:03 smithi181 ceph-mon[131296]: pgmap v434: 97 pgs: 97 active+clean; 494 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 853 op/s 2023-08-25T09:13:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:03 smithi099 ceph-mon[97641]: pgmap v434: 97 pgs: 97 active+clean; 494 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 853 op/s 2023-08-25T09:13:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:03 smithi157 ceph-mon[130879]: pgmap v434: 97 pgs: 97 active+clean; 494 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 853 op/s 2023-08-25T09:13:04.246 INFO:tasks.workunit.client.0.smithi099.stdout: 10 36252 30.80 MB/sec warmup 69 sec latency 238.326 ms 2023-08-25T09:13:04.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:04 smithi181 ceph-mon[131296]: osdmap e104: 12 total, 12 up, 12 in 2023-08-25T09:13:04.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:04 smithi099 ceph-mon[97641]: osdmap e104: 12 total, 12 up, 12 in 2023-08-25T09:13:04.885 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:04 smithi157 ceph-mon[130879]: osdmap e104: 12 total, 12 up, 12 in 2023-08-25T09:13:05.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 36414 30.49 MB/sec warmup 70 sec latency 804.555 ms 2023-08-25T09:13:05.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:05 smithi181 ceph-mon[131296]: pgmap v436: 97 pgs: 97 active+clean; 494 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 661 op/s 2023-08-25T09:13:05.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:05 smithi099 ceph-mon[97641]: pgmap v436: 97 pgs: 97 active+clean; 494 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 661 op/s 2023-08-25T09:13:05.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:05 smithi157 ceph-mon[130879]: pgmap v436: 97 pgs: 97 active+clean; 494 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 661 op/s 2023-08-25T09:13:06.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 36812 30.38 MB/sec warmup 71 sec latency 683.907 ms 2023-08-25T09:13:07.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 37506 30.34 MB/sec warmup 72 sec latency 161.924 ms 2023-08-25T09:13:07.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:07 smithi181 ceph-mon[131296]: pgmap v437: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-08-25T09:13:07.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:07 smithi099 ceph-mon[97641]: pgmap v437: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-08-25T09:13:07.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:07 smithi157 ceph-mon[130879]: pgmap v437: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-08-25T09:13:08.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 38524 30.90 MB/sec warmup 73 sec latency 47.132 ms 2023-08-25T09:13:09.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 39214 30.90 MB/sec warmup 74 sec latency 57.752 ms 2023-08-25T09:13:09.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:09 smithi181 ceph-mon[131296]: pgmap v438: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-08-25T09:13:09.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:09 smithi181 ceph-mon[131296]: osdmap e105: 12 total, 12 up, 12 in 2023-08-25T09:13:09.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:09 smithi099 ceph-mon[97641]: pgmap v438: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-08-25T09:13:09.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:09 smithi099 ceph-mon[97641]: osdmap e105: 12 total, 12 up, 12 in 2023-08-25T09:13:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:09 smithi157 ceph-mon[130879]: pgmap v438: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-08-25T09:13:09.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:09 smithi157 ceph-mon[130879]: osdmap e105: 12 total, 12 up, 12 in 2023-08-25T09:13:10.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 40017 31.02 MB/sec warmup 75 sec latency 76.084 ms 2023-08-25T09:13:11.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 40967 31.38 MB/sec warmup 76 sec latency 57.208 ms 2023-08-25T09:13:11.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:11 smithi181 ceph-mon[131296]: pgmap v440: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 15 MiB/s wr, 641 op/s 2023-08-25T09:13:11.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:11 smithi099 ceph-mon[97641]: pgmap v440: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 15 MiB/s wr, 641 op/s 2023-08-25T09:13:11.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:11 smithi157 ceph-mon[130879]: pgmap v440: 97 pgs: 97 active+clean; 577 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 15 MiB/s wr, 641 op/s 2023-08-25T09:13:12.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 41706 31.47 MB/sec warmup 77 sec latency 87.716 ms 2023-08-25T09:13:13.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 42447 31.53 MB/sec warmup 78 sec latency 65.371 ms 2023-08-25T09:13:13.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:13 smithi181 ceph-mon[131296]: pgmap v441: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2023-08-25T09:13:13.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:13 smithi099 ceph-mon[97641]: pgmap v441: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2023-08-25T09:13:13.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:13 smithi157 ceph-mon[130879]: pgmap v441: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2023-08-25T09:13:14.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 43398 31.66 MB/sec warmup 79 sec latency 49.835 ms 2023-08-25T09:13:15.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 44330 32.02 MB/sec warmup 80 sec latency 53.814 ms 2023-08-25T09:13:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:15 smithi099 ceph-mon[97641]: pgmap v442: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 936 op/s 2023-08-25T09:13:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 -- 172.21.15.99:0/3718533653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2824101280 msgr2=0x7f282410d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 --2- 172.21.15.99:0/3718533653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2824101280 0x7f282410d790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b0a0 tx=0x7f281c00b3b0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 -- 172.21.15.99:0/3718533653 shutdown_connections 2023-08-25T09:13:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 --2- 172.21.15.99:0/3718533653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2824101280 0x7f282410d790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 --2- 172.21.15.99:0/3718533653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28241068d0 0x7f2824106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 --2- 172.21.15.99:0/3718533653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28241008c0 0x7f2824100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 -- 172.21.15.99:0/3718533653 >> 172.21.15.99:0/3718533653 conn(0x7f28240fc3d0 msgr2=0x7f28240fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 -- 172.21.15.99:0/3718533653 shutdown_connections 2023-08-25T09:13:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.590+0000 7f28297ff700 1 -- 172.21.15.99:0/3718533653 wait complete. 2023-08-25T09:13:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.592+0000 7f28297ff700 1 Processor -- start 2023-08-25T09:13:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.592+0000 7f28297ff700 1 -- start start 2023-08-25T09:13:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f28297ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28241008c0 0x7f282419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f28297ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824101280 0x7f282419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f28297ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28241068d0 0x7f2824198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f28297ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2824199150 con 0x7f28241068d0 2023-08-25T09:13:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f28297ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28241992c0 con 0x7f28241008c0 2023-08-25T09:13:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f28297ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2824199480 con 0x7f2824101280 2023-08-25T09:13:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28241008c0 0x7f282419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28241008c0 0x7f282419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50498/0 (socket says 172.21.15.99:50498) 2023-08-25T09:13:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f2822ffd700 1 -- 172.21.15.99:0/3557264447 learned_addr learned my addr 172.21.15.99:0/3557264447 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f28227fc700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824101280 0x7f282419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.593+0000 7f2822ffd700 1 -- 172.21.15.99:0/3557264447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824101280 msgr2=0x7f282419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f2822ffd700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824101280 0x7f282419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f2822ffd700 1 -- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28241068d0 msgr2=0x7f2824198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f2822ffd700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28241068d0 0x7f2824198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f2822ffd700 1 -- 172.21.15.99:0/3557264447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c009d00 con 0x7f28241008c0 2023-08-25T09:13:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f28227fc700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824101280 0x7f282419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:13:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f2822ffd700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28241008c0 0x7f282419ebb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f281400e9a0 tx=0x7f281400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f280bfff700 1 -- 172.21.15.99:0/3557264447 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814009790 con 0x7f28241008c0 2023-08-25T09:13:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f280bfff700 1 -- 172.21.15.99:0/3557264447 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f281400cbd0 con 0x7f28241008c0 2023-08-25T09:13:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f28297ff700 1 -- 172.21.15.99:0/3557264447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2824199700 con 0x7f28241008c0 2023-08-25T09:13:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f280bfff700 1 -- 172.21.15.99:0/3557264447 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814018680 con 0x7f28241008c0 2023-08-25T09:13:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.594+0000 7f28297ff700 1 -- 172.21.15.99:0/3557264447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2824107970 con 0x7f28241008c0 2023-08-25T09:13:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.599+0000 7f280bfff700 1 -- 172.21.15.99:0/3557264447 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f281400cd40 con 0x7f28241008c0 2023-08-25T09:13:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.600+0000 7f283d031b80 1 Processor -- start 2023-08-25T09:13:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.600+0000 7f280bfff700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f280c0a0140 0x7f280c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.600+0000 7f283d031b80 1 -- start start 2023-08-25T09:13:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.600+0000 7f280bfff700 1 -- 172.21.15.99:0/3557264447 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2814014070 con 0x7f28241008c0 2023-08-25T09:13:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f283d031b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f283d031b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f283d031b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f283d031b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa71e750 con 0x55d1aa7bbf40 2023-08-25T09:13:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f283d031b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa70f720 con 0x55d1aa7bc860 2023-08-25T09:13:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f283d031b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa70d040 con 0x55d1aa7b6990 2023-08-25T09:13:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f28237fe700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f280c0a0140 0x7f280c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35952/0 (socket says 172.21.15.99:35952) 2023-08-25T09:13:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f2822ffd700 1 -- 172.21.15.99:0/1508876700 learned_addr learned my addr 172.21.15.99:0/1508876700 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f28227fc700 1 --2- 172.21.15.99:0/1508876700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f2822ffd700 1 -- 172.21.15.99:0/1508876700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 msgr2=0x55d1aa7b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f2822ffd700 1 --2- 172.21.15.99:0/1508876700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f2822ffd700 1 -- 172.21.15.99:0/1508876700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bc860 msgr2=0x55d1aa7b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f2822ffd700 1 --2- 172.21.15.99:0/1508876700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.601+0000 7f2822ffd700 1 -- 172.21.15.99:0/1508876700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1aa5885d0 con 0x55d1aa7bbf40 2023-08-25T09:13:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.602+0000 7f28237fe700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f280c0a0140 0x7f280c0a2600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f281c009610 tx=0x7f281c008df0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.602+0000 7f2822ffd700 1 --2- 172.21.15.99:0/1508876700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bc320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f281400f010 tx=0x7f2814099740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753f04612376a78c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.602+0000 7f28097fa700 1 -- 172.21.15.99:0/1508876700 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281409cdf0 con 0x55d1aa7bbf40 2023-08-25T09:13:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.602+0000 7f28097fa700 1 -- 172.21.15.99:0/1508876700 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28140a0dd0 con 0x55d1aa7bbf40 2023-08-25T09:13:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.602+0000 7f28097fa700 1 -- 172.21.15.99:0/1508876700 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814093bd0 con 0x55d1aa7bbf40 2023-08-25T09:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 -- 172.21.15.99:0/1508876700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bbf40 msgr2=0x55d1aa7bc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 --2- 172.21.15.99:0/1508876700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bc320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f281400f010 tx=0x7f2814099740 comp rx=0 tx=0).stop 2023-08-25T09:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 -- 172.21.15.99:0/1508876700 shutdown_connections 2023-08-25T09:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 --2- 172.21.15.99:0/1508876700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 --2- 172.21.15.99:0/1508876700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bc320 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 --2- 172.21.15.99:0/1508876700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 -- 172.21.15.99:0/1508876700 >> 172.21.15.99:0/1508876700 conn(0x55d1aa72b080 msgr2=0x55d1aa7b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 -- 172.21.15.99:0/1508876700 shutdown_connections 2023-08-25T09:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.603+0000 7f283d031b80 1 -- 172.21.15.99:0/1508876700 wait complete. 2023-08-25T09:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.604+0000 7f283d031b80 1 -- start start 2023-08-25T09:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.604+0000 7f283d031b80 1 Processor -- start 2023-08-25T09:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f283d031b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 0x55d1aa7c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f283d031b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f283d031b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f283d031b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa708930 con 0x55d1aa7bc860 2023-08-25T09:13:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f283d031b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa6a6b50 con 0x55d1aa7bbf40 2023-08-25T09:13:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f283d031b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa6a6850 con 0x55d1aa7b6990 2023-08-25T09:13:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f283d031b80 10 client.0 _ll_register_callbacks cb 0x55d1aa5f0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f28227fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f28227fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50512/0 (socket says 172.21.15.99:50512) 2023-08-25T09:13:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f28227fc700 1 -- 172.21.15.99:0/1694928369 learned_addr learned my addr 172.21.15.99:0/1694928369 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f2822ffd700 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 0x55d1aa7c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f28237fe700 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f28227fc700 1 -- 172.21.15.99:0/1694928369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 msgr2=0x55d1aa7c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.605+0000 7f28227fc700 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 0x55d1aa7c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f28227fc700 1 -- 172.21.15.99:0/1694928369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bc860 msgr2=0x55d1aa7c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f28227fc700 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f28227fc700 1 -- 172.21.15.99:0/1694928369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2824107970 con 0x55d1aa7bbf40 2023-08-25T09:13:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f28237fe700 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:13:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f28227fc700 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f28180126e0 tx=0x7f28180129f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c107fc08e2e8c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f2806ff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:13:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f2806ff5700 1 -- 172.21.15.99:0/1694928369 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818017070 con 0x55d1aa7bbf40 2023-08-25T09:13:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f283d031b80 1 -- 172.21.15.99:0/1694928369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1aa786160 con 0x55d1aa7bbf40 2023-08-25T09:13:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f2806ff5700 1 -- 172.21.15.99:0/1694928369 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2818004240 con 0x55d1aa7bbf40 2023-08-25T09:13:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f2806ff5700 1 -- 172.21.15.99:0/1694928369 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2818004400 con 0x55d1aa7bbf40 2023-08-25T09:13:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f283d031b80 10 client.26411 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.606+0000 7f283d031b80 1 -- 172.21.15.99:0/1694928369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55d1aa534d80 con 0x55d1aa7bbf40 2023-08-25T09:13:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f2806ff5700 1 -- 172.21.15.99:0/1694928369 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2818028080 con 0x55d1aa7bbf40 2023-08-25T09:13:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f2806ff5700 1 -- 172.21.15.99:0/1694928369 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2818011830 con 0x55d1aa7bbf40 2023-08-25T09:13:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f283d031b80 10 client.26411 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f283d031b80 10 client.26411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f283d031b80 20 client.26411 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f283d031b80 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1aa7d0260 0x55d1aa7d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f283d031b80 4 client.26411 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:13:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f283d031b80 1 -- 172.21.15.99:0/1694928369 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1aa6aeb40 con 0x55d1aa7d0260 2023-08-25T09:13:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.607+0000 7f2822ffd700 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1aa7d0260 0x55d1aa7d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.608+0000 7f2806ff5700 10 client.26411 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.608+0000 7f2822ffd700 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1aa7d0260 0x55d1aa7d2680 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f93907c03bfcc5 server_cookie=8cf69d8571e7353a in_seq=0 out_seq=0 2023-08-25T09:13:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.611+0000 7f2806ff5700 1 -- 172.21.15.99:0/1694928369 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d1aa6aeb40 con 0x55d1aa7d0260 2023-08-25T09:13:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.611+0000 7f2806ff5700 10 client.26411 handle_command_reply: tid=0 2023-08-25T09:13:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.611+0000 7f283d031b80 1 client.26411 shutdown 2023-08-25T09:13:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 -- 172.21.15.99:0/1694928369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 msgr2=0x55d1aa7bf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f28180126e0 tx=0x7f28180129f0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 -- 172.21.15.99:0/1694928369 shutdown_connections 2023-08-25T09:13:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7b6990 0x55d1aa7c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1aa7d0260 0x55d1aa7d2680 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 --2- 172.21.15.99:0/1694928369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f2806ff5700 0 client.26411 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 -- 172.21.15.99:0/1694928369 >> 172.21.15.99:0/1694928369 conn(0x55d1aa72b080 msgr2=0x55d1aa7ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 -- 172.21.15.99:0/1694928369 shutdown_connections 2023-08-25T09:13:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 1 -- 172.21.15.99:0/1694928369 wait complete. 2023-08-25T09:13:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.612+0000 7f283d031b80 20 client.26411 trim_cache size 0 max 16384 2023-08-25T09:13:15.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:15 smithi157 ceph-mon[130879]: pgmap v442: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 936 op/s 2023-08-25T09:13:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.629+0000 7f283d031b80 1 Processor -- start 2023-08-25T09:13:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.629+0000 7f283d031b80 1 -- start start 2023-08-25T09:13:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f283d031b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f283d031b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f283d031b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f283d031b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa58a8c0 con 0x55d1aa7b6990 2023-08-25T09:13:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f283d031b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa7c77b0 con 0x55d1aa7bbf40 2023-08-25T09:13:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f283d031b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa7c0680 con 0x55d1aa7bc860 2023-08-25T09:13:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f28227fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f28227fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:50522/0 (socket says 172.21.15.99:50522) 2023-08-25T09:13:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f28227fc700 1 -- 172.21.15.99:0/1813628383 learned_addr learned my addr 172.21.15.99:0/1813628383 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f28237fe700 1 --2- 172.21.15.99:0/1813628383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f28227fc700 1 -- 172.21.15.99:0/1813628383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 msgr2=0x55d1aa7c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.630+0000 7f28227fc700 1 --2- 172.21.15.99:0/1813628383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.631+0000 7f28227fc700 1 -- 172.21.15.99:0/1813628383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 msgr2=0x55d1aa7b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.631+0000 7f28227fc700 1 --2- 172.21.15.99:0/1813628383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.631+0000 7f28227fc700 1 -- 172.21.15.99:0/1813628383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1aa51dc00 con 0x55d1aa7bbf40 2023-08-25T09:13:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.631+0000 7f28227fc700 1 --2- 172.21.15.99:0/1813628383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f28180126b0 tx=0x7f2818012e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2edb81e3df24d94 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.631+0000 7f2809ffb700 1 -- 172.21.15.99:0/1813628383 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818017040 con 0x55d1aa7bbf40 2023-08-25T09:13:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.631+0000 7f2809ffb700 1 -- 172.21.15.99:0/1813628383 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1aa786160 con 0x55d1aa7bbf40 2023-08-25T09:13:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.631+0000 7f2809ffb700 1 -- 172.21.15.99:0/1813628383 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180038e0 con 0x55d1aa7bbf40 2023-08-25T09:13:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 -- 172.21.15.99:0/1813628383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 msgr2=0x55d1aa7bf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 --2- 172.21.15.99:0/1813628383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f28180126b0 tx=0x7f2818012e80 comp rx=0 tx=0).stop 2023-08-25T09:13:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 -- 172.21.15.99:0/1813628383 shutdown_connections 2023-08-25T09:13:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 --2- 172.21.15.99:0/1813628383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 --2- 172.21.15.99:0/1813628383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 --2- 172.21.15.99:0/1813628383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7bf900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 -- 172.21.15.99:0/1813628383 >> 172.21.15.99:0/1813628383 conn(0x55d1aa72b080 msgr2=0x55d1aa7d6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 -- 172.21.15.99:0/1813628383 shutdown_connections 2023-08-25T09:13:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 -- 172.21.15.99:0/1813628383 wait complete. 2023-08-25T09:13:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.632+0000 7f283d031b80 1 -- start start 2023-08-25T09:13:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.633+0000 7f283d031b80 1 Processor -- start 2023-08-25T09:13:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.633+0000 7f283d031b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.633+0000 7f283d031b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.633+0000 7f283d031b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.633+0000 7f283d031b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa7b9600 con 0x55d1aa7b6990 2023-08-25T09:13:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.633+0000 7f283d031b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa7b7e90 con 0x55d1aa7bbf40 2023-08-25T09:13:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.633+0000 7f283d031b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1aa7b8050 con 0x55d1aa7bc860 2023-08-25T09:13:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.633+0000 7f283d031b80 10 client.0 _ll_register_callbacks cb 0x7f2818009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28227fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:43992/0 (socket says 172.21.15.99:43992) 2023-08-25T09:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 -- 172.21.15.99:0/263536845 learned_addr learned my addr 172.21.15.99:0/263536845 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 -- 172.21.15.99:0/263536845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 msgr2=0x55d1aa7b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7b8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 -- 172.21.15.99:0/263536845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 msgr2=0x55d1aa7b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 -- 172.21.15.99:0/263536845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1aa534d80 con 0x55d1aa7bc860 2023-08-25T09:13:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28227fc700 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:13:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f2822ffd700 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:13:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f28237fe700 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b7950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b0a0 tx=0x7f281c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d400e04abb0aa0a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.634+0000 7f2806ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:13:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f2806ff5700 1 -- 172.21.15.99:0/263536845 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c00e040 con 0x55d1aa7bc860 2023-08-25T09:13:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 1 -- 172.21.15.99:0/263536845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1aa7a9600 con 0x55d1aa7bc860 2023-08-25T09:13:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f2806ff5700 1 -- 172.21.15.99:0/263536845 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f281c026040 con 0x55d1aa7bc860 2023-08-25T09:13:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f2806ff5700 1 -- 172.21.15.99:0/263536845 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f281c01d480 con 0x55d1aa7bc860 2023-08-25T09:13:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 10 client.26401 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 1 -- 172.21.15.99:0/263536845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55d1aa55b1f0 con 0x55d1aa7bc860 2023-08-25T09:13:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f2806ff5700 1 -- 172.21.15.99:0/263536845 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1aa7a9600 con 0x55d1aa7bc860 2023-08-25T09:13:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f2806ff5700 1 -- 172.21.15.99:0/263536845 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f281c005280 con 0x55d1aa7bc860 2023-08-25T09:13:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 10 client.26401 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 10 client.26401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 20 client.26401 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1aa824e50 0x55d1aa827270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 4 client.26401 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:13:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.635+0000 7f283d031b80 1 -- 172.21.15.99:0/263536845 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d1aa701910 con 0x55d1aa824e50 2023-08-25T09:13:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.636+0000 7f2822ffd700 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1aa824e50 0x55d1aa827270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.636+0000 7f2822ffd700 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1aa824e50 0x55d1aa827270 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56888736e0dd68 server_cookie=d655d2315f618327 in_seq=0 out_seq=0 2023-08-25T09:13:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.636+0000 7f2806ff5700 10 client.26401 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:15.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:15 smithi181 ceph-mon[131296]: pgmap v442: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 936 op/s 2023-08-25T09:13:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f2806ff5700 1 -- 172.21.15.99:0/263536845 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d1aa701910 con 0x55d1aa824e50 2023-08-25T09:13:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f2806ff5700 10 client.26401 handle_command_reply: tid=0 2023-08-25T09:13:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 client.26401 shutdown 2023-08-25T09:13:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 -- 172.21.15.99:0/263536845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 msgr2=0x55d1aa7b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b7950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b0a0 tx=0x7f281c0099e0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 -- 172.21.15.99:0/263536845 shutdown_connections 2023-08-25T09:13:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aa7bc860 0x55d1aa7b7950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1aa7b6990 0x55d1aa7b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55d1aa824e50 0x55d1aa827270 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 --2- 172.21.15.99:0/263536845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1aa7bbf40 0x55d1aa7b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 -- 172.21.15.99:0/263536845 >> 172.21.15.99:0/263536845 conn(0x55d1aa72b080 msgr2=0x55d1aa7b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f2806ff5700 0 client.26401 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 -- 172.21.15.99:0/263536845 shutdown_connections 2023-08-25T09:13:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.883+0000 7f283d031b80 1 -- 172.21.15.99:0/263536845 wait complete. 2023-08-25T09:13:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.884+0000 7f283d031b80 20 client.26401 trim_cache size 0 max 16384 2023-08-25T09:13:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:13:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:13:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:13:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:13:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 -- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f280c0a0140 msgr2=0x7f280c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f280c0a0140 0x7f280c0a2600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f281c009610 tx=0x7f281c008df0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 -- 172.21.15.99:0/3557264447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28241008c0 msgr2=0x7f282419ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28241008c0 0x7f282419ebb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f281400e9a0 tx=0x7f281400ed60 comp rx=0 tx=0).stop 2023-08-25T09:13:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 -- 172.21.15.99:0/3557264447 shutdown_connections 2023-08-25T09:13:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f280c0a0140 0x7f280c0a2600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28241068d0 0x7f2824198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28241008c0 0x7f282419ebb0 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f281400e9a0 tx=0x7f281400ed60 comp rx=0 tx=0).stop 2023-08-25T09:13:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 --2- 172.21.15.99:0/3557264447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824101280 0x7f282419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 -- 172.21.15.99:0/3557264447 >> 172.21.15.99:0/3557264447 conn(0x7f28240fc3d0 msgr2=0x7f282410bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.886+0000 7f28297ff700 1 -- 172.21.15.99:0/3557264447 shutdown_connections 2023-08-25T09:13:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:15.887+0000 7f28297ff700 1 -- 172.21.15.99:0/3557264447 wait complete. 2023-08-25T09:13:16.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 44948 32.15 MB/sec warmup 81 sec latency 147.246 ms 2023-08-25T09:13:17.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 45594 32.13 MB/sec warmup 82 sec latency 36.825 ms 2023-08-25T09:13:17.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:17 smithi181 ceph-mon[131296]: pgmap v443: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 827 op/s 2023-08-25T09:13:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:17 smithi099 ceph-mon[97641]: pgmap v443: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 827 op/s 2023-08-25T09:13:17.884 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:17 smithi157 ceph-mon[130879]: pgmap v443: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 827 op/s 2023-08-25T09:13:18.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 46280 32.10 MB/sec warmup 83 sec latency 83.630 ms 2023-08-25T09:13:19.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 47178 32.34 MB/sec warmup 84 sec latency 50.179 ms 2023-08-25T09:13:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:19 smithi181 ceph-mon[131296]: pgmap v444: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 827 op/s 2023-08-25T09:13:19.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:19 smithi099 ceph-mon[97641]: pgmap v444: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 827 op/s 2023-08-25T09:13:19.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:19 smithi157 ceph-mon[130879]: pgmap v444: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 827 op/s 2023-08-25T09:13:20.247 INFO:tasks.workunit.client.0.smithi099.stdout: 10 48104 32.42 MB/sec warmup 85 sec latency 76.701 ms 2023-08-25T09:13:21.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 48970 32.74 MB/sec warmup 86 sec latency 51.316 ms 2023-08-25T09:13:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:21 smithi181 ceph-mon[131296]: pgmap v445: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 5.9 MiB/s wr, 745 op/s 2023-08-25T09:13:21.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:21 smithi099 ceph-mon[97641]: pgmap v445: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 5.9 MiB/s wr, 745 op/s 2023-08-25T09:13:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:21 smithi157 ceph-mon[130879]: pgmap v445: 97 pgs: 97 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 5.9 MiB/s wr, 745 op/s 2023-08-25T09:13:22.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 49787 32.87 MB/sec warmup 87 sec latency 33.053 ms 2023-08-25T09:13:23.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 50558 32.98 MB/sec warmup 88 sec latency 53.323 ms 2023-08-25T09:13:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:23 smithi181 ceph-mon[131296]: pgmap v446: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 925 op/s 2023-08-25T09:13:23.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:23 smithi099 ceph-mon[97641]: pgmap v446: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 925 op/s 2023-08-25T09:13:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:23 smithi157 ceph-mon[130879]: pgmap v446: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 925 op/s 2023-08-25T09:13:24.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 51285 32.95 MB/sec warmup 89 sec latency 50.891 ms 2023-08-25T09:13:25.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 52309 33.15 MB/sec warmup 90 sec latency 42.514 ms 2023-08-25T09:13:25.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:25 smithi099 ceph-mon[97641]: pgmap v447: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2023-08-25T09:13:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:25 smithi157 ceph-mon[130879]: pgmap v447: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2023-08-25T09:13:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:25 smithi181 ceph-mon[131296]: pgmap v447: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2023-08-25T09:13:26.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 53180 33.41 MB/sec warmup 91 sec latency 159.187 ms 2023-08-25T09:13:27.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 53752 33.41 MB/sec warmup 92 sec latency 301.658 ms 2023-08-25T09:13:27.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:27 smithi181 ceph-mon[131296]: pgmap v448: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 789 op/s 2023-08-25T09:13:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:27 smithi099 ceph-mon[97641]: pgmap v448: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 789 op/s 2023-08-25T09:13:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:27 smithi157 ceph-mon[130879]: pgmap v448: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 789 op/s 2023-08-25T09:13:28.248 INFO:tasks.workunit.client.0.smithi099.stdout: 10 54355 33.37 MB/sec warmup 93 sec latency 357.422 ms 2023-08-25T09:13:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 -- 172.21.15.99:0/3224394052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9378101270 msgr2=0x7f937810d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 --2- 172.21.15.99:0/3224394052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9378101270 0x7f937810d780 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9370009f90 tx=0x7f937000b330 comp rx=0 tx=0).stop 2023-08-25T09:13:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 -- 172.21.15.99:0/3224394052 shutdown_connections 2023-08-25T09:13:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 --2- 172.21.15.99:0/3224394052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9378101270 0x7f937810d780 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 --2- 172.21.15.99:0/3224394052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93781068c0 0x7f9378106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 --2- 172.21.15.99:0/3224394052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93781008b0 0x7f9378100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 -- 172.21.15.99:0/3224394052 >> 172.21.15.99:0/3224394052 conn(0x7f93780fc380 msgr2=0x7f93780fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 -- 172.21.15.99:0/3224394052 shutdown_connections 2023-08-25T09:13:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.081+0000 7f937edff700 1 -- 172.21.15.99:0/3224394052 wait complete. 2023-08-25T09:13:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.082+0000 7f937edff700 1 Processor -- start 2023-08-25T09:13:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.082+0000 7f937edff700 1 -- start start 2023-08-25T09:13:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.082+0000 7f937edff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93781008b0 0x7f937819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.082+0000 7f937edff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9378101270 0x7f937819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937edff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93781068c0 0x7f93781989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937edff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9378199020 con 0x7f93781008b0 2023-08-25T09:13:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937edff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9378199190 con 0x7f9378101270 2023-08-25T09:13:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937edff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9378199350 con 0x7f93781068c0 2023-08-25T09:13:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93781008b0 0x7f937819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93781008b0 0x7f937819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57062/0 (socket says 172.21.15.99:57062) 2023-08-25T09:13:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937cb9b700 1 -- 172.21.15.99:0/597843891 learned_addr learned my addr 172.21.15.99:0/597843891 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f9377fff700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9378101270 0x7f937819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937d39c700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93781068c0 0x7f93781989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937cb9b700 1 -- 172.21.15.99:0/597843891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93781068c0 msgr2=0x7f93781989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937cb9b700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93781068c0 0x7f93781989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937cb9b700 1 -- 172.21.15.99:0/597843891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9378101270 msgr2=0x7f937819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937cb9b700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9378101270 0x7f937819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f937cb9b700 1 -- 172.21.15.99:0/597843891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370009c40 con 0x7f93781008b0 2023-08-25T09:13:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.083+0000 7f9377fff700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9378101270 0x7f937819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:13:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.084+0000 7f937cb9b700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93781008b0 0x7f937819eb50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f936c00d980 tx=0x7f936c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.084+0000 7f9375ffb700 1 -- 172.21.15.99:0/597843891 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c00f890 con 0x7f93781008b0 2023-08-25T09:13:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.084+0000 7f937edff700 1 -- 172.21.15.99:0/597843891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9378199630 con 0x7f93781008b0 2023-08-25T09:13:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.084+0000 7f9375ffb700 1 -- 172.21.15.99:0/597843891 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f936c010460 con 0x7f93781008b0 2023-08-25T09:13:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.084+0000 7f9375ffb700 1 -- 172.21.15.99:0/597843891 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c009600 con 0x7f93781008b0 2023-08-25T09:13:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.084+0000 7f937edff700 1 -- 172.21.15.99:0/597843891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93781a5b40 con 0x7f93781008b0 2023-08-25T09:13:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.088+0000 7f9375ffb700 1 -- 172.21.15.99:0/597843891 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f936c0105d0 con 0x7f93781008b0 2023-08-25T09:13:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.089+0000 7f9392644b80 1 Processor -- start 2023-08-25T09:13:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.089+0000 7f9375ffb700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93680a00c0 0x7f93680a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9375ffb700 1 -- 172.21.15.99:0/597843891 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f936c0cb390 con 0x7f93781008b0 2023-08-25T09:13:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9392644b80 1 -- start start 2023-08-25T09:13:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f937d39c700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93680a00c0 0x7f93680a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9392644b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x55556255e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9392644b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562558450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9392644b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562558dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9392644b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555624c0750 con 0x55556255df40 2023-08-25T09:13:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9392644b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555624b1720 con 0x555562558990 2023-08-25T09:13:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9392644b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555624af040 con 0x55556255e860 2023-08-25T09:13:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f937d39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562558dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f9377fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562558450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f937d39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562558dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:55086/0 (socket says 172.21.15.99:55086) 2023-08-25T09:13:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x55556255e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.090+0000 7f937d39c700 1 -- 172.21.15.99:0/3047879293 learned_addr learned my addr 172.21.15.99:0/3047879293 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f937cb9b700 1 -- 172.21.15.99:0/3047879293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 msgr2=0x555562558450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f937cb9b700 1 --2- 172.21.15.99:0/3047879293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562558450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f937cb9b700 1 -- 172.21.15.99:0/3047879293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 msgr2=0x555562558dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f937cb9b700 1 --2- 172.21.15.99:0/3047879293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562558dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f937cb9b700 1 -- 172.21.15.99:0/3047879293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55556232a5d0 con 0x55556255df40 2023-08-25T09:13:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f937d39c700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93680a00c0 0x7f93680a2580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9370009f90 tx=0x7f9370004b70 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f937cb9b700 1 --2- 172.21.15.99:0/3047879293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x55556255e320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f936c0c8a70 tx=0x7f936c098410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb72b421c15e9e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f9362ffd700 1 -- 172.21.15.99:0/3047879293 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c098a30 con 0x55556255df40 2023-08-25T09:13:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f9362ffd700 1 -- 172.21.15.99:0/3047879293 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f936c09c860 con 0x55556255df40 2023-08-25T09:13:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.091+0000 7f9362ffd700 1 -- 172.21.15.99:0/3047879293 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c097f20 con 0x55556255df40 2023-08-25T09:13:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.092+0000 7f9392644b80 1 -- 172.21.15.99:0/3047879293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 msgr2=0x55556255e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.092+0000 7f9392644b80 1 --2- 172.21.15.99:0/3047879293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x55556255e320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f936c0c8a70 tx=0x7f936c098410 comp rx=0 tx=0).stop 2023-08-25T09:13:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.092+0000 7f9392644b80 1 -- 172.21.15.99:0/3047879293 shutdown_connections 2023-08-25T09:13:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.092+0000 7f9392644b80 1 --2- 172.21.15.99:0/3047879293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562558dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.092+0000 7f9392644b80 1 --2- 172.21.15.99:0/3047879293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x55556255e320 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.092+0000 7f9392644b80 1 --2- 172.21.15.99:0/3047879293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562558450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.092+0000 7f9392644b80 1 -- 172.21.15.99:0/3047879293 >> 172.21.15.99:0/3047879293 conn(0x5555624cd080 msgr2=0x555562552d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.093+0000 7f9392644b80 1 -- 172.21.15.99:0/3047879293 shutdown_connections 2023-08-25T09:13:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.093+0000 7f9392644b80 1 -- 172.21.15.99:0/3047879293 wait complete. 2023-08-25T09:13:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.093+0000 7f9392644b80 1 -- start start 2023-08-25T09:13:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.093+0000 7f9392644b80 1 Processor -- start 2023-08-25T09:13:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f9392644b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562564700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f9392644b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x555562561910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f9392644b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562562000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f9392644b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555624aa930 con 0x55556255df40 2023-08-25T09:13:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f9392644b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562448b50 con 0x555562558990 2023-08-25T09:13:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f9392644b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562448850 con 0x55556255e860 2023-08-25T09:13:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f9392644b80 10 client.0 _ll_register_callbacks cb 0x555562392a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562564700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562564700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:55092/0 (socket says 172.21.15.99:55092) 2023-08-25T09:13:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f937cb9b700 1 -- 172.21.15.99:0/4004531407 learned_addr learned my addr 172.21.15.99:0/4004531407 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f9377fff700 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x555562561910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.094+0000 7f937d39c700 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562562000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9377fff700 1 -- 172.21.15.99:0/4004531407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 msgr2=0x555562562000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9377fff700 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562562000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9377fff700 1 -- 172.21.15.99:0/4004531407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 msgr2=0x555562564700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9377fff700 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562564700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9377fff700 1 -- 172.21.15.99:0/4004531407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93781a5b40 con 0x55556255df40 2023-08-25T09:13:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f937d39c700 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562562000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:13:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9377fff700 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x555562561910 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9364012cd0 tx=0x7f9364009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaf857190090bce server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9357fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:13:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9357fff700 1 -- 172.21.15.99:0/4004531407 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364017070 con 0x55556255df40 2023-08-25T09:13:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.095+0000 7f9357fff700 1 -- 172.21.15.99:0/4004531407 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9364004050 con 0x55556255df40 2023-08-25T09:13:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.096+0000 7f9392644b80 1 -- 172.21.15.99:0/4004531407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555562528160 con 0x55556255df40 2023-08-25T09:13:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.096+0000 7f9357fff700 1 -- 172.21.15.99:0/4004531407 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9364004210 con 0x55556255df40 2023-08-25T09:13:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.096+0000 7f9392644b80 10 client.16353 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.096+0000 7f9392644b80 1 -- 172.21.15.99:0/4004531407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5555622d6d80 con 0x55556255df40 2023-08-25T09:13:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.096+0000 7f9357fff700 1 -- 172.21.15.99:0/4004531407 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555562528160 con 0x55556255df40 2023-08-25T09:13:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f9357fff700 1 -- 172.21.15.99:0/4004531407 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9364003810 con 0x55556255df40 2023-08-25T09:13:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f9392644b80 10 client.16353 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f9392644b80 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f9392644b80 20 client.16353 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f9392644b80 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x555562568d80 0x55556256b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f9392644b80 4 client.16353 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:13:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f9392644b80 1 -- 172.21.15.99:0/4004531407 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555562450b40 con 0x555562568d80 2023-08-25T09:13:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f937cb9b700 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x555562568d80 0x55556256b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f937cb9b700 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x555562568d80 0x55556256b1a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83c90ddffa6e877 server_cookie=19790cfccea393ea in_seq=0 out_seq=0 2023-08-25T09:13:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.097+0000 7f9357fff700 10 client.16353 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.098+0000 7f9357fff700 1 -- 172.21.15.99:0/4004531407 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555562450b40 con 0x555562568d80 2023-08-25T09:13:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.098+0000 7f9357fff700 10 client.16353 handle_command_reply: tid=0 2023-08-25T09:13:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 client.16353 shutdown 2023-08-25T09:13:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 -- 172.21.15.99:0/4004531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 msgr2=0x555562561910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x555562561910 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9364012cd0 tx=0x7f9364009d40 comp rx=0 tx=0).stop 2023-08-25T09:13:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 -- 172.21.15.99:0/4004531407 shutdown_connections 2023-08-25T09:13:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255e860 0x555562562000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x555562568d80 0x55556256b1a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555562558990 0x555562564700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 --2- 172.21.15.99:0/4004531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55556255df40 0x555562561910 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9357fff700 0 client.16353 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 -- 172.21.15.99:0/4004531407 >> 172.21.15.99:0/4004531407 conn(0x5555624cd080 msgr2=0x55556254d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 -- 172.21.15.99:0/4004531407 shutdown_connections 2023-08-25T09:13:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 1 -- 172.21.15.99:0/4004531407 wait complete. 2023-08-25T09:13:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.099+0000 7f9392644b80 20 client.16353 trim_cache size 0 max 16384 2023-08-25T09:13:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.117+0000 7f9392644b80 1 Processor -- start 2023-08-25T09:13:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.117+0000 7f9392644b80 1 -- start start 2023-08-25T09:13:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.117+0000 7f9392644b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.117+0000 7f9392644b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x555562564230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.117+0000 7f9392644b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556256cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.117+0000 7f9392644b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556232c8c0 con 0x555562558990 2023-08-25T09:13:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.117+0000 7f9392644b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556254f990 con 0x55556255e860 2023-08-25T09:13:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.117+0000 7f9392644b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562553840 con 0x55556255df40 2023-08-25T09:13:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f9377fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x555562564230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937d39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556256cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57086/0 (socket says 172.21.15.99:57086) 2023-08-25T09:13:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f9377fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x555562564230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:59366/0 (socket says 172.21.15.99:59366) 2023-08-25T09:13:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 -- 172.21.15.99:0/1582969228 learned_addr learned my addr 172.21.15.99:0/1582969228 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 -- 172.21.15.99:0/1582969228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 msgr2=0x555562564230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 --2- 172.21.15.99:0/1582969228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x555562564230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 -- 172.21.15.99:0/1582969228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 msgr2=0x55556256cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 --2- 172.21.15.99:0/1582969228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556256cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 -- 172.21.15.99:0/1582969228 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555622bfc00 con 0x555562558990 2023-08-25T09:13:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.118+0000 7f937cb9b700 1 --2- 172.21.15.99:0/1582969228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625719f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f936c0989c0 tx=0x7f936c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0483d632c64584 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f93637fe700 1 -- 172.21.15.99:0/1582969228 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c098ab0 con 0x555562558990 2023-08-25T09:13:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f93637fe700 1 -- 172.21.15.99:0/1582969228 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f936c0a0840 con 0x555562558990 2023-08-25T09:13:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f93637fe700 1 -- 172.21.15.99:0/1582969228 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0981f0 con 0x555562558990 2023-08-25T09:13:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f9392644b80 1 -- 172.21.15.99:0/1582969228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 msgr2=0x5555625719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f9392644b80 1 --2- 172.21.15.99:0/1582969228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625719f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f936c0989c0 tx=0x7f936c000bc0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f9392644b80 1 -- 172.21.15.99:0/1582969228 shutdown_connections 2023-08-25T09:13:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f9392644b80 1 --2- 172.21.15.99:0/1582969228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556256cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f9392644b80 1 --2- 172.21.15.99:0/1582969228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625719f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f9392644b80 1 --2- 172.21.15.99:0/1582969228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x555562564230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.119+0000 7f9392644b80 1 -- 172.21.15.99:0/1582969228 >> 172.21.15.99:0/1582969228 conn(0x5555624cd080 msgr2=0x55556255baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.120+0000 7f9392644b80 1 -- 172.21.15.99:0/1582969228 shutdown_connections 2023-08-25T09:13:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.120+0000 7f9392644b80 1 -- 172.21.15.99:0/1582969228 wait complete. 2023-08-25T09:13:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.120+0000 7f9392644b80 1 -- start start 2023-08-25T09:13:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.120+0000 7f9392644b80 1 Processor -- start 2023-08-25T09:13:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f9392644b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f9392644b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x55556255a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f9392644b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556255add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f9392644b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562561bd0 con 0x555562558990 2023-08-25T09:13:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f9392644b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556255b310 con 0x55556255e860 2023-08-25T09:13:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f9392644b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556255b4d0 con 0x55556255df40 2023-08-25T09:13:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f9392644b80 10 client.0 _ll_register_callbacks cb 0x7f93640127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f937cb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57096/0 (socket says 172.21.15.99:57096) 2023-08-25T09:13:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f937cb9b700 1 -- 172.21.15.99:0/2081531218 learned_addr learned my addr 172.21.15.99:0/2081531218 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.121+0000 7f937d39c700 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556255add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f9377fff700 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x55556255a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f937cb9b700 1 -- 172.21.15.99:0/2081531218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 msgr2=0x55556255a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f937cb9b700 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x55556255a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f937cb9b700 1 -- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 msgr2=0x55556255add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f937cb9b700 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556255add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f937cb9b700 1 -- 172.21.15.99:0/2081531218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55556232a5d0 con 0x555562558990 2023-08-25T09:13:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f937d39c700 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556255add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:13:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f937cb9b700 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625614f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f936c0c8ff0 tx=0x7f936c098930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3e5627da1ed2fe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f9357fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:13:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f9357fff700 1 -- 172.21.15.99:0/2081531218 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c098030 con 0x555562558990 2023-08-25T09:13:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f9357fff700 1 -- 172.21.15.99:0/2081531218 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f936c0ae040 con 0x555562558990 2023-08-25T09:13:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.122+0000 7f9392644b80 1 -- 172.21.15.99:0/2081531218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55556254b600 con 0x555562558990 2023-08-25T09:13:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9357fff700 1 -- 172.21.15.99:0/2081531218 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f936c098190 con 0x555562558990 2023-08-25T09:13:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9392644b80 10 client.16359 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9392644b80 1 -- 172.21.15.99:0/2081531218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5555622fd1f0 con 0x555562558990 2023-08-25T09:13:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9357fff700 1 -- 172.21.15.99:0/2081531218 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f936c07b080 con 0x555562558990 2023-08-25T09:13:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9357fff700 1 -- 172.21.15.99:0/2081531218 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f936c064020 con 0x555562558990 2023-08-25T09:13:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9392644b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9392644b80 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9392644b80 20 client.16359 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9392644b80 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5555625d7b80 0x5555625d9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9392644b80 4 client.16359 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:13:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.123+0000 7f9392644b80 1 -- 172.21.15.99:0/2081531218 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555624a3910 con 0x5555625d7b80 2023-08-25T09:13:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.124+0000 7f9377fff700 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5555625d7b80 0x5555625d9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.124+0000 7f9377fff700 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5555625d7b80 0x5555625d9f70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982b75ca39d913ab server_cookie=98cc2c11259419d6 in_seq=0 out_seq=0 2023-08-25T09:13:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.124+0000 7f9357fff700 10 client.16359 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:29.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 55143 33.45 MB/sec warmup 94 sec latency 116.312 ms 2023-08-25T09:13:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.350+0000 7f9357fff700 1 -- 172.21.15.99:0/2081531218 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555624a3910 con 0x5555625d7b80 2023-08-25T09:13:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.350+0000 7f9357fff700 10 client.16359 handle_command_reply: tid=0 2023-08-25T09:13:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.350+0000 7f9392644b80 1 client.16359 shutdown 2023-08-25T09:13:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.350+0000 7f9392644b80 1 -- 172.21.15.99:0/2081531218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 msgr2=0x5555625614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.350+0000 7f9392644b80 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625614f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f936c0c8ff0 tx=0x7f936c098930 comp rx=0 tx=0).stop 2023-08-25T09:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9392644b80 1 -- 172.21.15.99:0/2081531218 shutdown_connections 2023-08-25T09:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9392644b80 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5555625d7b80 0x5555625d9f70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9392644b80 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55556255e860 0x55556255add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9392644b80 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555562558990 0x5555625614f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9392644b80 1 --2- 172.21.15.99:0/2081531218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55556255df40 0x55556255a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9392644b80 1 -- 172.21.15.99:0/2081531218 >> 172.21.15.99:0/2081531218 conn(0x5555624cd080 msgr2=0x55556255ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9357fff700 0 client.16359 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9392644b80 1 -- 172.21.15.99:0/2081531218 shutdown_connections 2023-08-25T09:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.351+0000 7f9392644b80 1 -- 172.21.15.99:0/2081531218 wait complete. 2023-08-25T09:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f9392644b80 20 client.16359 trim_cache size 0 max 16384 2023-08-25T09:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:13:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 -- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93680a00c0 msgr2=0x7f93680a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93680a00c0 0x7f93680a2580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9370009f90 tx=0x7f9370004b70 comp rx=0 tx=0).stop 2023-08-25T09:13:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 -- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93781008b0 msgr2=0x7f937819eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93781008b0 0x7f937819eb50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f936c00d980 tx=0x7f936c00dc90 comp rx=0 tx=0).stop 2023-08-25T09:13:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 -- 172.21.15.99:0/597843891 shutdown_connections 2023-08-25T09:13:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f93680a00c0 0x7f93680a2580 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93781068c0 0x7f93781989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93781008b0 0x7f937819eb50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 --2- 172.21.15.99:0/597843891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9378101270 0x7f937819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.352+0000 7f937edff700 1 -- 172.21.15.99:0/597843891 >> 172.21.15.99:0/597843891 conn(0x7f93780fc380 msgr2=0x7f937810bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.353+0000 7f937edff700 1 -- 172.21.15.99:0/597843891 shutdown_connections 2023-08-25T09:13:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:29.353+0000 7f937edff700 1 -- 172.21.15.99:0/597843891 wait complete. 2023-08-25T09:13:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:29 smithi181 ceph-mon[131296]: pgmap v449: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-08-25T09:13:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:29 smithi099 ceph-mon[97641]: pgmap v449: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-08-25T09:13:29.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:29 smithi157 ceph-mon[130879]: pgmap v449: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-08-25T09:13:30.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 56006 33.56 MB/sec warmup 95 sec latency 174.591 ms 2023-08-25T09:13:31.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 56363 33.46 MB/sec warmup 96 sec latency 555.374 ms 2023-08-25T09:13:31.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:31 smithi181 ceph-mon[131296]: pgmap v450: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-08-25T09:13:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:31 smithi099 ceph-mon[97641]: pgmap v450: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-08-25T09:13:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:31 smithi157 ceph-mon[130879]: pgmap v450: 97 pgs: 97 active+clean; 612 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-08-25T09:13:32.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 56686 33.25 MB/sec warmup 97 sec latency 584.736 ms 2023-08-25T09:13:33.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 57338 33.31 MB/sec warmup 98 sec latency 138.030 ms 2023-08-25T09:13:33.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:33 smithi181 ceph-mon[131296]: pgmap v451: 97 pgs: 97 active+clean; 712 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 893 op/s 2023-08-25T09:13:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:33 smithi099 ceph-mon[97641]: pgmap v451: 97 pgs: 97 active+clean; 712 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 893 op/s 2023-08-25T09:13:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:33 smithi157 ceph-mon[130879]: pgmap v451: 97 pgs: 97 active+clean; 712 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 893 op/s 2023-08-25T09:13:34.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 58155 33.39 MB/sec warmup 99 sec latency 41.695 ms 2023-08-25T09:13:35.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 59124 33.67 MB/sec warmup 100 sec latency 41.375 ms 2023-08-25T09:13:35.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:35 smithi099 ceph-mon[97641]: pgmap v452: 97 pgs: 97 active+clean; 712 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 658 op/s 2023-08-25T09:13:35.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:35 smithi157 ceph-mon[130879]: pgmap v452: 97 pgs: 97 active+clean; 712 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 658 op/s 2023-08-25T09:13:35.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:35 smithi181 ceph-mon[131296]: pgmap v452: 97 pgs: 97 active+clean; 712 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 658 op/s 2023-08-25T09:13:36.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 59761 33.65 MB/sec warmup 101 sec latency 40.187 ms 2023-08-25T09:13:37.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 59954 33.38 MB/sec warmup 102 sec latency 326.312 ms 2023-08-25T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:37 smithi181 ceph-mon[131296]: pgmap v453: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 16 MiB/s wr, 987 op/s 2023-08-25T09:13:37.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:37 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:13:37.819 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:37 smithi099 ceph-mon[97641]: pgmap v453: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 16 MiB/s wr, 987 op/s 2023-08-25T09:13:37.819 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:37 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:13:37.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:37 smithi157 ceph-mon[130879]: pgmap v453: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 16 MiB/s wr, 987 op/s 2023-08-25T09:13:37.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:37 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:13:38.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 60421 33.25 MB/sec warmup 103 sec latency 104.279 ms 2023-08-25T09:13:38.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:38 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:13:38.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:38 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:13:38.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:38 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:13:38.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:38 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:13:38.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:38 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:13:38.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:38 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:13:38.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:38 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:13:38.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:38 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:13:38.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:38 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:13:39.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 61447 33.51 MB/sec warmup 104 sec latency 58.514 ms 2023-08-25T09:13:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:39 smithi181 ceph-mon[131296]: pgmap v454: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 770 op/s 2023-08-25T09:13:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:39 smithi099 ceph-mon[97641]: pgmap v454: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 770 op/s 2023-08-25T09:13:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:39 smithi157 ceph-mon[130879]: pgmap v454: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 770 op/s 2023-08-25T09:13:40.250 INFO:tasks.workunit.client.0.smithi099.stdout: 10 62374 33.66 MB/sec warmup 105 sec latency 40.831 ms 2023-08-25T09:13:41.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 63153 33.76 MB/sec warmup 106 sec latency 63.765 ms 2023-08-25T09:13:41.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:41 smithi181 ceph-mon[131296]: pgmap v455: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 770 op/s 2023-08-25T09:13:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:41 smithi099 ceph-mon[97641]: pgmap v455: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 770 op/s 2023-08-25T09:13:41.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:41 smithi157 ceph-mon[130879]: pgmap v455: 97 pgs: 97 active+clean; 641 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 770 op/s 2023-08-25T09:13:42.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 63959 33.82 MB/sec warmup 107 sec latency 59.787 ms 2023-08-25T09:13:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/4180214065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c8107e60 msgr2=0x7fa2c81082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/4180214065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c8107e60 0x7fa2c81082e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009990 tx=0x7fa2bc009ca0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/4180214065 shutdown_connections 2023-08-25T09:13:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/4180214065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2c81159b0 0x7fa2c8117de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/4180214065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c810b010 0x7fa2c8107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/4180214065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c8107e60 0x7fa2c81082e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/4180214065 >> 172.21.15.99:0/4180214065 conn(0x7fa2c806ce20 msgr2=0x7fa2c806d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/4180214065 shutdown_connections 2023-08-25T09:13:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.543+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/4180214065 wait complete. 2023-08-25T09:13:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.545+0000 7fa2ce9ff700 1 Processor -- start 2023-08-25T09:13:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.545+0000 7fa2ce9ff700 1 -- start start 2023-08-25T09:13:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.545+0000 7fa2ce9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c8107e60 0x7fa2c81a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.545+0000 7fa2ce9ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2c810b010 0x7fa2c81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.545+0000 7fa2ce9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c81159b0 0x7fa2c81a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.545+0000 7fa2ce9ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c81a55c0 con 0x7fa2c810b010 2023-08-25T09:13:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.545+0000 7fa2ce9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c81a5730 con 0x7fa2c81159b0 2023-08-25T09:13:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.545+0000 7fa2ce9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c81a58f0 con 0x7fa2c8107e60 2023-08-25T09:13:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c8107e60 0x7fa2c81a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c8107e60 0x7fa2c81a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:39296/0 (socket says 172.21.15.99:39296) 2023-08-25T09:13:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2c7fff700 1 -- 172.21.15.99:0/202840008 learned_addr learned my addr 172.21.15.99:0/202840008 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c81159b0 0x7fa2c81a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2c810b010 0x7fa2c81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/202840008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c8107e60 msgr2=0x7fa2c81a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c8107e60 0x7fa2c81a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2c810b010 msgr2=0x7fa2c81a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.546+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2c810b010 0x7fa2c81a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.547+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/202840008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc0095f0 con 0x7fa2c81159b0 2023-08-25T09:13:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.547+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c81159b0 0x7fa2c81a4fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0007ee0 tx=0x7fa2c000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.547+0000 7fa2c57fa700 1 -- 172.21.15.99:0/202840008 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000ddd0 con 0x7fa2c81159b0 2023-08-25T09:13:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.547+0000 7fa2c57fa700 1 -- 172.21.15.99:0/202840008 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2c000f040 con 0x7fa2c81159b0 2023-08-25T09:13:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.547+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/202840008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c81a5bd0 con 0x7fa2c81159b0 2023-08-25T09:13:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.547+0000 7fa2c57fa700 1 -- 172.21.15.99:0/202840008 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0005a00 con 0x7fa2c81159b0 2023-08-25T09:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.547+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/202840008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c810b6b0 con 0x7fa2c81159b0 2023-08-25T09:13:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.554+0000 7fa2e22c4b80 1 Processor -- start 2023-08-25T09:13:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.555+0000 7fa2c57fa700 1 -- 172.21.15.99:0/202840008 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa2c0005020 con 0x7fa2c81159b0 2023-08-25T09:13:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.555+0000 7fa2e22c4b80 1 -- start start 2023-08-25T09:13:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.555+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a12501f40 0x560a12502320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.555+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a124fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.555+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a124fc990 0x560a124fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.555+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a12464750 con 0x560a12501f40 2023-08-25T09:13:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.555+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a12455720 con 0x560a124fc990 2023-08-25T09:13:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.555+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a12453040 con 0x560a12502860 2023-08-25T09:13:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.556+0000 7fa2c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a124fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.556+0000 7fa2c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a124fc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:39298/0 (socket says 172.21.15.99:39298) 2023-08-25T09:13:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.556+0000 7fa2c7fff700 1 -- 172.21.15.99:0/550911907 learned_addr learned my addr 172.21.15.99:0/550911907 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.556+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a12501f40 0x560a12502320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.556+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a124fc990 0x560a124fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.556+0000 7fa2c7fff700 1 -- 172.21.15.99:0/550911907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a124fc990 msgr2=0x560a124fcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a124fc990 0x560a124fcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2c7fff700 1 -- 172.21.15.99:0/550911907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a12501f40 msgr2=0x560a12502320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a12501f40 0x560a12502320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2c7fff700 1 -- 172.21.15.99:0/550911907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a122ce5d0 con 0x560a12502860 2023-08-25T09:13:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a124fc450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80078c0 tx=0x7fa2b8007bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61329640fbb7e92 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2c57fa700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2b00a0180 0x7fa2b00a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2ae7fc700 1 -- 172.21.15.99:0/550911907 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800d660 con 0x560a12502860 2023-08-25T09:13:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2c57fa700 1 -- 172.21.15.99:0/202840008 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa2c00cae70 con 0x7fa2c81159b0 2023-08-25T09:13:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2ae7fc700 1 -- 172.21.15.99:0/550911907 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2b800dc80 con 0x560a12502860 2023-08-25T09:13:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.557+0000 7fa2ae7fc700 1 -- 172.21.15.99:0/550911907 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8034d50 con 0x560a12502860 2023-08-25T09:13:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.558+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2b00a0180 0x7fa2b00a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.558+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2b00a0180 0x7fa2b00a2640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa2c81a2ce0 tx=0x7fa2bc005040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.558+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/550911907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 msgr2=0x560a124fc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.558+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a124fc450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80078c0 tx=0x7fa2b8007bd0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.559+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/550911907 shutdown_connections 2023-08-25T09:13:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.559+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a124fc990 0x560a124fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.559+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a12501f40 0x560a12502320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.559+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/550911907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a124fc450 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.559+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/550911907 >> 172.21.15.99:0/550911907 conn(0x560a12471080 msgr2=0x560a124f6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.559+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/550911907 shutdown_connections 2023-08-25T09:13:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.559+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/550911907 wait complete. 2023-08-25T09:13:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.559+0000 7fa2e22c4b80 1 -- start start 2023-08-25T09:13:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.560+0000 7fa2e22c4b80 1 Processor -- start 2023-08-25T09:13:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.560+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a125084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.560+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12505720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.560+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a12505c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.560+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1244e930 con 0x560a124fc990 2023-08-25T09:13:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.560+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a123ecb50 con 0x560a12502860 2023-08-25T09:13:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.560+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a123ec850 con 0x560a12501f40 2023-08-25T09:13:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.560+0000 7fa2e22c4b80 10 client.0 _ll_register_callbacks cb 0x560a12336a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12505720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12505720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:39314/0 (socket says 172.21.15.99:39314) 2023-08-25T09:13:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2c77fe700 1 -- 172.21.15.99:0/358513795 learned_addr learned my addr 172.21.15.99:0/358513795 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a12505c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a125084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/358513795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 msgr2=0x560a12505720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12505720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/358513795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 msgr2=0x560a125084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a125084a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/358513795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b800e040 con 0x560a12502860 2023-08-25T09:13:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.561+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12505720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:13:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.562+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a12505c60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00c8550 tx=0x7fa2c0009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affe3b6a77e76e00 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.562+0000 7fa2a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:13:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.562+0000 7fa2a77fe700 1 -- 172.21.15.99:0/358513795 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c009c120 con 0x560a12502860 2023-08-25T09:13:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.562+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/358513795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a124cc160 con 0x560a12502860 2023-08-25T09:13:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.562+0000 7fa2a77fe700 1 -- 172.21.15.99:0/358513795 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa2c009c340 con 0x560a12502860 2023-08-25T09:13:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.562+0000 7fa2a77fe700 1 -- 172.21.15.99:0/358513795 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2c00a0320 con 0x560a12502860 2023-08-25T09:13:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.562+0000 7fa2e22c4b80 10 client.26447 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.562+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/358513795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560a1227ad80 con 0x560a12502860 2023-08-25T09:13:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.563+0000 7fa2a77fe700 1 -- 172.21.15.99:0/358513795 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2c00b4080 con 0x560a12502860 2023-08-25T09:13:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.563+0000 7fa2a77fe700 1 -- 172.21.15.99:0/358513795 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa2c00b6130 con 0x560a12502860 2023-08-25T09:13:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.564+0000 7fa2e22c4b80 10 client.26447 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.564+0000 7fa2e22c4b80 10 client.26447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.564+0000 7fa2e22c4b80 20 client.26447 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.564+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560a12516230 0x560a12518650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.564+0000 7fa2e22c4b80 4 client.26447 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:13:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.564+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/358513795 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a123f4b40 con 0x560a12516230 2023-08-25T09:13:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.564+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560a12516230 0x560a12518650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.565+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560a12516230 0x560a12518650 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30109089dec11a7e server_cookie=4926dd17886040ed in_seq=0 out_seq=0 2023-08-25T09:13:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.565+0000 7fa2a77fe700 10 client.26447 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.567+0000 7fa2a77fe700 1 -- 172.21.15.99:0/358513795 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560a123f4b40 con 0x560a12516230 2023-08-25T09:13:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.567+0000 7fa2a77fe700 10 client.26447 handle_command_reply: tid=0 2023-08-25T09:13:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.567+0000 7fa2e22c4b80 1 client.26447 shutdown 2023-08-25T09:13:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 msgr2=0x560a12505c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a12505c60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00c8550 tx=0x7fa2c0009e20 comp rx=0 tx=0).stop 2023-08-25T09:13:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/358513795 shutdown_connections 2023-08-25T09:13:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560a12516230 0x560a12518650 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a12505c60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a125084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/358513795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12505720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/358513795 >> 172.21.15.99:0/358513795 conn(0x560a12471080 msgr2=0x560a124f1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2a77fe700 0 client.26447 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/358513795 shutdown_connections 2023-08-25T09:13:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/358513795 wait complete. 2023-08-25T09:13:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.568+0000 7fa2e22c4b80 20 client.26447 trim_cache size 0 max 16384 2023-08-25T09:13:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.588+0000 7fa2e22c4b80 1 Processor -- start 2023-08-25T09:13:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2e22c4b80 1 -- start start 2023-08-25T09:13:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a124fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12503350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a1250d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a122d08c0 con 0x560a124fc990 2023-08-25T09:13:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a125061f0 con 0x560a12502860 2023-08-25T09:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a124f7280 con 0x560a12501f40 2023-08-25T09:13:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2ccf9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a1250d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2ccf9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a1250d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40460/0 (socket says 172.21.15.99:40460) 2023-08-25T09:13:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/2252081478 learned_addr learned my addr 172.21.15.99:0/2252081478 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.589+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a124fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/2252081478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 msgr2=0x560a12503350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12503350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/2252081478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 msgr2=0x560a124fc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a124fc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2ccf9c700 1 -- 172.21.15.99:0/2252081478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a124fd040 con 0x560a12502860 2023-08-25T09:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a124fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a1250d370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00c8600 tx=0x7fa2c0097a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea814a53f26b761 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2aeffd700 1 -- 172.21.15.99:0/2252081478 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0097c90 con 0x560a12502860 2023-08-25T09:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2aeffd700 1 -- 172.21.15.99:0/2252081478 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a124cc160 con 0x560a12502860 2023-08-25T09:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.590+0000 7fa2aeffd700 1 -- 172.21.15.99:0/2252081478 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c009ff80 con 0x560a12502860 2023-08-25T09:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.591+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/2252081478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 msgr2=0x560a1250d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.591+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a1250d370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00c8600 tx=0x7fa2c0097a00 comp rx=0 tx=0).stop 2023-08-25T09:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.591+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/2252081478 shutdown_connections 2023-08-25T09:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.591+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12502860 0x560a1250d370 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.591+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a124fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.591+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/2252081478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12501f40 0x560a12503350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.591+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/2252081478 >> 172.21.15.99:0/2252081478 conn(0x560a12471080 msgr2=0x560a12502c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.591+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/2252081478 shutdown_connections 2023-08-25T09:13:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.592+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/2252081478 wait complete. 2023-08-25T09:13:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.593+0000 7fa2e22c4b80 1 -- start start 2023-08-25T09:13:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.593+0000 7fa2e22c4b80 1 Processor -- start 2023-08-25T09:13:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.594+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a1250b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.594+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12501f40 0x560a1250ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.594+0000 7fa2e22c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a12569060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.594+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1250c230 con 0x560a124fc990 2023-08-25T09:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.594+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a125695d0 con 0x560a12501f40 2023-08-25T09:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.594+0000 7fa2e22c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a12569790 con 0x560a12502860 2023-08-25T09:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.594+0000 7fa2e22c4b80 10 client.0 _ll_register_callbacks cb 0x7fa2c0098050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12501f40 0x560a1250ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12501f40 0x560a1250ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40470/0 (socket says 172.21.15.99:40470) 2023-08-25T09:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c77fe700 1 -- 172.21.15.99:0/3278865369 learned_addr learned my addr 172.21.15.99:0/3278865369 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2ccf9c700 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a12569060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a1250b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c77fe700 1 -- 172.21.15.99:0/3278865369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 msgr2=0x560a12569060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a12569060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c77fe700 1 -- 172.21.15.99:0/3278865369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 msgr2=0x560a1250b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a1250b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.595+0000 7fa2c77fe700 1 -- 172.21.15.99:0/3278865369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a1227ad80 con 0x560a12501f40 2023-08-25T09:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.596+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a1250b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.596+0000 7fa2c77fe700 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12501f40 0x560a1250ba90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc006010 tx=0x7fa2bc022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551f0caf7e7c2ca server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.596+0000 7fa2a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.597+0000 7fa2a77fe700 1 -- 172.21.15.99:0/3278865369 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc037040 con 0x560a12501f40 2023-08-25T09:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.597+0000 7fa2a77fe700 1 -- 172.21.15.99:0/3278865369 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa2bc00f930 con 0x560a12501f40 2023-08-25T09:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.597+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/3278865369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a124ef600 con 0x560a12501f40 2023-08-25T09:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.597+0000 7fa2a77fe700 1 -- 172.21.15.99:0/3278865369 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2bc025070 con 0x560a12501f40 2023-08-25T09:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.597+0000 7fa2e22c4b80 10 client.26453 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.597+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/3278865369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560a12263c00 con 0x560a12501f40 2023-08-25T09:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.597+0000 7fa2a77fe700 1 -- 172.21.15.99:0/3278865369 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a124ef600 con 0x560a12501f40 2023-08-25T09:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2a77fe700 1 -- 172.21.15.99:0/3278865369 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa2bc030a70 con 0x560a12501f40 2023-08-25T09:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2e22c4b80 10 client.26453 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2e22c4b80 10 client.26453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2e22c4b80 20 client.26453 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560a1256afe0 0x560a1256d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2e22c4b80 4 client.26453 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/3278865369 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a12447910 con 0x560a1256afe0 2023-08-25T09:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560a1256afe0 0x560a1256d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2c7fff700 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560a1256afe0 0x560a1256d400 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d64ed084483551 server_cookie=b2ff2ac3d0a76d74 in_seq=0 out_seq=0 2023-08-25T09:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.598+0000 7fa2a77fe700 10 client.26453 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2a77fe700 1 -- 172.21.15.99:0/3278865369 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a12447910 con 0x560a1256afe0 2023-08-25T09:13:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2a77fe700 10 client.26453 handle_command_reply: tid=0 2023-08-25T09:13:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 client.26453 shutdown 2023-08-25T09:13:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/3278865369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12501f40 msgr2=0x560a1250ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12501f40 0x560a1250ba90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc006010 tx=0x7fa2bc022040 comp rx=0 tx=0).stop 2023-08-25T09:13:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/3278865369 shutdown_connections 2023-08-25T09:13:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a12502860 0x560a12569060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a124fc990 0x560a1250b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x560a1256afe0 0x560a1256d400 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 --2- 172.21.15.99:0/3278865369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a12501f40 0x560a1250ba90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2a77fe700 0 client.26453 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/3278865369 >> 172.21.15.99:0/3278865369 conn(0x560a12471080 msgr2=0x560a124f0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/3278865369 shutdown_connections 2023-08-25T09:13:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 1 -- 172.21.15.99:0/3278865369 wait complete. 2023-08-25T09:13:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.803+0000 7fa2e22c4b80 20 client.26453 trim_cache size 0 max 16384 2023-08-25T09:13:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:13:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:13:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:13:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:13:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2b00a0180 msgr2=0x7fa2b00a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2b00a0180 0x7fa2b00a2640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa2c81a2ce0 tx=0x7fa2bc005040 comp rx=0 tx=0).stop 2023-08-25T09:13:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/202840008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c81159b0 msgr2=0x7fa2c81a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c81159b0 0x7fa2c81a4fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0007ee0 tx=0x7fa2c000d3b0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/202840008 shutdown_connections 2023-08-25T09:13:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa2b00a0180 0x7fa2b00a2640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c81159b0 0x7fa2c81a4fc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2c8107e60 0x7fa2c81a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 --2- 172.21.15.99:0/202840008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2c810b010 0x7fa2c81a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/202840008 >> 172.21.15.99:0/202840008 conn(0x7fa2c806ce20 msgr2=0x7fa2c8116fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.804+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/202840008 shutdown_connections 2023-08-25T09:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:42.805+0000 7fa2ce9ff700 1 -- 172.21.15.99:0/202840008 wait complete. 2023-08-25T09:13:43.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 64480 33.77 MB/sec warmup 108 sec latency 128.431 ms 2023-08-25T09:13:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:43 smithi099 ceph-mon[97641]: pgmap v456: 97 pgs: 97 active+clean; 659 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 1.19k op/s 2023-08-25T09:13:43.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:43 smithi157 ceph-mon[130879]: pgmap v456: 97 pgs: 97 active+clean; 659 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 1.19k op/s 2023-08-25T09:13:43.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:43 smithi181 ceph-mon[131296]: pgmap v456: 97 pgs: 97 active+clean; 659 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 1.19k op/s 2023-08-25T09:13:44.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 65461 33.98 MB/sec warmup 109 sec latency 35.309 ms 2023-08-25T09:13:45.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 66058 33.91 MB/sec warmup 110 sec latency 96.262 ms 2023-08-25T09:13:45.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:45 smithi099 ceph-mon[97641]: pgmap v457: 97 pgs: 97 active+clean; 659 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 744 op/s 2023-08-25T09:13:45.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:45 smithi157 ceph-mon[130879]: pgmap v457: 97 pgs: 97 active+clean; 659 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 744 op/s 2023-08-25T09:13:45.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:45 smithi181 ceph-mon[131296]: pgmap v457: 97 pgs: 97 active+clean; 659 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 744 op/s 2023-08-25T09:13:46.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 66914 33.99 MB/sec warmup 111 sec latency 57.840 ms 2023-08-25T09:13:47.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 67805 34.17 MB/sec warmup 112 sec latency 43.588 ms 2023-08-25T09:13:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:47 smithi181 ceph-mon[131296]: pgmap v458: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-08-25T09:13:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:47 smithi099 ceph-mon[97641]: pgmap v458: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-08-25T09:13:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:47 smithi157 ceph-mon[130879]: pgmap v458: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-08-25T09:13:48.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 68546 34.26 MB/sec warmup 113 sec latency 84.526 ms 2023-08-25T09:13:49.251 INFO:tasks.workunit.client.0.smithi099.stdout: 10 69360 34.35 MB/sec warmup 114 sec latency 39.830 ms 2023-08-25T09:13:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:49 smithi181 ceph-mon[131296]: pgmap v459: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-08-25T09:13:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:49 smithi099 ceph-mon[97641]: pgmap v459: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-08-25T09:13:49.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:49 smithi157 ceph-mon[130879]: pgmap v459: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-08-25T09:13:50.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 70097 34.33 MB/sec warmup 115 sec latency 50.644 ms 2023-08-25T09:13:51.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 71034 34.47 MB/sec warmup 116 sec latency 54.492 ms 2023-08-25T09:13:51.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:51 smithi181 ceph-mon[131296]: pgmap v460: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-08-25T09:13:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:51 smithi099 ceph-mon[97641]: pgmap v460: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-08-25T09:13:51.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:51 smithi157 ceph-mon[130879]: pgmap v460: 97 pgs: 97 active+clean; 676 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-08-25T09:13:52.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 72001 34.62 MB/sec warmup 117 sec latency 45.084 ms 2023-08-25T09:13:53.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 72825 34.76 MB/sec warmup 118 sec latency 34.046 ms 2023-08-25T09:13:53.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:53 smithi181 ceph-mon[131296]: pgmap v461: 97 pgs: 97 active+clean; 705 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 975 op/s 2023-08-25T09:13:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:53 smithi099 ceph-mon[97641]: pgmap v461: 97 pgs: 97 active+clean; 705 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 975 op/s 2023-08-25T09:13:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:53 smithi157 ceph-mon[130879]: pgmap v461: 97 pgs: 97 active+clean; 705 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 975 op/s 2023-08-25T09:13:54.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 73615 34.86 MB/sec warmup 119 sec latency 44.024 ms 2023-08-25T09:13:55.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:55 smithi099 ceph-mon[97641]: pgmap v462: 97 pgs: 97 active+clean; 705 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.3 MiB/s wr, 560 op/s 2023-08-25T09:13:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:55 smithi157 ceph-mon[130879]: pgmap v462: 97 pgs: 97 active+clean; 705 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.3 MiB/s wr, 560 op/s 2023-08-25T09:13:55.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:55 smithi181 ceph-mon[131296]: pgmap v462: 97 pgs: 97 active+clean; 705 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.3 MiB/s wr, 560 op/s 2023-08-25T09:13:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 -- 172.21.15.99:0/341545782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 msgr2=0x7f71f0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 --2- 172.21.15.99:0/341545782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 0x7f71f0100d20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f71e40098f0 tx=0x7f71e4009c00 comp rx=0 tx=0).stop 2023-08-25T09:13:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 -- 172.21.15.99:0/341545782 shutdown_connections 2023-08-25T09:13:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 --2- 172.21.15.99:0/341545782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f0101260 0x7f71f010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 --2- 172.21.15.99:0/341545782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f01068b0 0x7f71f0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 --2- 172.21.15.99:0/341545782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 0x7f71f0100d20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 -- 172.21.15.99:0/341545782 >> 172.21.15.99:0/341545782 conn(0x7f71f00fc370 msgr2=0x7f71f00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 -- 172.21.15.99:0/341545782 shutdown_connections 2023-08-25T09:13:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.987+0000 7f71f61ff700 1 -- 172.21.15.99:0/341545782 wait complete. 2023-08-25T09:13:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.989+0000 7f71f61ff700 1 Processor -- start 2023-08-25T09:13:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.989+0000 7f71f61ff700 1 -- start start 2023-08-25T09:13:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71f61ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 0x7f71f019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71f61ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f0101260 0x7f71f019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71f61ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f01068b0 0x7f71f0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71f61ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0199070 con 0x7f71f0101260 2023-08-25T09:13:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71f61ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01991e0 con 0x7f71f01068b0 2023-08-25T09:13:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71f61ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01993a0 con 0x7f71f01008a0 2023-08-25T09:13:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 0x7f71f019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 0x7f71f019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:55226/0 (socket says 172.21.15.99:55226) 2023-08-25T09:13:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71ef7fe700 1 -- 172.21.15.99:0/2975657745 learned_addr learned my addr 172.21.15.99:0/2975657745 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.990+0000 7f71effff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f01068b0 0x7f71f0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ef7fe700 1 -- 172.21.15.99:0/2975657745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f01068b0 msgr2=0x7f71f0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f01068b0 0x7f71f0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ef7fe700 1 -- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f0101260 msgr2=0x7f71f019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f0101260 0x7f71f019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ef7fe700 1 -- 172.21.15.99:0/2975657745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e8009c40 con 0x7f71f01008a0 2023-08-25T09:13:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 0x7f71f019eba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71e000e9c0 tx=0x7f71e000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ecff9700 1 -- 172.21.15.99:0/2975657745 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000cbd0 con 0x7f71f01008a0 2023-08-25T09:13:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71f61ff700 1 -- 172.21.15.99:0/2975657745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e4009520 con 0x7f71f01008a0 2023-08-25T09:13:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ecff9700 1 -- 172.21.15.99:0/2975657745 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71e000eea0 con 0x7f71f01008a0 2023-08-25T09:13:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.991+0000 7f71ecff9700 1 -- 172.21.15.99:0/2975657745 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0005520 con 0x7f71f01008a0 2023-08-25T09:13:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.992+0000 7f71f61ff700 1 -- 172.21.15.99:0/2975657745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f0199870 con 0x7f71f01008a0 2023-08-25T09:13:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.996+0000 7f71ecff9700 1 -- 172.21.15.99:0/2975657745 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f71e0004750 con 0x7f71f01008a0 2023-08-25T09:13:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f7209bf9b80 1 Processor -- start 2023-08-25T09:13:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f7209bf9b80 1 -- start start 2023-08-25T09:13:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851075f40 0x562851076320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851076860 0x562851070450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x562851070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562850fd8750 con 0x562851076860 2023-08-25T09:13:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562850fc9720 con 0x562851070990 2023-08-25T09:13:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562850fc7040 con 0x562851075f40 2023-08-25T09:13:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851076860 0x562851070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.997+0000 7f71eeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x562851070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851076860 0x562851070450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60830/0 (socket says 172.21.15.99:60830) 2023-08-25T09:13:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71ef7fe700 1 -- 172.21.15.99:0/1451353551 learned_addr learned my addr 172.21.15.99:0/1451353551 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71eeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x562851070dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51528/0 (socket says 172.21.15.99:51528) 2023-08-25T09:13:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71effff700 1 --2- 172.21.15.99:0/1451353551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851075f40 0x562851076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71eeffd700 1 -- 172.21.15.99:0/1451353551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851075f40 msgr2=0x562851076320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71eeffd700 1 --2- 172.21.15.99:0/1451353551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851075f40 0x562851076320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71eeffd700 1 -- 172.21.15.99:0/1451353551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851076860 msgr2=0x562851070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71eeffd700 1 --2- 172.21.15.99:0/1451353551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851076860 0x562851070450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71eeffd700 1 -- 172.21.15.99:0/1451353551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562850e425d0 con 0x562851070990 2023-08-25T09:13:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71ecff9700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71d80a00c0 0x7f71d80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71eeffd700 1 --2- 172.21.15.99:0/1451353551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x562851070dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009810 tx=0x7f71e4009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6bba41417371688 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.998+0000 7f71ecff9700 1 -- 172.21.15.99:0/2975657745 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f71e0014070 con 0x7f71f01008a0 2023-08-25T09:13:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.999+0000 7f71d5ffb700 1 -- 172.21.15.99:0/1451353551 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e401c070 con 0x562851070990 2023-08-25T09:13:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.999+0000 7f71d5ffb700 1 -- 172.21.15.99:0/1451353551 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71e400b560 con 0x562851070990 2023-08-25T09:13:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.999+0000 7f71effff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71d80a00c0 0x7f71d80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:55.999+0000 7f71d5ffb700 1 -- 172.21.15.99:0/1451353551 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40052c0 con 0x562851070990 2023-08-25T09:13:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f71effff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71d80a00c0 0x7f71d80a2580 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71f019f8a0 tx=0x7f71e800b040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 -- 172.21.15.99:0/1451353551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 msgr2=0x562851070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/1451353551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x562851070dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009810 tx=0x7f71e4009480 comp rx=0 tx=0).stop 2023-08-25T09:13:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 -- 172.21.15.99:0/1451353551 shutdown_connections 2023-08-25T09:13:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/1451353551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x562851070dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/1451353551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851075f40 0x562851076320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/1451353551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851076860 0x562851070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 -- 172.21.15.99:0/1451353551 >> 172.21.15.99:0/1451353551 conn(0x562850fe5080 msgr2=0x56285106ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 -- 172.21.15.99:0/1451353551 shutdown_connections 2023-08-25T09:13:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.000+0000 7f7209bf9b80 1 -- 172.21.15.99:0/1451353551 wait complete. 2023-08-25T09:13:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.001+0000 7f7209bf9b80 1 -- start start 2023-08-25T09:13:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.001+0000 7f7209bf9b80 1 Processor -- start 2023-08-25T09:13:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x56285107c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851079900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 0x56285107a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562850fc2930 con 0x562851075f40 2023-08-25T09:13:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562850f60b50 con 0x562851070990 2023-08-25T09:13:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562850f60850 con 0x562851076860 2023-08-25T09:13:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f7209bf9b80 10 client.0 _ll_register_callbacks cb 0x562850eaaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x56285107c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x56285107c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:51540/0 (socket says 172.21.15.99:51540) 2023-08-25T09:13:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 0x56285107a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71ef7fe700 1 -- 172.21.15.99:0/500413799 learned_addr learned my addr 172.21.15.99:0/500413799 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71eeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851079900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71ef7fe700 1 -- 172.21.15.99:0/500413799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 msgr2=0x56285107a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 0x56285107a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71ef7fe700 1 -- 172.21.15.99:0/500413799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 msgr2=0x562851079900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851079900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.002+0000 7f71ef7fe700 1 -- 172.21.15.99:0/500413799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f010e070 con 0x562851070990 2023-08-25T09:13:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x56285107c6f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71e0095190 tx=0x7f71e00950a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2faa7d6205a5e59 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f71ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:13:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f71ceffd700 1 -- 172.21.15.99:0/500413799 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00985f0 con 0x562851070990 2023-08-25T09:13:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f71ceffd700 1 -- 172.21.15.99:0/500413799 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f71e0016330 con 0x562851070990 2023-08-25T09:13:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f7209bf9b80 1 -- 172.21.15.99:0/500413799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562851040160 con 0x562851070990 2023-08-25T09:13:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f71ceffd700 1 -- 172.21.15.99:0/500413799 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e0099390 con 0x562851070990 2023-08-25T09:13:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f7209bf9b80 10 client.26465 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f7209bf9b80 1 -- 172.21.15.99:0/500413799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x562850deed80 con 0x562851070990 2023-08-25T09:13:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f71ceffd700 1 -- 172.21.15.99:0/500413799 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562851040160 con 0x562851070990 2023-08-25T09:13:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f71ceffd700 1 -- 172.21.15.99:0/500413799 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f71e0098180 con 0x562851070990 2023-08-25T09:13:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f7209bf9b80 10 client.26465 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.003+0000 7f7209bf9b80 10 client.26465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.004+0000 7f7209bf9b80 20 client.26465 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.004+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56285108a210 0x56285108c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.004+0000 7f7209bf9b80 4 client.26465 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:13:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.004+0000 7f7209bf9b80 1 -- 172.21.15.99:0/500413799 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562850f68b40 con 0x56285108a210 2023-08-25T09:13:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.004+0000 7f71eeffd700 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56285108a210 0x56285108c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.004+0000 7f71ceffd700 10 client.26465 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.004+0000 7f71eeffd700 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56285108a210 0x56285108c630 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff00939f01ebe8c server_cookie=c741de1e4db6eb8c in_seq=0 out_seq=0 2023-08-25T09:13:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f71ceffd700 1 -- 172.21.15.99:0/500413799 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562850f68b40 con 0x56285108a210 2023-08-25T09:13:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f71ceffd700 10 client.26465 handle_command_reply: tid=0 2023-08-25T09:13:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 client.26465 shutdown 2023-08-25T09:13:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 -- 172.21.15.99:0/500413799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 msgr2=0x56285107c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x56285107c6f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71e0095190 tx=0x7f71e00950a0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 -- 172.21.15.99:0/500413799 shutdown_connections 2023-08-25T09:13:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 0x56285107a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56285108a210 0x56285108c630 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851070990 0x56285107c6f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/500413799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851079900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f7209bf9b80 1 -- 172.21.15.99:0/500413799 >> 172.21.15.99:0/500413799 conn(0x562850fe5080 msgr2=0x562851065820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.005+0000 7f71ceffd700 0 client.26465 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.006+0000 7f7209bf9b80 1 -- 172.21.15.99:0/500413799 shutdown_connections 2023-08-25T09:13:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.008+0000 7f7209bf9b80 1 -- 172.21.15.99:0/500413799 wait complete. 2023-08-25T09:13:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.008+0000 7f7209bf9b80 20 client.26465 trim_cache size 0 max 16384 2023-08-25T09:13:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.024+0000 7f7209bf9b80 1 Processor -- start 2023-08-25T09:13:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.024+0000 7f7209bf9b80 1 -- start start 2023-08-25T09:13:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851070990 0x562851070450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851075f40 0x5628510812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 0x56285107c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562850e448c0 con 0x562851070990 2023-08-25T09:13:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285107aee0 con 0x562851075f40 2023-08-25T09:13:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562851086ba0 con 0x562851076860 2023-08-25T09:13:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851070990 0x562851070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851070990 0x562851070450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60848/0 (socket says 172.21.15.99:60848) 2023-08-25T09:13:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71ef7fe700 1 -- 172.21.15.99:0/147382404 learned_addr learned my addr 172.21.15.99:0/147382404 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71eeffd700 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851075f40 0x5628510812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71effff700 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 0x56285107c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71ef7fe700 1 -- 172.21.15.99:0/147382404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 msgr2=0x56285107c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 0x56285107c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71ef7fe700 1 -- 172.21.15.99:0/147382404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851075f40 msgr2=0x5628510812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.025+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851075f40 0x5628510812a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.026+0000 7f71ef7fe700 1 -- 172.21.15.99:0/147382404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56285106cd00 con 0x562851070990 2023-08-25T09:13:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.026+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851070990 0x562851070450 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c5b0 tx=0x7f71e000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9496e6cf2bbd08cb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.026+0000 7f71d67fc700 1 -- 172.21.15.99:0/147382404 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0098a60 con 0x562851070990 2023-08-25T09:13:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.026+0000 7f71d67fc700 1 -- 172.21.15.99:0/147382404 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71e0016330 con 0x562851070990 2023-08-25T09:13:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.026+0000 7f71d67fc700 1 -- 172.21.15.99:0/147382404 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e009cac0 con 0x562851070990 2023-08-25T09:13:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 -- 172.21.15.99:0/147382404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851070990 msgr2=0x562851070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851070990 0x562851070450 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c5b0 tx=0x7f71e000c6e0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 -- 172.21.15.99:0/147382404 shutdown_connections 2023-08-25T09:13:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851076860 0x56285107c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851070990 0x562851070450 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/147382404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851075f40 0x5628510812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 -- 172.21.15.99:0/147382404 >> 172.21.15.99:0/147382404 conn(0x562850fe5080 msgr2=0x56285106abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 -- 172.21.15.99:0/147382404 shutdown_connections 2023-08-25T09:13:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 -- 172.21.15.99:0/147382404 wait complete. 2023-08-25T09:13:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.027+0000 7f7209bf9b80 1 -- start start 2023-08-25T09:13:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.028+0000 7f7209bf9b80 1 Processor -- start 2023-08-25T09:13:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.028+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851070990 0x562851072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.028+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.028+0000 7f7209bf9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851076860 0x562851071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.028+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562851073380 con 0x562851075f40 2023-08-25T09:13:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.028+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628510734c0 con 0x562851076860 2023-08-25T09:13:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.028+0000 7f7209bf9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562851071dc0 con 0x562851070990 2023-08-25T09:13:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.028+0000 7f7209bf9b80 10 client.0 _ll_register_callbacks cb 0x562850eaaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:13:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71eeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851070990 0x562851072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71eeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60856/0 (socket says 172.21.15.99:60856) 2023-08-25T09:13:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71eeffd700 1 -- 172.21.15.99:0/80378955 learned_addr learned my addr 172.21.15.99:0/80378955 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:13:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71effff700 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851076860 0x562851071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71ef7fe700 1 -- 172.21.15.99:0/80378955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851076860 msgr2=0x562851071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851076860 0x562851071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71ef7fe700 1 -- 172.21.15.99:0/80378955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 msgr2=0x562851072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:13:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71ef7fe700 1 -- 172.21.15.99:0/80378955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562850deed80 con 0x562851070990 2023-08-25T09:13:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71eeffd700 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:13:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71ef7fe700 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851070990 0x562851072900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f71e000f010 tx=0x7f71e000c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12023002d333bdfe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:13:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.029+0000 7f71ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:13:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f71ceffd700 1 -- 172.21.15.99:0/80378955 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00b4070 con 0x562851070990 2023-08-25T09:13:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 1 -- 172.21.15.99:0/80378955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562851063600 con 0x562851070990 2023-08-25T09:13:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f71ceffd700 1 -- 172.21.15.99:0/80378955 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f71e000cd30 con 0x562851070990 2023-08-25T09:13:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f71ceffd700 1 -- 172.21.15.99:0/80378955 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e0098a60 con 0x562851070990 2023-08-25T09:13:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 10 client.26446 fetch_fsmap learned FSMap version 47 2023-08-25T09:13:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 1 -- 172.21.15.99:0/80378955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x562850dd7c00 con 0x562851070990 2023-08-25T09:13:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f71ceffd700 1 -- 172.21.15.99:0/80378955 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71e00c6020 con 0x562851070990 2023-08-25T09:13:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f71ceffd700 1 -- 172.21.15.99:0/80378955 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f71e00cb0d0 con 0x562851070990 2023-08-25T09:13:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 10 client.26446 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:13:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 10 client.26446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:13:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 20 client.26446 populate_metadata read hostname 'smithi099' 2023-08-25T09:13:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5628510de600 0x5628510e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:13:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 4 client.26446 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:13:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.030+0000 7f7209bf9b80 1 -- 172.21.15.99:0/80378955 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562850fbb910 con 0x5628510de600 2023-08-25T09:13:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.031+0000 7f71eeffd700 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5628510de600 0x5628510e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:13:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.031+0000 7f71eeffd700 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5628510de600 0x5628510e09f0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262b7e174a18e219 server_cookie=801935a6edc00491 in_seq=0 out_seq=0 2023-08-25T09:13:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.031+0000 7f71ceffd700 10 client.26446 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:56.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 74960 34.93 MB/sec execute 1 sec latency 131.292 ms 2023-08-25T09:13:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.293+0000 7f71ceffd700 1 -- 172.21.15.99:0/80378955 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562850fbb910 con 0x5628510de600 2023-08-25T09:13:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.293+0000 7f71ceffd700 10 client.26446 handle_command_reply: tid=0 2023-08-25T09:13:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.293+0000 7f7209bf9b80 1 client.26446 shutdown 2023-08-25T09:13:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 -- 172.21.15.99:0/80378955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851070990 msgr2=0x562851072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851070990 0x562851072900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f71e000f010 tx=0x7f71e000c750 comp rx=0 tx=0).stop 2023-08-25T09:13:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 -- 172.21.15.99:0/80378955 shutdown_connections 2023-08-25T09:13:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5628510de600 0x5628510e09f0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562851076860 0x562851071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f71ceffd700 0 client.26446 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:13:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562851070990 0x562851072900 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 --2- 172.21.15.99:0/80378955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562851075f40 0x562851072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 -- 172.21.15.99:0/80378955 >> 172.21.15.99:0/80378955 conn(0x562850fe5080 msgr2=0x562851073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 -- 172.21.15.99:0/80378955 shutdown_connections 2023-08-25T09:13:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 1 -- 172.21.15.99:0/80378955 wait complete. 2023-08-25T09:13:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.294+0000 7f7209bf9b80 20 client.26446 trim_cache size 0 max 16384 2023-08-25T09:13:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:13:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:13:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:13:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:13:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 -- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71d80a00c0 msgr2=0x7f71d80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71d80a00c0 0x7f71d80a2580 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71f019f8a0 tx=0x7f71e800b040 comp rx=0 tx=0).stop 2023-08-25T09:13:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 -- 172.21.15.99:0/2975657745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 msgr2=0x7f71f019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:13:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 0x7f71f019eba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71e000e9c0 tx=0x7f71e000ed80 comp rx=0 tx=0).stop 2023-08-25T09:13:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 -- 172.21.15.99:0/2975657745 shutdown_connections 2023-08-25T09:13:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f71d80a00c0 0x7f71d80a2580 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f01068b0 0x7f71f0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71f01008a0 0x7f71f019eba0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 --2- 172.21.15.99:0/2975657745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f0101260 0x7f71f019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:13:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 -- 172.21.15.99:0/2975657745 >> 172.21.15.99:0/2975657745 conn(0x7f71f00fc370 msgr2=0x7f71f010bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:13:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 -- 172.21.15.99:0/2975657745 shutdown_connections 2023-08-25T09:13:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:13:56.295+0000 7f71f61ff700 1 -- 172.21.15.99:0/2975657745 wait complete. 2023-08-25T09:13:57.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 75629 32.91 MB/sec execute 2 sec latency 259.916 ms 2023-08-25T09:13:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:57 smithi181 ceph-mon[131296]: pgmap v463: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 871 op/s 2023-08-25T09:13:57.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:57 smithi099 ceph-mon[97641]: pgmap v463: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 871 op/s 2023-08-25T09:13:57.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:57 smithi157 ceph-mon[130879]: pgmap v463: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 871 op/s 2023-08-25T09:13:58.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 75985 26.81 MB/sec execute 3 sec latency 457.413 ms 2023-08-25T09:13:59.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 76710 32.57 MB/sec execute 4 sec latency 71.922 ms 2023-08-25T09:13:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:13:59 smithi181 ceph-mon[131296]: pgmap v464: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2023-08-25T09:13:59.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:13:59 smithi099 ceph-mon[97641]: pgmap v464: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2023-08-25T09:13:59.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:13:59 smithi157 ceph-mon[130879]: pgmap v464: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2023-08-25T09:14:00.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 77501 35.66 MB/sec execute 5 sec latency 65.164 ms 2023-08-25T09:14:01.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 77667 31.62 MB/sec execute 6 sec latency 724.062 ms 2023-08-25T09:14:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:01 smithi181 ceph-mon[131296]: pgmap v465: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2023-08-25T09:14:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:01 smithi099 ceph-mon[97641]: pgmap v465: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2023-08-25T09:14:01.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:01 smithi157 ceph-mon[130879]: pgmap v465: 97 pgs: 97 active+clean; 733 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2023-08-25T09:14:02.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 77667 27.10 MB/sec execute 7 sec latency 1724.177 ms 2023-08-25T09:14:03.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 77667 23.71 MB/sec execute 8 sec latency 2724.243 ms 2023-08-25T09:14:03.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:03 smithi099 ceph-mon[97641]: pgmap v466: 97 pgs: 97 active+clean; 735 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 883 op/s 2023-08-25T09:14:03.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:03 smithi157 ceph-mon[130879]: pgmap v466: 97 pgs: 97 active+clean; 735 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 883 op/s 2023-08-25T09:14:03.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:03 smithi181 ceph-mon[131296]: pgmap v466: 97 pgs: 97 active+clean; 735 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 883 op/s 2023-08-25T09:14:04.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 77985 22.11 MB/sec execute 9 sec latency 2897.766 ms 2023-08-25T09:14:05.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 78158 20.50 MB/sec execute 10 sec latency 216.226 ms 2023-08-25T09:14:05.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:05 smithi099 ceph-mon[97641]: pgmap v467: 97 pgs: 97 active+clean; 735 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.9 MiB/s wr, 653 op/s 2023-08-25T09:14:05.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:05 smithi157 ceph-mon[130879]: pgmap v467: 97 pgs: 97 active+clean; 735 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.9 MiB/s wr, 653 op/s 2023-08-25T09:14:05.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:05 smithi181 ceph-mon[131296]: pgmap v467: 97 pgs: 97 active+clean; 735 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.9 MiB/s wr, 653 op/s 2023-08-25T09:14:06.252 INFO:tasks.workunit.client.0.smithi099.stdout: 10 78516 20.71 MB/sec execute 11 sec latency 137.933 ms 2023-08-25T09:14:07.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 79486 22.59 MB/sec execute 12 sec latency 40.111 ms 2023-08-25T09:14:07.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:07 smithi181 ceph-mon[131296]: pgmap v468: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 1.23k op/s 2023-08-25T09:14:07.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:07 smithi099 ceph-mon[97641]: pgmap v468: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 1.23k op/s 2023-08-25T09:14:07.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:07 smithi157 ceph-mon[130879]: pgmap v468: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 1.23k op/s 2023-08-25T09:14:08.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 80348 25.04 MB/sec execute 13 sec latency 73.821 ms 2023-08-25T09:14:09.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 80830 25.34 MB/sec execute 14 sec latency 93.448 ms 2023-08-25T09:14:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.543+0000 7f77569ff700 1 -- 172.21.15.99:0/2697899814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 msgr2=0x7f7750107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.543+0000 7f77569ff700 1 --2- 172.21.15.99:0/2697899814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 0x7f7750107920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f77440098f0 tx=0x7f7744009c00 comp rx=0 tx=0).stop 2023-08-25T09:14:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.548+0000 7f77569ff700 1 -- 172.21.15.99:0/2697899814 shutdown_connections 2023-08-25T09:14:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.548+0000 7f77569ff700 1 --2- 172.21.15.99:0/2697899814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f775011a100 0x7f775011c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.548+0000 7f77569ff700 1 --2- 172.21.15.99:0/2697899814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 0x7f7750107920 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.548+0000 7f77569ff700 1 --2- 172.21.15.99:0/2697899814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775010f4e0 0x7f7750107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.548+0000 7f77569ff700 1 -- 172.21.15.99:0/2697899814 >> 172.21.15.99:0/2697899814 conn(0x7f775006c9e0 msgr2=0x7f775006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.554+0000 7f77569ff700 1 -- 172.21.15.99:0/2697899814 shutdown_connections 2023-08-25T09:14:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.555+0000 7f77569ff700 1 -- 172.21.15.99:0/2697899814 wait complete. 2023-08-25T09:14:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.557+0000 7f77569ff700 1 Processor -- start 2023-08-25T09:14:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.559+0000 7f77569ff700 1 -- start start 2023-08-25T09:14:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.559+0000 7f77569ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 0x7f77501ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.559+0000 7f77569ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775010f4e0 0x7f77501abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.559+0000 7f77569ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f775011a100 0x7f77501a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.559+0000 7f77569ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77501a5d60 con 0x7f775010f070 2023-08-25T09:14:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.559+0000 7f77569ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77501a5ed0 con 0x7f775011a100 2023-08-25T09:14:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.559+0000 7f77569ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77501a6090 con 0x7f775010f4e0 2023-08-25T09:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.560+0000 7f7754f9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f775011a100 0x7f77501a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.560+0000 7f7754f9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f775011a100 0x7f77501a56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:42328/0 (socket says 172.21.15.99:42328) 2023-08-25T09:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.560+0000 7f7754f9c700 1 -- 172.21.15.99:0/1563532647 learned_addr learned my addr 172.21.15.99:0/1563532647 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.560+0000 7f774ffff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 0x7f77501ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.561+0000 7f774ffff700 1 -- 172.21.15.99:0/1563532647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775010f4e0 msgr2=0x7f77501abdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T09:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.561+0000 7f774ffff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775010f4e0 0x7f77501abdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.561+0000 7f774ffff700 1 -- 172.21.15.99:0/1563532647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f775011a100 msgr2=0x7f77501a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.561+0000 7f774ffff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f775011a100 0x7f77501a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.561+0000 7f774ffff700 1 -- 172.21.15.99:0/1563532647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744009520 con 0x7f775010f070 2023-08-25T09:14:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.562+0000 7f774ffff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 0x7f77501ab890 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7744009810 tx=0x7f774400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.563+0000 7f774d7fa700 1 -- 172.21.15.99:0/1563532647 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774401c070 con 0x7f775010f070 2023-08-25T09:14:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.563+0000 7f774d7fa700 1 -- 172.21.15.99:0/1563532647 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7744005040 con 0x7f775010f070 2023-08-25T09:14:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.563+0000 7f774d7fa700 1 -- 172.21.15.99:0/1563532647 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744003990 con 0x7f775010f070 2023-08-25T09:14:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.566+0000 7f77569ff700 1 -- 172.21.15.99:0/1563532647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77501a6310 con 0x7f775010f070 2023-08-25T09:14:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.566+0000 7f77569ff700 1 -- 172.21.15.99:0/1563532647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77501a6560 con 0x7f775010f070 2023-08-25T09:14:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.571+0000 7f776a2e6b80 1 Processor -- start 2023-08-25T09:14:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.571+0000 7f776a2e6b80 1 -- start start 2023-08-25T09:14:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b2c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2c860 0x564847b26450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b26dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847a8e750 con 0x564847b26990 2023-08-25T09:14:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847a7f720 con 0x564847b2c860 2023-08-25T09:14:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847a7d040 con 0x564847b2bf40 2023-08-25T09:14:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f774ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2c860 0x564847b26450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f774ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2c860 0x564847b26450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:42342/0 (socket says 172.21.15.99:42342) 2023-08-25T09:14:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.572+0000 7f774ffff700 1 -- 172.21.15.99:0/871982446 learned_addr learned my addr 172.21.15.99:0/871982446 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.573+0000 7f774f7fe700 1 --2- 172.21.15.99:0/871982446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b26dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.574+0000 7f774ffff700 1 -- 172.21.15.99:0/871982446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 msgr2=0x564847b2c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.574+0000 7f774ffff700 1 --2- 172.21.15.99:0/871982446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b2c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.574+0000 7f774ffff700 1 -- 172.21.15.99:0/871982446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 msgr2=0x564847b26dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.574+0000 7f774ffff700 1 --2- 172.21.15.99:0/871982446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b26dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.574+0000 7f774ffff700 1 -- 172.21.15.99:0/871982446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648478f85d0 con 0x564847b2c860 2023-08-25T09:14:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.577+0000 7f774d7fa700 1 -- 172.21.15.99:0/1563532647 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f7744020030 con 0x7f775010f070 2023-08-25T09:14:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.578+0000 7f774ffff700 1 --2- 172.21.15.99:0/871982446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2c860 0x564847b26450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7744009c40 tx=0x7f7744005540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f2e01c66e366eb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.580+0000 7f774d7fa700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f77380a0290 0x7f77380a2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.580+0000 7f774d7fa700 1 -- 172.21.15.99:0/1563532647 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f77440cdab0 con 0x7f775010f070 2023-08-25T09:14:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.580+0000 7f77367fc700 1 -- 172.21.15.99:0/871982446 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744005280 con 0x564847b2c860 2023-08-25T09:14:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.580+0000 7f77367fc700 1 -- 172.21.15.99:0/871982446 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7744032040 con 0x564847b2c860 2023-08-25T09:14:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.580+0000 7f77367fc700 1 -- 172.21.15.99:0/871982446 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774409f110 con 0x564847b2c860 2023-08-25T09:14:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.580+0000 7f7754f9c700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f77380a0290 0x7f77380a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.581+0000 7f776a2e6b80 1 -- 172.21.15.99:0/871982446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2c860 msgr2=0x564847b26450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.581+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/871982446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2c860 0x564847b26450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7744009c40 tx=0x7f7744005540 comp rx=0 tx=0).stop 2023-08-25T09:14:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.583+0000 7f776a2e6b80 1 -- 172.21.15.99:0/871982446 shutdown_connections 2023-08-25T09:14:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.583+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/871982446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b26dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.583+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/871982446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b2c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.583+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/871982446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2c860 0x564847b26450 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.583+0000 7f776a2e6b80 1 -- 172.21.15.99:0/871982446 >> 172.21.15.99:0/871982446 conn(0x564847a9b080 msgr2=0x564847b20d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.584+0000 7f7754f9c700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f77380a0290 0x7f77380a2750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7748009550 tx=0x7f7748009040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.584+0000 7f776a2e6b80 1 -- 172.21.15.99:0/871982446 shutdown_connections 2023-08-25T09:14:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.585+0000 7f776a2e6b80 1 -- 172.21.15.99:0/871982446 wait complete. 2023-08-25T09:14:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.585+0000 7f776a2e6b80 1 -- start start 2023-08-25T09:14:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.586+0000 7f776a2e6b80 1 Processor -- start 2023-08-25T09:14:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.587+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b26990 0x564847b2faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.587+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b30030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.587+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.587+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847a78930 con 0x564847b30570 2023-08-25T09:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.587+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847a16b50 con 0x564847b26990 2023-08-25T09:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.587+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847a16850 con 0x564847b2bf40 2023-08-25T09:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.587+0000 7f776a2e6b80 10 client.0 _ll_register_callbacks cb 0x564847960a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.588+0000 7f7754f9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.588+0000 7f7754f9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52030/0 (socket says 172.21.15.99:52030) 2023-08-25T09:14:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.588+0000 7f7754f9c700 1 -- 172.21.15.99:0/1651427625 learned_addr learned my addr 172.21.15.99:0/1651427625 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.588+0000 7f7754f9c700 1 -- 172.21.15.99:0/1651427625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 msgr2=0x564847b30030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T09:14:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.588+0000 7f7754f9c700 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b30030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.589+0000 7f7754f9c700 1 -- 172.21.15.99:0/1651427625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b26990 msgr2=0x564847b2faf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T09:14:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.589+0000 7f7754f9c700 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b26990 0x564847b2faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.589+0000 7f7754f9c700 1 -- 172.21.15.99:0/1651427625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77501a6310 con 0x564847b30570 2023-08-25T09:14:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.589+0000 7f7754f9c700 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3bfb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7748009bd0 tx=0x7f77480091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb10fb8523a186b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.590+0000 7f772f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:14:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.590+0000 7f772f7fe700 1 -- 172.21.15.99:0/1651427625 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748016c90 con 0x564847b30570 2023-08-25T09:14:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.590+0000 7f772f7fe700 1 -- 172.21.15.99:0/1651427625 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f7748016df0 con 0x564847b30570 2023-08-25T09:14:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.590+0000 7f776a2e6b80 1 -- 172.21.15.99:0/1651427625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564847af6160 con 0x564847b30570 2023-08-25T09:14:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.591+0000 7f772f7fe700 1 -- 172.21.15.99:0/1651427625 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7748021400 con 0x564847b30570 2023-08-25T09:14:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.591+0000 7f776a2e6b80 10 client.16404 fetch_fsmap learned FSMap version 47 2023-08-25T09:14:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.591+0000 7f776a2e6b80 1 -- 172.21.15.99:0/1651427625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5648478a4d80 con 0x564847b30570 2023-08-25T09:14:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.592+0000 7f772f7fe700 1 -- 172.21.15.99:0/1651427625 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564847af6160 con 0x564847b30570 2023-08-25T09:14:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.592+0000 7f772f7fe700 1 -- 172.21.15.99:0/1651427625 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7748029070 con 0x564847b30570 2023-08-25T09:14:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.592+0000 7f776a2e6b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:14:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.592+0000 7f776a2e6b80 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:14:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.592+0000 7f776a2e6b80 20 client.16404 populate_metadata read hostname 'smithi099' 2023-08-25T09:14:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.592+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564847b40420 0x564847b42840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.592+0000 7f776a2e6b80 4 client.16404 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:14:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.592+0000 7f776a2e6b80 1 -- 172.21.15.99:0/1651427625 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564847a1eb40 con 0x564847b40420 2023-08-25T09:14:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.593+0000 7f774ffff700 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564847b40420 0x564847b42840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.593+0000 7f774ffff700 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564847b40420 0x564847b42840 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3959d8fc8e02ac9d server_cookie=82bd6e5bba3d0a99 in_seq=0 out_seq=0 2023-08-25T09:14:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.593+0000 7f772f7fe700 10 client.16404 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.594+0000 7f772f7fe700 1 -- 172.21.15.99:0/1651427625 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564847a1eb40 con 0x564847b40420 2023-08-25T09:14:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.594+0000 7f772f7fe700 10 client.16404 handle_command_reply: tid=0 2023-08-25T09:14:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.594+0000 7f776a2e6b80 1 client.16404 shutdown 2023-08-25T09:14:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.597+0000 7f776a2e6b80 1 -- 172.21.15.99:0/1651427625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 msgr2=0x564847b3bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.597+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3bfb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7748009bd0 tx=0x7f77480091d0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.597+0000 7f776a2e6b80 1 -- 172.21.15.99:0/1651427625 shutdown_connections 2023-08-25T09:14:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.597+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3bfb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.597+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564847b40420 0x564847b42840 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.597+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b26990 0x564847b2faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.597+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/1651427625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b30030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.597+0000 7f776a2e6b80 1 -- 172.21.15.99:0/1651427625 >> 172.21.15.99:0/1651427625 conn(0x564847a9b080 msgr2=0x564847b1b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.598+0000 7f772f7fe700 0 client.16404 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.600+0000 7f776a2e6b80 1 -- 172.21.15.99:0/1651427625 shutdown_connections 2023-08-25T09:14:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.600+0000 7f776a2e6b80 1 -- 172.21.15.99:0/1651427625 wait complete. 2023-08-25T09:14:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.600+0000 7f776a2e6b80 20 client.16404 trim_cache size 0 max 16384 2023-08-25T09:14:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.631+0000 7f776a2e6b80 1 Processor -- start 2023-08-25T09:14:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.632+0000 7f776a2e6b80 1 -- start start 2023-08-25T09:14:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.632+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b26990 0x564847b26450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.632+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.633+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.633+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648478fa8c0 con 0x564847b30570 2023-08-25T09:14:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.633+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847b19460 con 0x564847b26990 2023-08-25T09:14:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.633+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847b21250 con 0x564847b2bf40 2023-08-25T09:14:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.633+0000 7f774f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.633+0000 7f774f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b36f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:34258/0 (socket says 172.21.15.99:34258) 2023-08-25T09:14:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.633+0000 7f774f7fe700 1 -- 172.21.15.99:0/721175867 learned_addr learned my addr 172.21.15.99:0/721175867 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.637+0000 7f774f7fe700 1 -- 172.21.15.99:0/721175867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b26990 msgr2=0x564847b26450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.637+0000 7f774f7fe700 1 --2- 172.21.15.99:0/721175867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b26990 0x564847b26450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.637+0000 7f774f7fe700 1 -- 172.21.15.99:0/721175867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 msgr2=0x564847b3b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.637+0000 7f774f7fe700 1 --2- 172.21.15.99:0/721175867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.637+0000 7f774f7fe700 1 -- 172.21.15.99:0/721175867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564847b27040 con 0x564847b2bf40 2023-08-25T09:14:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.638+0000 7f774f7fe700 1 --2- 172.21.15.99:0/721175867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b36f40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7740009bd0 tx=0x7f7740009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6ca6edfad560d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.639+0000 7f7736ffd700 1 -- 172.21.15.99:0/721175867 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740011c30 con 0x564847b2bf40 2023-08-25T09:14:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.639+0000 7f7736ffd700 1 -- 172.21.15.99:0/721175867 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77400044f0 con 0x564847b2bf40 2023-08-25T09:14:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.639+0000 7f7736ffd700 1 -- 172.21.15.99:0/721175867 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740005280 con 0x564847b2bf40 2023-08-25T09:14:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.640+0000 7f776a2e6b80 1 -- 172.21.15.99:0/721175867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 msgr2=0x564847b36f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.640+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/721175867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b36f40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7740009bd0 tx=0x7f7740009ee0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.642+0000 7f776a2e6b80 1 -- 172.21.15.99:0/721175867 shutdown_connections 2023-08-25T09:14:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.642+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/721175867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b30570 0x564847b3b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.642+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/721175867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b26990 0x564847b26450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.642+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/721175867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b2bf40 0x564847b36f40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.642+0000 7f776a2e6b80 1 -- 172.21.15.99:0/721175867 >> 172.21.15.99:0/721175867 conn(0x564847a9b080 msgr2=0x564847b29b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.646+0000 7f776a2e6b80 1 -- 172.21.15.99:0/721175867 shutdown_connections 2023-08-25T09:14:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.646+0000 7f776a2e6b80 1 -- 172.21.15.99:0/721175867 wait complete. 2023-08-25T09:14:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.647+0000 7f776a2e6b80 1 -- start start 2023-08-25T09:14:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.647+0000 7f776a2e6b80 1 Processor -- start 2023-08-25T09:14:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.648+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b35110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.648+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2bf40 0x564847b35650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.648+0000 7f776a2e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b30570 0x564847b93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.648+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847b35db0 con 0x564847b26990 2023-08-25T09:14:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.648+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847b937b0 con 0x564847b2bf40 2023-08-25T09:14:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.648+0000 7f776a2e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847b93970 con 0x564847b30570 2023-08-25T09:14:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.648+0000 7f776a2e6b80 10 client.0 _ll_register_callbacks cb 0x7f7748015440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:14:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.649+0000 7f774ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b35110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.649+0000 7f774ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b35110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52048/0 (socket says 172.21.15.99:52048) 2023-08-25T09:14:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.649+0000 7f774ffff700 1 -- 172.21.15.99:0/3791045058 learned_addr learned my addr 172.21.15.99:0/3791045058 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.649+0000 7f774ffff700 1 -- 172.21.15.99:0/3791045058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b30570 msgr2=0x564847b93270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.649+0000 7f774ffff700 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b30570 0x564847b93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.649+0000 7f774ffff700 1 -- 172.21.15.99:0/3791045058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2bf40 msgr2=0x564847b35650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T09:14:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.649+0000 7f774ffff700 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2bf40 0x564847b35650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.649+0000 7f774ffff700 1 -- 172.21.15.99:0/3791045058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7740009880 con 0x564847b26990 2023-08-25T09:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.650+0000 7f774ffff700 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b35110 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f77440cb240 tx=0x7f7744097020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d791ebd1fbf522b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.650+0000 7f772f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.650+0000 7f772f7fe700 1 -- 172.21.15.99:0/3791045058 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774409ae20 con 0x564847b26990 2023-08-25T09:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.650+0000 7f772f7fe700 1 -- 172.21.15.99:0/3791045058 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f774409b9e0 con 0x564847b26990 2023-08-25T09:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.651+0000 7f776a2e6b80 1 -- 172.21.15.99:0/3791045058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564847b19600 con 0x564847b26990 2023-08-25T09:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.651+0000 7f772f7fe700 1 -- 172.21.15.99:0/3791045058 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f774409ac40 con 0x564847b26990 2023-08-25T09:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.652+0000 7f776a2e6b80 10 client.16407 fetch_fsmap learned FSMap version 47 2023-08-25T09:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.652+0000 7f776a2e6b80 1 -- 172.21.15.99:0/3791045058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56484788dc00 con 0x564847b26990 2023-08-25T09:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.652+0000 7f772f7fe700 1 -- 172.21.15.99:0/3791045058 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564847b19600 con 0x564847b26990 2023-08-25T09:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.652+0000 7f772f7fe700 1 -- 172.21.15.99:0/3791045058 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f77440979f0 con 0x564847b26990 2023-08-25T09:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.653+0000 7f776a2e6b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.653+0000 7f776a2e6b80 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.653+0000 7f776a2e6b80 20 client.16407 populate_metadata read hostname 'smithi099' 2023-08-25T09:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.653+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564847b94db0 0x564847b97210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.653+0000 7f776a2e6b80 4 client.16407 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:14:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.653+0000 7f776a2e6b80 1 -- 172.21.15.99:0/3791045058 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564847a71910 con 0x564847b94db0 2023-08-25T09:14:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.653+0000 7f774f7fe700 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564847b94db0 0x564847b97210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.654+0000 7f774f7fe700 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564847b94db0 0x564847b97210 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71535884e86561ef server_cookie=16d10aeb910610e in_seq=0 out_seq=0 2023-08-25T09:14:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.655+0000 7f772f7fe700 10 client.16407 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:09 smithi099 ceph-mon[97641]: pgmap v469: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 18 MiB/s wr, 918 op/s 2023-08-25T09:14:09.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:09 smithi099 ceph-mon[97641]: osdmap e106: 12 total, 12 up, 12 in 2023-08-25T09:14:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:09 smithi157 ceph-mon[130879]: pgmap v469: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 18 MiB/s wr, 918 op/s 2023-08-25T09:14:09.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:09 smithi157 ceph-mon[130879]: osdmap e106: 12 total, 12 up, 12 in 2023-08-25T09:14:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.886+0000 7f772f7fe700 1 -- 172.21.15.99:0/3791045058 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564847a71910 con 0x564847b94db0 2023-08-25T09:14:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.886+0000 7f772f7fe700 10 client.16407 handle_command_reply: tid=0 2023-08-25T09:14:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.886+0000 7f776a2e6b80 1 client.16407 shutdown 2023-08-25T09:14:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.895+0000 7f776a2e6b80 1 -- 172.21.15.99:0/3791045058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 msgr2=0x564847b35110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.895+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b35110 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f77440cb240 tx=0x7f7744097020 comp rx=0 tx=0).stop 2023-08-25T09:14:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.895+0000 7f776a2e6b80 1 -- 172.21.15.99:0/3791045058 shutdown_connections 2023-08-25T09:14:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.895+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564847b30570 0x564847b93270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.895+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564847b26990 0x564847b35110 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.895+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564847b94db0 0x564847b97210 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.896+0000 7f776a2e6b80 1 --2- 172.21.15.99:0/3791045058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564847b2bf40 0x564847b35650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.896+0000 7f776a2e6b80 1 -- 172.21.15.99:0/3791045058 >> 172.21.15.99:0/3791045058 conn(0x564847a9b080 msgr2=0x564847b3b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.896+0000 7f772f7fe700 0 client.16407 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.896+0000 7f776a2e6b80 1 -- 172.21.15.99:0/3791045058 shutdown_connections 2023-08-25T09:14:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.896+0000 7f776a2e6b80 1 -- 172.21.15.99:0/3791045058 wait complete. 2023-08-25T09:14:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.899+0000 7f776a2e6b80 20 client.16407 trim_cache size 0 max 16384 2023-08-25T09:14:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:14:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:14:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:14:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:14:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.900+0000 7f77569ff700 1 -- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f77380a0290 msgr2=0x7f77380a2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.901+0000 7f77569ff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f77380a0290 0x7f77380a2750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7748009550 tx=0x7f7748009040 comp rx=0 tx=0).stop 2023-08-25T09:14:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.901+0000 7f77569ff700 1 -- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 msgr2=0x7f77501ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.901+0000 7f77569ff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 0x7f77501ab890 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7744009810 tx=0x7f774400fa00 comp rx=0 tx=0).stop 2023-08-25T09:14:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.902+0000 7f77569ff700 1 -- 172.21.15.99:0/1563532647 shutdown_connections 2023-08-25T09:14:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.902+0000 7f77569ff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f77380a0290 0x7f77380a2750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.902+0000 7f77569ff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f775011a100 0x7f77501a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.902+0000 7f77569ff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f775010f070 0x7f77501ab890 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.902+0000 7f77569ff700 1 --2- 172.21.15.99:0/1563532647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775010f4e0 0x7f77501abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.902+0000 7f77569ff700 1 -- 172.21.15.99:0/1563532647 >> 172.21.15.99:0/1563532647 conn(0x7f775006c9e0 msgr2=0x7f775010ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.905+0000 7f77569ff700 1 -- 172.21.15.99:0/1563532647 shutdown_connections 2023-08-25T09:14:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:09.906+0000 7f77569ff700 1 -- 172.21.15.99:0/1563532647 wait complete. 2023-08-25T09:14:09.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:09 smithi181 ceph-mon[131296]: pgmap v469: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 18 MiB/s wr, 918 op/s 2023-08-25T09:14:09.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:09 smithi181 ceph-mon[131296]: osdmap e106: 12 total, 12 up, 12 in 2023-08-25T09:14:10.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 81065 24.60 MB/sec execute 15 sec latency 178.338 ms 2023-08-25T09:14:11.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 81643 24.54 MB/sec execute 16 sec latency 268.341 ms 2023-08-25T09:14:11.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:11 smithi099 ceph-mon[97641]: pgmap v471: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 22 MiB/s wr, 1.10k op/s 2023-08-25T09:14:11.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:11 smithi157 ceph-mon[130879]: pgmap v471: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 22 MiB/s wr, 1.10k op/s 2023-08-25T09:14:11.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:11 smithi181 ceph-mon[131296]: pgmap v471: 97 pgs: 97 active+clean; 740 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 22 MiB/s wr, 1.10k op/s 2023-08-25T09:14:12.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 82385 25.37 MB/sec execute 17 sec latency 50.676 ms 2023-08-25T09:14:13.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 83270 26.85 MB/sec execute 18 sec latency 47.367 ms 2023-08-25T09:14:13.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:13 smithi099 ceph-mon[97641]: pgmap v472: 97 pgs: 97 active+clean; 749 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 986 op/s 2023-08-25T09:14:13.895 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:13 smithi157 ceph-mon[130879]: pgmap v472: 97 pgs: 97 active+clean; 749 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 986 op/s 2023-08-25T09:14:13.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:13 smithi181 ceph-mon[131296]: pgmap v472: 97 pgs: 97 active+clean; 749 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 986 op/s 2023-08-25T09:14:14.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 84130 27.86 MB/sec execute 19 sec latency 45.240 ms 2023-08-25T09:14:15.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 84921 28.53 MB/sec execute 20 sec latency 53.644 ms 2023-08-25T09:14:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:15 smithi099 ceph-mon[97641]: osdmap e107: 12 total, 12 up, 12 in 2023-08-25T09:14:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:15 smithi099 ceph-mon[97641]: pgmap v474: 97 pgs: 97 active+clean; 749 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 368 op/s 2023-08-25T09:14:15.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:15 smithi157 ceph-mon[130879]: osdmap e107: 12 total, 12 up, 12 in 2023-08-25T09:14:15.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:15 smithi157 ceph-mon[130879]: pgmap v474: 97 pgs: 97 active+clean; 749 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 368 op/s 2023-08-25T09:14:15.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:15 smithi181 ceph-mon[131296]: osdmap e107: 12 total, 12 up, 12 in 2023-08-25T09:14:15.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:15 smithi181 ceph-mon[131296]: pgmap v474: 97 pgs: 97 active+clean; 749 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 368 op/s 2023-08-25T09:14:16.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 85762 29.39 MB/sec execute 21 sec latency 45.385 ms 2023-08-25T09:14:17.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 86478 29.57 MB/sec execute 22 sec latency 81.540 ms 2023-08-25T09:14:17.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:17 smithi181 ceph-mon[131296]: pgmap v475: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 739 op/s 2023-08-25T09:14:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:17 smithi099 ceph-mon[97641]: pgmap v475: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 739 op/s 2023-08-25T09:14:17.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:17 smithi157 ceph-mon[130879]: pgmap v475: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 739 op/s 2023-08-25T09:14:18.253 INFO:tasks.workunit.client.0.smithi099.stdout: 10 87300 30.14 MB/sec execute 23 sec latency 71.625 ms 2023-08-25T09:14:19.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 88172 31.08 MB/sec execute 24 sec latency 42.529 ms 2023-08-25T09:14:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:19 smithi181 ceph-mon[131296]: pgmap v476: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.1 MiB/s wr, 595 op/s 2023-08-25T09:14:19.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:19 smithi099 ceph-mon[97641]: pgmap v476: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.1 MiB/s wr, 595 op/s 2023-08-25T09:14:19.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:19 smithi157 ceph-mon[130879]: pgmap v476: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.1 MiB/s wr, 595 op/s 2023-08-25T09:14:20.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 88859 31.40 MB/sec execute 25 sec latency 122.681 ms 2023-08-25T09:14:21.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 89641 31.75 MB/sec execute 26 sec latency 62.624 ms 2023-08-25T09:14:21.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:21 smithi181 ceph-mon[131296]: pgmap v477: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.0 MiB/s wr, 591 op/s 2023-08-25T09:14:21.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:21 smithi099 ceph-mon[97641]: pgmap v477: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.0 MiB/s wr, 591 op/s 2023-08-25T09:14:21.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:21 smithi157 ceph-mon[130879]: pgmap v477: 97 pgs: 97 active+clean; 769 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.0 MiB/s wr, 591 op/s 2023-08-25T09:14:22.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 90304 31.80 MB/sec execute 27 sec latency 76.582 ms 2023-08-25T09:14:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.089+0000 7fa853dff700 1 -- 172.21.15.99:0/1235885364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa84c1068b0 msgr2=0x7fa84c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.089+0000 7fa853dff700 1 --2- 172.21.15.99:0/1235885364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa84c1068b0 0x7fa84c106c90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa8480098f0 tx=0x7fa848009c00 comp rx=0 tx=0).stop 2023-08-25T09:14:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.089+0000 7fa853dff700 1 -- 172.21.15.99:0/1235885364 shutdown_connections 2023-08-25T09:14:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.089+0000 7fa853dff700 1 --2- 172.21.15.99:0/1235885364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa84c101260 0x7fa84c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.089+0000 7fa853dff700 1 --2- 172.21.15.99:0/1235885364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa84c1068b0 0x7fa84c106c90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.089+0000 7fa853dff700 1 --2- 172.21.15.99:0/1235885364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c1008a0 0x7fa84c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.089+0000 7fa853dff700 1 -- 172.21.15.99:0/1235885364 >> 172.21.15.99:0/1235885364 conn(0x7fa84c0fc370 msgr2=0x7fa84c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.089+0000 7fa853dff700 1 -- 172.21.15.99:0/1235885364 shutdown_connections 2023-08-25T09:14:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.090+0000 7fa853dff700 1 -- 172.21.15.99:0/1235885364 wait complete. 2023-08-25T09:14:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.091+0000 7fa853dff700 1 Processor -- start 2023-08-25T09:14:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.091+0000 7fa853dff700 1 -- start start 2023-08-25T09:14:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa853dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c1008a0 0x7fa84c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa853dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa84c101260 0x7fa84c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa853dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa84c1068b0 0x7fa84c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa853dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c196e80 con 0x7fa84c101260 2023-08-25T09:14:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa853dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c196ff0 con 0x7fa84c1008a0 2023-08-25T09:14:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa853dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1971b0 con 0x7fa84c1068b0 2023-08-25T09:14:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa851b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c1008a0 0x7fa84c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa851b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c1008a0 0x7fa84c19c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:53868/0 (socket says 172.21.15.99:53868) 2023-08-25T09:14:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa851b9b700 1 -- 172.21.15.99:0/226114358 learned_addr learned my addr 172.21.15.99:0/226114358 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa85139a700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa84c101260 0x7fa84c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.092+0000 7fa85239c700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa84c1068b0 0x7fa84c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa851b9b700 1 -- 172.21.15.99:0/226114358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa84c1068b0 msgr2=0x7fa84c196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa851b9b700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa84c1068b0 0x7fa84c196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa851b9b700 1 -- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa84c101260 msgr2=0x7fa84c19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa851b9b700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa84c101260 0x7fa84c19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa851b9b700 1 -- 172.21.15.99:0/226114358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa844009c40 con 0x7fa84c1008a0 2023-08-25T09:14:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa851b9b700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c1008a0 0x7fa84c19c9b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa848009810 tx=0x7fa84800bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa842ffd700 1 -- 172.21.15.99:0/226114358 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84801c070 con 0x7fa84c1008a0 2023-08-25T09:14:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa853dff700 1 -- 172.21.15.99:0/226114358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa848009520 con 0x7fa84c1008a0 2023-08-25T09:14:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa842ffd700 1 -- 172.21.15.99:0/226114358 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa848005040 con 0x7fa84c1008a0 2023-08-25T09:14:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.093+0000 7fa842ffd700 1 -- 172.21.15.99:0/226114358 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480049c0 con 0x7fa84c1008a0 2023-08-25T09:14:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.094+0000 7fa853dff700 1 -- 172.21.15.99:0/226114358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c072b50 con 0x7fa84c1008a0 2023-08-25T09:14:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa867695b80 1 Processor -- start 2023-08-25T09:14:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa867695b80 1 -- start start 2023-08-25T09:14:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa842ffd700 1 -- 172.21.15.99:0/226114358 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa848020070 con 0x7fa84c1008a0 2023-08-25T09:14:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa867695b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa867695b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa867695b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa867695b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a13124750 con 0x564a131c1f40 2023-08-25T09:14:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa867695b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a13115720 con 0x564a131bc990 2023-08-25T09:14:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.099+0000 7fa867695b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a13113040 con 0x564a131c2860 2023-08-25T09:14:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85239c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85139a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85139a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131bcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:53884/0 (socket says 172.21.15.99:53884) 2023-08-25T09:14:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85139a700 1 -- 172.21.15.99:0/1955994175 learned_addr learned my addr 172.21.15.99:0/1955994175 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85239c700 1 -- 172.21.15.99:0/1955994175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 msgr2=0x564a131bc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85239c700 1 --2- 172.21.15.99:0/1955994175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85239c700 1 -- 172.21.15.99:0/1955994175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 msgr2=0x564a131bcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85239c700 1 --2- 172.21.15.99:0/1955994175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131bcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.100+0000 7fa85239c700 1 -- 172.21.15.99:0/1955994175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a12f8e5d0 con 0x564a131c1f40 2023-08-25T09:14:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.101+0000 7fa85239c700 1 --2- 172.21.15.99:0/1955994175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c2320 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa844009fd0 tx=0x7fa844009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9287a42b443a18ec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.101+0000 7fa842ffd700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa8380a0140 0x7fa8380a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.101+0000 7fa837fff700 1 -- 172.21.15.99:0/1955994175 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400e040 con 0x564a131c1f40 2023-08-25T09:14:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.101+0000 7fa837fff700 1 -- 172.21.15.99:0/1955994175 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84400f040 con 0x564a131c1f40 2023-08-25T09:14:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.101+0000 7fa837fff700 1 -- 172.21.15.99:0/1955994175 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844004920 con 0x564a131c1f40 2023-08-25T09:14:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.101+0000 7fa842ffd700 1 -- 172.21.15.99:0/226114358 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa8480cc110 con 0x7fa84c1008a0 2023-08-25T09:14:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.101+0000 7fa851b9b700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa8380a0140 0x7fa8380a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.102+0000 7fa867695b80 1 -- 172.21.15.99:0/1955994175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 msgr2=0x564a131c2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.102+0000 7fa867695b80 1 --2- 172.21.15.99:0/1955994175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c2320 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa844009fd0 tx=0x7fa844009920 comp rx=0 tx=0).stop 2023-08-25T09:14:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.103+0000 7fa867695b80 1 -- 172.21.15.99:0/1955994175 shutdown_connections 2023-08-25T09:14:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.103+0000 7fa867695b80 1 --2- 172.21.15.99:0/1955994175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131bcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.103+0000 7fa867695b80 1 --2- 172.21.15.99:0/1955994175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c2320 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.103+0000 7fa867695b80 1 --2- 172.21.15.99:0/1955994175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131bc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.103+0000 7fa867695b80 1 -- 172.21.15.99:0/1955994175 >> 172.21.15.99:0/1955994175 conn(0x564a13131080 msgr2=0x564a131b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.103+0000 7fa851b9b700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa8380a0140 0x7fa8380a2600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a1030 tx=0x7fa848005880 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.103+0000 7fa867695b80 1 -- 172.21.15.99:0/1955994175 shutdown_connections 2023-08-25T09:14:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.103+0000 7fa867695b80 1 -- 172.21.15.99:0/1955994175 wait complete. 2023-08-25T09:14:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.104+0000 7fa867695b80 1 -- start start 2023-08-25T09:14:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.104+0000 7fa867695b80 1 Processor -- start 2023-08-25T09:14:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.105+0000 7fa867695b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.105+0000 7fa867695b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.105+0000 7fa867695b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.105+0000 7fa867695b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1310e930 con 0x564a131c1f40 2023-08-25T09:14:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.105+0000 7fa867695b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a130acb50 con 0x564a131bc990 2023-08-25T09:14:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.105+0000 7fa867695b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a130ac850 con 0x564a131c2860 2023-08-25T09:14:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.105+0000 7fa867695b80 10 client.0 _ll_register_callbacks cb 0x564a12ff6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:14:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.105+0000 7fa85239c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85239c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:41126/0 (socket says 172.21.15.99:41126) 2023-08-25T09:14:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85239c700 1 -- 172.21.15.99:0/3222996434 learned_addr learned my addr 172.21.15.99:0/3222996434 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85239c700 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85139a700 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85239c700 1 -- 172.21.15.99:0/3222996434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 msgr2=0x564a131c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85239c700 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85239c700 1 -- 172.21.15.99:0/3222996434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 msgr2=0x564a131c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85239c700 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.106+0000 7fa85239c700 1 -- 172.21.15.99:0/3222996434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84400e330 con 0x564a131c2860 2023-08-25T09:14:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.107+0000 7fa85239c700 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa844003a00 tx=0x7fa844006040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6cb1f32e7b17a25 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.107+0000 7fa8357fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:14:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.107+0000 7fa8357fa700 1 -- 172.21.15.99:0/3222996434 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844003a60 con 0x564a131c2860 2023-08-25T09:14:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.107+0000 7fa867695b80 1 -- 172.21.15.99:0/3222996434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a1318c160 con 0x564a131c2860 2023-08-25T09:14:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.107+0000 7fa8357fa700 1 -- 172.21.15.99:0/3222996434 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa844003bc0 con 0x564a131c2860 2023-08-25T09:14:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.107+0000 7fa8357fa700 1 -- 172.21.15.99:0/3222996434 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa844003d80 con 0x564a131c2860 2023-08-25T09:14:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.107+0000 7fa867695b80 10 client.26461 fetch_fsmap learned FSMap version 47 2023-08-25T09:14:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.107+0000 7fa867695b80 1 -- 172.21.15.99:0/3222996434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x564a12f3ad80 con 0x564a131c2860 2023-08-25T09:14:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.108+0000 7fa8357fa700 1 -- 172.21.15.99:0/3222996434 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa844031020 con 0x564a131c2860 2023-08-25T09:14:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.108+0000 7fa8357fa700 1 -- 172.21.15.99:0/3222996434 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa844005050 con 0x564a131c2860 2023-08-25T09:14:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.108+0000 7fa867695b80 10 client.26461 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:14:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.108+0000 7fa867695b80 10 client.26461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:14:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.108+0000 7fa867695b80 20 client.26461 populate_metadata read hostname 'smithi099' 2023-08-25T09:14:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.108+0000 7fa867695b80 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564a131d61f0 0x564a131d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.108+0000 7fa867695b80 4 client.26461 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:14:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.108+0000 7fa867695b80 1 -- 172.21.15.99:0/3222996434 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a130b4b40 con 0x564a131d61f0 2023-08-25T09:14:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.110+0000 7fa85139a700 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564a131d61f0 0x564a131d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.111+0000 7fa85139a700 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564a131d61f0 0x564a131d8610 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487ba4ab39026b60 server_cookie=cad590ae6ac87e7d in_seq=0 out_seq=0 2023-08-25T09:14:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.111+0000 7fa8357fa700 10 client.26461 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa8357fa700 1 -- 172.21.15.99:0/3222996434 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a130b4b40 con 0x564a131d61f0 2023-08-25T09:14:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa8357fa700 10 client.26461 handle_command_reply: tid=0 2023-08-25T09:14:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 client.26461 shutdown 2023-08-25T09:14:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 -- 172.21.15.99:0/3222996434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 msgr2=0x564a131c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa844003a00 tx=0x7fa844006040 comp rx=0 tx=0).stop 2023-08-25T09:14:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 -- 172.21.15.99:0/3222996434 shutdown_connections 2023-08-25T09:14:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5e40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564a131d61f0 0x564a131d8610 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131bc990 0x564a131c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 --2- 172.21.15.99:0/3222996434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c1f40 0x564a131c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.113+0000 7fa867695b80 1 -- 172.21.15.99:0/3222996434 >> 172.21.15.99:0/3222996434 conn(0x564a13131080 msgr2=0x564a131b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.114+0000 7fa8357fa700 0 client.26461 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.114+0000 7fa867695b80 1 -- 172.21.15.99:0/3222996434 shutdown_connections 2023-08-25T09:14:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.114+0000 7fa867695b80 1 -- 172.21.15.99:0/3222996434 wait complete. 2023-08-25T09:14:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.114+0000 7fa867695b80 20 client.26461 trim_cache size 0 max 16384 2023-08-25T09:14:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa867695b80 1 Processor -- start 2023-08-25T09:14:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa867695b80 1 -- start start 2023-08-25T09:14:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa867695b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131bc990 0x564a131bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa867695b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa867695b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa867695b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a131c2720 con 0x564a131bc990 2023-08-25T09:14:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa867695b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a12f908c0 con 0x564a131c1f40 2023-08-25T09:14:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa867695b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a131c69c0 con 0x564a131c2860 2023-08-25T09:14:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.134+0000 7fa85239c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131bc990 0x564a131bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85139a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85239c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131bc990 0x564a131bc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57332/0 (socket says 172.21.15.99:57332) 2023-08-25T09:14:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85239c700 1 -- 172.21.15.99:0/1314116792 learned_addr learned my addr 172.21.15.99:0/1314116792 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85239c700 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85139a700 1 -- 172.21.15.99:0/1314116792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 msgr2=0x564a131c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85139a700 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85139a700 1 -- 172.21.15.99:0/1314116792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131bc990 msgr2=0x564a131bc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85139a700 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131bc990 0x564a131bc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85139a700 1 -- 172.21.15.99:0/1314116792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a12f23c00 con 0x564a131c1f40 2023-08-25T09:14:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85239c700 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131bc990 0x564a131bc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:14:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa85139a700 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131cd240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x564a131d8e80 tx=0x7fa83c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42c0e6db01e61760 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.135+0000 7fa840bff700 1 -- 172.21.15.99:0/1314116792 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c017040 con 0x564a131c1f40 2023-08-25T09:14:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa840bff700 1 -- 172.21.15.99:0/1314116792 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa83c018040 con 0x564a131c1f40 2023-08-25T09:14:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa840bff700 1 -- 172.21.15.99:0/1314116792 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004d60 con 0x564a131c1f40 2023-08-25T09:14:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa867695b80 1 -- 172.21.15.99:0/1314116792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 msgr2=0x564a131cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa867695b80 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131cd240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x564a131d8e80 tx=0x7fa83c009310 comp rx=0 tx=0).stop 2023-08-25T09:14:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa867695b80 1 -- 172.21.15.99:0/1314116792 shutdown_connections 2023-08-25T09:14:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa867695b80 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131c2860 0x564a131c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa867695b80 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131bc990 0x564a131bc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa867695b80 1 --2- 172.21.15.99:0/1314116792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131cd240 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa867695b80 1 -- 172.21.15.99:0/1314116792 >> 172.21.15.99:0/1314116792 conn(0x564a13131080 msgr2=0x564a131c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.136+0000 7fa867695b80 1 -- 172.21.15.99:0/1314116792 shutdown_connections 2023-08-25T09:14:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.137+0000 7fa867695b80 1 -- 172.21.15.99:0/1314116792 wait complete. 2023-08-25T09:14:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.137+0000 7fa867695b80 1 -- start start 2023-08-25T09:14:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.137+0000 7fa867695b80 1 Processor -- start 2023-08-25T09:14:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa867695b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131bc990 0x564a131beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa867695b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa867695b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c2860 0x564a131bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa867695b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a131be090 con 0x564a131c2860 2023-08-25T09:14:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa867695b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a131be200 con 0x564a131c1f40 2023-08-25T09:14:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa867695b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a131be3c0 con 0x564a131bc990 2023-08-25T09:14:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa867695b80 10 client.0 _ll_register_callbacks cb 0x7fa844004810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:14:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85139a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85239c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131bc990 0x564a131beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85139a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131beff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:53922/0 (socket says 172.21.15.99:53922) 2023-08-25T09:14:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85239c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131bc990 0x564a131beab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:41142/0 (socket says 172.21.15.99:41142) 2023-08-25T09:14:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85139a700 1 -- 172.21.15.99:0/950452552 learned_addr learned my addr 172.21.15.99:0/950452552 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85239c700 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c2860 0x564a131bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85239c700 1 -- 172.21.15.99:0/950452552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 msgr2=0x564a131beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85239c700 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85239c700 1 -- 172.21.15.99:0/950452552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c2860 msgr2=0x564a131bd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.138+0000 7fa85239c700 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c2860 0x564a131bd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.139+0000 7fa85239c700 1 -- 172.21.15.99:0/950452552 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a12f3ad80 con 0x564a131bc990 2023-08-25T09:14:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.139+0000 7fa85239c700 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131bc990 0x564a131beab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa844005c30 tx=0x7fa844023040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53242c288b0ed916 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.139+0000 7fa8357fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:14:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.139+0000 7fa8357fa700 1 -- 172.21.15.99:0/950452552 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844004ce0 con 0x564a131bc990 2023-08-25T09:14:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.139+0000 7fa867695b80 1 -- 172.21.15.99:0/950452552 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a131af600 con 0x564a131bc990 2023-08-25T09:14:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.139+0000 7fa8357fa700 1 -- 172.21.15.99:0/950452552 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa844032080 con 0x564a131bc990 2023-08-25T09:14:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa8357fa700 1 -- 172.21.15.99:0/950452552 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa844005050 con 0x564a131bc990 2023-08-25T09:14:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa867695b80 10 client.26467 fetch_fsmap learned FSMap version 47 2023-08-25T09:14:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa867695b80 1 -- 172.21.15.99:0/950452552 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fa83c009bd0 con 0x564a131bc990 2023-08-25T09:14:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa8357fa700 1 -- 172.21.15.99:0/950452552 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a131af600 con 0x564a131bc990 2023-08-25T09:14:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa8357fa700 1 -- 172.21.15.99:0/950452552 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa844015450 con 0x564a131bc990 2023-08-25T09:14:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa867695b80 10 client.26467 fetch_fsmap finished waiting for FSMap version 47 2023-08-25T09:14:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa867695b80 10 client.26467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:14:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa867695b80 20 client.26467 populate_metadata read hostname 'smithi099' 2023-08-25T09:14:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa867695b80 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564a1322ae10 0x564a1322d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa867695b80 4 client.26467 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:14:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.140+0000 7fa867695b80 1 -- 172.21.15.99:0/950452552 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a13107910 con 0x564a1322ae10 2023-08-25T09:14:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.141+0000 7fa85139a700 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564a1322ae10 0x564a1322d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.141+0000 7fa85139a700 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564a1322ae10 0x564a1322d230 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f185449502b517a server_cookie=85d3fde6aa36acf2 in_seq=0 out_seq=0 2023-08-25T09:14:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.141+0000 7fa8357fa700 10 client.26467 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:23.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 91160 32.28 MB/sec execute 28 sec latency 110.695 ms 2023-08-25T09:14:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:14:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:14:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:14:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:14:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.381+0000 7fa8357fa700 1 -- 172.21.15.99:0/950452552 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a13107910 con 0x564a1322ae10 2023-08-25T09:14:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.381+0000 7fa8357fa700 10 client.26467 handle_command_reply: tid=0 2023-08-25T09:14:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.381+0000 7fa867695b80 1 client.26467 shutdown 2023-08-25T09:14:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.381+0000 7fa867695b80 1 -- 172.21.15.99:0/950452552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131bc990 msgr2=0x564a131beab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.381+0000 7fa867695b80 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131bc990 0x564a131beab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa844005c30 tx=0x7fa844023040 comp rx=0 tx=0).stop 2023-08-25T09:14:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 1 -- 172.21.15.99:0/950452552 shutdown_connections 2023-08-25T09:14:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a131c2860 0x564a131bd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a131bc990 0x564a131beab0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x564a1322ae10 0x564a1322d230 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 1 --2- 172.21.15.99:0/950452552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a131c1f40 0x564a131beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 1 -- 172.21.15.99:0/950452552 >> 172.21.15.99:0/950452552 conn(0x564a13131080 msgr2=0x564a131c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa8357fa700 0 client.26467 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 1 -- 172.21.15.99:0/950452552 shutdown_connections 2023-08-25T09:14:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 1 -- 172.21.15.99:0/950452552 wait complete. 2023-08-25T09:14:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.382+0000 7fa867695b80 20 client.26467 trim_cache size 0 max 16384 2023-08-25T09:14:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.387+0000 7fa853dff700 1 -- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa8380a0140 msgr2=0x7fa8380a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.387+0000 7fa853dff700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa8380a0140 0x7fa8380a2600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a1030 tx=0x7fa848005880 comp rx=0 tx=0).stop 2023-08-25T09:14:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.387+0000 7fa853dff700 1 -- 172.21.15.99:0/226114358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c1008a0 msgr2=0x7fa84c19c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.387+0000 7fa853dff700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c1008a0 0x7fa84c19c9b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa848009810 tx=0x7fa84800bfc0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.388+0000 7fa853dff700 1 -- 172.21.15.99:0/226114358 shutdown_connections 2023-08-25T09:14:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.388+0000 7fa853dff700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa8380a0140 0x7fa8380a2600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.388+0000 7fa853dff700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa84c1068b0 0x7fa84c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.388+0000 7fa853dff700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c1008a0 0x7fa84c19c9b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.388+0000 7fa853dff700 1 --2- 172.21.15.99:0/226114358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa84c101260 0x7fa84c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.388+0000 7fa853dff700 1 -- 172.21.15.99:0/226114358 >> 172.21.15.99:0/226114358 conn(0x7fa84c0fc370 msgr2=0x7fa84c0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.391+0000 7fa853dff700 1 -- 172.21.15.99:0/226114358 shutdown_connections 2023-08-25T09:14:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:23.391+0000 7fa853dff700 1 -- 172.21.15.99:0/226114358 wait complete. 2023-08-25T09:14:23.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:23 smithi181 ceph-mon[131296]: pgmap v478: 97 pgs: 97 active+clean; 819 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 583 op/s 2023-08-25T09:14:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:23 smithi099 ceph-mon[97641]: pgmap v478: 97 pgs: 97 active+clean; 819 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 583 op/s 2023-08-25T09:14:23.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:23 smithi157 ceph-mon[130879]: pgmap v478: 97 pgs: 97 active+clean; 819 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 583 op/s 2023-08-25T09:14:24.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 92030 33.13 MB/sec execute 29 sec latency 144.669 ms 2023-08-25T09:14:25.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 92763 33.09 MB/sec execute 30 sec latency 48.444 ms 2023-08-25T09:14:25.562 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:25 smithi099 ceph-mon[97641]: pgmap v479: 97 pgs: 97 active+clean; 819 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 577 op/s 2023-08-25T09:14:25.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:25 smithi157 ceph-mon[130879]: pgmap v479: 97 pgs: 97 active+clean; 819 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 577 op/s 2023-08-25T09:14:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:25 smithi181 ceph-mon[131296]: pgmap v479: 97 pgs: 97 active+clean; 819 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 577 op/s 2023-08-25T09:14:26.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 93559 33.42 MB/sec execute 31 sec latency 40.444 ms 2023-08-25T09:14:27.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 94386 33.92 MB/sec execute 32 sec latency 112.619 ms 2023-08-25T09:14:27.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:27 smithi181 ceph-mon[131296]: pgmap v480: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 809 op/s 2023-08-25T09:14:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:27 smithi099 ceph-mon[97641]: pgmap v480: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 809 op/s 2023-08-25T09:14:27.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:27 smithi157 ceph-mon[130879]: pgmap v480: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 809 op/s 2023-08-25T09:14:28.254 INFO:tasks.workunit.client.0.smithi099.stdout: 10 94971 33.62 MB/sec execute 33 sec latency 367.539 ms 2023-08-25T09:14:29.256 INFO:tasks.workunit.client.0.smithi099.stdout: 10 95238 33.08 MB/sec execute 34 sec latency 601.668 ms 2023-08-25T09:14:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:29 smithi099 ceph-mon[97641]: pgmap v481: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-08-25T09:14:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:29 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:14:29.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:29 smithi157 ceph-mon[130879]: pgmap v481: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-08-25T09:14:29.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:29 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:14:29.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:29 smithi181 ceph-mon[131296]: pgmap v481: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-08-25T09:14:29.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:29 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:14:30.256 INFO:tasks.workunit.client.0.smithi099.stdout: 10 95700 32.71 MB/sec execute 35 sec latency 572.722 ms 2023-08-25T09:14:31.256 INFO:tasks.workunit.client.0.smithi099.stdout: 10 96419 33.06 MB/sec execute 36 sec latency 106.264 ms 2023-08-25T09:14:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:31 smithi099 ceph-mon[97641]: pgmap v482: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-08-25T09:14:31.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:31 smithi157 ceph-mon[130879]: pgmap v482: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-08-25T09:14:31.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:31 smithi181 ceph-mon[131296]: pgmap v482: 97 pgs: 97 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-08-25T09:14:32.256 INFO:tasks.workunit.client.0.smithi099.stdout: 10 97194 33.38 MB/sec execute 37 sec latency 78.574 ms 2023-08-25T09:14:33.256 INFO:tasks.workunit.client.0.smithi099.stdout: 10 97833 33.37 MB/sec execute 38 sec latency 168.600 ms 2023-08-25T09:14:33.812 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:33 smithi099 ceph-mon[97641]: pgmap v483: 97 pgs: 97 active+clean; 856 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2023-08-25T09:14:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:33 smithi157 ceph-mon[130879]: pgmap v483: 97 pgs: 97 active+clean; 856 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2023-08-25T09:14:33.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:33 smithi181 ceph-mon[131296]: pgmap v483: 97 pgs: 97 active+clean; 856 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2023-08-25T09:14:34.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 98512 33.54 MB/sec execute 39 sec latency 102.795 ms 2023-08-25T09:14:35.257 INFO:tasks.workunit.client.0.smithi099.stdout: 10 98937 33.03 MB/sec execute 40 sec latency 477.469 ms 2023-08-25T09:14:35.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:35 smithi157 ceph-mon[130879]: pgmap v484: 97 pgs: 97 active+clean; 856 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 794 op/s 2023-08-25T09:14:35.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:35 smithi181 ceph-mon[131296]: pgmap v484: 97 pgs: 97 active+clean; 856 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 794 op/s 2023-08-25T09:14:35.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:35 smithi099 ceph-mon[97641]: pgmap v484: 97 pgs: 97 active+clean; 856 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 794 op/s 2023-08-25T09:14:36.257 INFO:tasks.workunit.client.0.smithi099.stdout: 10 99574 33.06 MB/sec execute 41 sec latency 109.963 ms 2023-08-25T09:14:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.595+0000 7f9ededff700 1 -- 172.21.15.99:0/658945738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed8101260 msgr2=0x7f9ed810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.595+0000 7f9ededff700 1 --2- 172.21.15.99:0/658945738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed8101260 0x7f9ed810d770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000b0a0 tx=0x7f9ed000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.595+0000 7f9ededff700 1 -- 172.21.15.99:0/658945738 shutdown_connections 2023-08-25T09:14:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.595+0000 7f9ededff700 1 --2- 172.21.15.99:0/658945738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed8101260 0x7f9ed810d770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.595+0000 7f9ededff700 1 --2- 172.21.15.99:0/658945738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed81068b0 0x7f9ed8106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.595+0000 7f9ededff700 1 --2- 172.21.15.99:0/658945738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ed81008a0 0x7f9ed8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.595+0000 7f9ededff700 1 -- 172.21.15.99:0/658945738 >> 172.21.15.99:0/658945738 conn(0x7f9ed80fc370 msgr2=0x7f9ed80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.596+0000 7f9ededff700 1 -- 172.21.15.99:0/658945738 shutdown_connections 2023-08-25T09:14:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.596+0000 7f9ededff700 1 -- 172.21.15.99:0/658945738 wait complete. 2023-08-25T09:14:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.597+0000 7f9ededff700 1 Processor -- start 2023-08-25T09:14:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.598+0000 7f9ededff700 1 -- start start 2023-08-25T09:14:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.598+0000 7f9ededff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed81008a0 0x7f9ed819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.598+0000 7f9ededff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ed8101260 0x7f9ed819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.598+0000 7f9ededff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed81068b0 0x7f9ed8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.598+0000 7f9ededff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed81991a0 con 0x7f9ed8101260 2023-08-25T09:14:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.598+0000 7f9ededff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed8199310 con 0x7f9ed81068b0 2023-08-25T09:14:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.598+0000 7f9ededff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed81994d0 con 0x7f9ed81008a0 2023-08-25T09:14:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed81068b0 0x7f9ed8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed81068b0 0x7f9ed8198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57498/0 (socket says 172.21.15.99:57498) 2023-08-25T09:14:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 -- 172.21.15.99:0/2393329141 learned_addr learned my addr 172.21.15.99:0/2393329141 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed81008a0 0x7f9ed819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ed8101260 0x7f9ed819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 -- 172.21.15.99:0/2393329141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed81008a0 msgr2=0x7f9ed819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed81008a0 0x7f9ed819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 -- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ed8101260 msgr2=0x7f9ed819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ed8101260 0x7f9ed819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 -- 172.21.15.99:0/2393329141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009d00 con 0x7f9ed81068b0 2023-08-25T09:14:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.599+0000 7f9edd39c700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed81068b0 0x7f9ed8198b30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009fd0 tx=0x7f9ed000bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.600+0000 7f9ed5ffb700 1 -- 172.21.15.99:0/2393329141 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000e040 con 0x7f9ed81068b0 2023-08-25T09:14:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.600+0000 7f9ed5ffb700 1 -- 172.21.15.99:0/2393329141 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ed0004e60 con 0x7f9ed81068b0 2023-08-25T09:14:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.600+0000 7f9ed5ffb700 1 -- 172.21.15.99:0/2393329141 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0005280 con 0x7f9ed81068b0 2023-08-25T09:14:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.600+0000 7f9ededff700 1 -- 172.21.15.99:0/2393329141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed8199750 con 0x7f9ed81068b0 2023-08-25T09:14:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.600+0000 7f9ededff700 1 -- 172.21.15.99:0/2393329141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed81a5cc0 con 0x7f9ed81068b0 2023-08-25T09:14:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.605+0000 7f9ed5ffb700 1 -- 172.21.15.99:0/2393329141 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9ed001d470 con 0x7f9ed81068b0 2023-08-25T09:14:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.607+0000 7f9ef280db80 1 Processor -- start 2023-08-25T09:14:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.607+0000 7f9ef280db80 1 -- start start 2023-08-25T09:14:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.607+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1100f40 0x558cf1101320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.607+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10fb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.607+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 0x558cf10fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.607+0000 7f9ef280db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf1063750 con 0x558cf10fb990 2023-08-25T09:14:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.607+0000 7f9ef280db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf1054720 con 0x558cf1101860 2023-08-25T09:14:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.607+0000 7f9ef280db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf1052040 con 0x558cf1100f40 2023-08-25T09:14:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edcb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edcb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10fb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57510/0 (socket says 172.21.15.99:57510) 2023-08-25T09:14:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edcb9b700 1 -- 172.21.15.99:0/3527183023 learned_addr learned my addr 172.21.15.99:0/3527183023 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9ed5ffb700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ec80a01c0 0x7f9ec80a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1100f40 0x558cf1101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 0x558cf10fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9ed5ffb700 1 -- 172.21.15.99:0/2393329141 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9ed00ccac0 con 0x7f9ed81068b0 2023-08-25T09:14:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edd39c700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ec80a01c0 0x7f9ec80a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edcb9b700 1 -- 172.21.15.99:0/3527183023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1100f40 msgr2=0x558cf1101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1100f40 0x558cf1101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edcb9b700 1 -- 172.21.15.99:0/3527183023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 msgr2=0x558cf10fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 0x558cf10fbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.608+0000 7f9edcb9b700 1 -- 172.21.15.99:0/3527183023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cf0ecd5d0 con 0x558cf1101860 2023-08-25T09:14:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.609+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10fb450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009d00 tx=0x7f9ecc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b277e758f4e6fc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.609+0000 7f9edd39c700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ec80a01c0 0x7f9ec80a2680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ed001dba0 tx=0x7f9ed000bf10 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.609+0000 7f9ec2ffd700 1 -- 172.21.15.99:0/3527183023 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc004230 con 0x558cf1101860 2023-08-25T09:14:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.609+0000 7f9ec2ffd700 1 -- 172.21.15.99:0/3527183023 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ecc004390 con 0x558cf1101860 2023-08-25T09:14:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.609+0000 7f9ec2ffd700 1 -- 172.21.15.99:0/3527183023 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc019820 con 0x558cf1101860 2023-08-25T09:14:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.610+0000 7f9ef280db80 1 -- 172.21.15.99:0/3527183023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 msgr2=0x558cf10fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.610+0000 7f9ef280db80 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10fb450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009d00 tx=0x7f9ecc011720 comp rx=0 tx=0).stop 2023-08-25T09:14:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.610+0000 7f9ef280db80 1 -- 172.21.15.99:0/3527183023 shutdown_connections 2023-08-25T09:14:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.610+0000 7f9ef280db80 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 0x558cf10fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.610+0000 7f9ef280db80 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1100f40 0x558cf1101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.610+0000 7f9ef280db80 1 --2- 172.21.15.99:0/3527183023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10fb450 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.610+0000 7f9ef280db80 1 -- 172.21.15.99:0/3527183023 >> 172.21.15.99:0/3527183023 conn(0x558cf1070080 msgr2=0x558cf10f5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.611+0000 7f9ef280db80 1 -- 172.21.15.99:0/3527183023 shutdown_connections 2023-08-25T09:14:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.611+0000 7f9ef280db80 1 -- 172.21.15.99:0/3527183023 wait complete. 2023-08-25T09:14:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.612+0000 7f9ef280db80 1 -- start start 2023-08-25T09:14:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.612+0000 7f9ef280db80 1 Processor -- start 2023-08-25T09:14:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf1107630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf11048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf1104e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ef280db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf104d930 con 0x558cf1100f40 2023-08-25T09:14:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ef280db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf0febb50 con 0x558cf1101860 2023-08-25T09:14:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ef280db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf0feb850 con 0x558cf10fb990 2023-08-25T09:14:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ef280db80 10 client.0 _ll_register_callbacks cb 0x558cf0f35a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:14:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ed7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf1107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ed7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf1107630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:47422/0 (socket says 172.21.15.99:47422) 2023-08-25T09:14:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ed7fff700 1 -- 172.21.15.99:0/2722902597 learned_addr learned my addr 172.21.15.99:0/2722902597 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.613+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf1104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf11048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9ed7fff700 1 -- 172.21.15.99:0/2722902597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 msgr2=0x558cf1104e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf1104e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9ed7fff700 1 -- 172.21.15.99:0/2722902597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 msgr2=0x558cf11048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf11048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9ed7fff700 1 -- 172.21.15.99:0/2722902597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc009960 con 0x558cf10fb990 2023-08-25T09:14:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf11048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:14:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf1107630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4013ca0 tx=0x7f9ec40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fcf75bd3b3a1eb0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9eb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:14:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9eb7fff700 1 -- 172.21.15.99:0/2722902597 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4011a70 con 0x558cf10fb990 2023-08-25T09:14:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.614+0000 7f9eb7fff700 1 -- 172.21.15.99:0/2722902597 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9ec4011bd0 con 0x558cf10fb990 2023-08-25T09:14:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.615+0000 7f9ef280db80 1 -- 172.21.15.99:0/2722902597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cf10cb160 con 0x558cf10fb990 2023-08-25T09:14:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.615+0000 7f9eb7fff700 1 -- 172.21.15.99:0/2722902597 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ec4011d90 con 0x558cf10fb990 2023-08-25T09:14:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.615+0000 7f9ef280db80 10 client.26479 fetch_fsmap learned FSMap version 48 2023-08-25T09:14:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.615+0000 7f9ef280db80 1 -- 172.21.15.99:0/2722902597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x558cf0e79d80 con 0x558cf10fb990 2023-08-25T09:14:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.615+0000 7f9eb7fff700 1 -- 172.21.15.99:0/2722902597 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cf10cb160 con 0x558cf10fb990 2023-08-25T09:14:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.615+0000 7f9eb7fff700 1 -- 172.21.15.99:0/2722902597 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9ec4004b90 con 0x558cf10fb990 2023-08-25T09:14:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.615+0000 7f9ef280db80 10 client.26479 fetch_fsmap finished waiting for FSMap version 48 2023-08-25T09:14:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.615+0000 7f9ef280db80 10 client.26479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:14:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.616+0000 7f9ef280db80 20 client.26479 populate_metadata read hostname 'smithi099' 2023-08-25T09:14:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.616+0000 7f9ef280db80 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558cf111da60 0x558cf111fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.616+0000 7f9ef280db80 4 client.26479 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:14:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.616+0000 7f9ef280db80 1 -- 172.21.15.99:0/2722902597 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cf0ff3b40 con 0x558cf111da60 2023-08-25T09:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.616+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558cf111da60 0x558cf111fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.617+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558cf111da60 0x558cf111fe80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7cf512cb8854ea server_cookie=c93fd0b6d443172c in_seq=0 out_seq=0 2023-08-25T09:14:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.617+0000 7f9eb7fff700 10 client.26479 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9eb7fff700 1 -- 172.21.15.99:0/2722902597 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558cf0ff3b40 con 0x558cf111da60 2023-08-25T09:14:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9eb7fff700 10 client.26479 handle_command_reply: tid=0 2023-08-25T09:14:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9ef280db80 1 client.26479 shutdown 2023-08-25T09:14:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9ef280db80 1 -- 172.21.15.99:0/2722902597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 msgr2=0x558cf1107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9ef280db80 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf1107630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4013ca0 tx=0x7f9ec40047c0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9ef280db80 1 -- 172.21.15.99:0/2722902597 shutdown_connections 2023-08-25T09:14:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9ef280db80 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558cf111da60 0x558cf111fe80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9ef280db80 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf1104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9ef280db80 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf1107630 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.619+0000 7f9ef280db80 1 --2- 172.21.15.99:0/2722902597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf11048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.619+0000 7f9ef280db80 1 -- 172.21.15.99:0/2722902597 >> 172.21.15.99:0/2722902597 conn(0x558cf1070080 msgr2=0x558cf10f0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.618+0000 7f9eb7fff700 0 client.26479 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.619+0000 7f9ef280db80 1 -- 172.21.15.99:0/2722902597 shutdown_connections 2023-08-25T09:14:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.619+0000 7f9ef280db80 1 -- 172.21.15.99:0/2722902597 wait complete. 2023-08-25T09:14:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.619+0000 7f9ef280db80 20 client.26479 trim_cache size 0 max 16384 2023-08-25T09:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.642+0000 7f9ef280db80 1 Processor -- start 2023-08-25T09:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.642+0000 7f9ef280db80 1 -- start start 2023-08-25T09:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.642+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf10fb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.642+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf1114b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.642+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.642+0000 7f9ef280db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf0ecf8c0 con 0x558cf1100f40 2023-08-25T09:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.642+0000 7f9ef280db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf10f68a0 con 0x558cf1101860 2023-08-25T09:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.642+0000 7f9ef280db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf10f70f0 con 0x558cf10fb990 2023-08-25T09:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf10fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9edcb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf1114b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf10fb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:47434/0 (socket says 172.21.15.99:47434) 2023-08-25T09:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 -- 172.21.15.99:0/192106226 learned_addr learned my addr 172.21.15.99:0/192106226 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 -- 172.21.15.99:0/192106226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 msgr2=0x558cf10feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 -- 172.21.15.99:0/192106226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 msgr2=0x558cf1114b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf1114b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 -- 172.21.15.99:0/192106226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cf0e62c00 con 0x558cf10fb990 2023-08-25T09:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf1114b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.643+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf10fb450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4001eb0 tx=0x7f9ec4015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c5444ee2fa41248 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.644+0000 7f9ec37fe700 1 -- 172.21.15.99:0/192106226 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec40130f0 con 0x558cf10fb990 2023-08-25T09:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.644+0000 7f9ec37fe700 1 -- 172.21.15.99:0/192106226 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ec4013250 con 0x558cf10fb990 2023-08-25T09:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.644+0000 7f9ec37fe700 1 -- 172.21.15.99:0/192106226 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4004a90 con 0x558cf10fb990 2023-08-25T09:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 -- 172.21.15.99:0/192106226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 msgr2=0x558cf10fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf10fb450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4001eb0 tx=0x7f9ec4015040 comp rx=0 tx=0).stop 2023-08-25T09:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 -- 172.21.15.99:0/192106226 shutdown_connections 2023-08-25T09:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1101860 0x558cf10feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf10fb990 0x558cf10fb450 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 --2- 172.21.15.99:0/192106226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf1100f40 0x558cf1114b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 -- 172.21.15.99:0/192106226 >> 172.21.15.99:0/192106226 conn(0x558cf1070080 msgr2=0x558cf111fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 -- 172.21.15.99:0/192106226 shutdown_connections 2023-08-25T09:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.645+0000 7f9ef280db80 1 -- 172.21.15.99:0/192106226 wait complete. 2023-08-25T09:14:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.646+0000 7f9ef280db80 1 -- start start 2023-08-25T09:14:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.646+0000 7f9ef280db80 1 Processor -- start 2023-08-25T09:14:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 0x558cf10fc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1100f40 0x558cf10fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9ef280db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1101860 0x558cf10fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9ef280db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf10fd8e0 con 0x558cf10fb990 2023-08-25T09:14:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9ef280db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf10fda50 con 0x558cf1100f40 2023-08-25T09:14:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9ef280db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf10fdc10 con 0x558cf1101860 2023-08-25T09:14:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9ef280db80 10 client.0 _ll_register_callbacks cb 0x7f9ec40139b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:14:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9edcb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1100f40 0x558cf10fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9edcb9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1100f40 0x558cf10fcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57536/0 (socket says 172.21.15.99:57536) 2023-08-25T09:14:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.647+0000 7f9edcb9b700 1 -- 172.21.15.99:0/459405506 learned_addr learned my addr 172.21.15.99:0/459405506 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.648+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 0x558cf10fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.648+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1101860 0x558cf10fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.648+0000 7f9edcb9b700 1 -- 172.21.15.99:0/459405506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1101860 msgr2=0x558cf10fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.648+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1101860 0x558cf10fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.648+0000 7f9edcb9b700 1 -- 172.21.15.99:0/459405506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 msgr2=0x558cf10fc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.648+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 0x558cf10fc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.648+0000 7f9edcb9b700 1 -- 172.21.15.99:0/459405506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cf0ecd5d0 con 0x558cf1100f40 2023-08-25T09:14:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9edcb9b700 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1100f40 0x558cf10fcdc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc000c00 tx=0x7f9ecc0096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc66ed8ba167e91 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9eb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:14:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9eb7fff700 1 -- 172.21.15.99:0/459405506 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc028d10 con 0x558cf1100f40 2023-08-25T09:14:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9eb7fff700 1 -- 172.21.15.99:0/459405506 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9ecc0281c0 con 0x558cf1100f40 2023-08-25T09:14:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9ef280db80 1 -- 172.21.15.99:0/459405506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cf10ee600 con 0x558cf1100f40 2023-08-25T09:14:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9eb7fff700 1 -- 172.21.15.99:0/459405506 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ecc028e70 con 0x558cf1100f40 2023-08-25T09:14:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9ef280db80 10 client.26516 fetch_fsmap learned FSMap version 48 2023-08-25T09:14:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9ef280db80 1 -- 172.21.15.99:0/459405506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f9ec4033040 con 0x558cf1100f40 2023-08-25T09:14:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.649+0000 7f9eb7fff700 1 -- 172.21.15.99:0/459405506 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cf10ee600 con 0x558cf1100f40 2023-08-25T09:14:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.650+0000 7f9eb7fff700 1 -- 172.21.15.99:0/459405506 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 48) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9ecc0053b0 con 0x558cf1100f40 2023-08-25T09:14:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.650+0000 7f9ef280db80 10 client.26516 fetch_fsmap finished waiting for FSMap version 48 2023-08-25T09:14:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.650+0000 7f9ef280db80 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:14:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.650+0000 7f9ef280db80 20 client.26516 populate_metadata read hostname 'smithi099' 2023-08-25T09:14:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.650+0000 7f9ef280db80 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558cf1172470 0x558cf1174890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.650+0000 7f9ef280db80 4 client.26516 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:14:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.650+0000 7f9ef280db80 1 -- 172.21.15.99:0/459405506 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558cf1046910 con 0x558cf1172470 2023-08-25T09:14:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.650+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558cf1172470 0x558cf1174890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.651+0000 7f9ed7fff700 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558cf1172470 0x558cf1174890 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4601b90b1ed923 server_cookie=fdb99987513a8e10 in_seq=0 out_seq=0 2023-08-25T09:14:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.651+0000 7f9eb7fff700 10 client.26516 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.802+0000 7f9eb7fff700 1 -- 172.21.15.99:0/459405506 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558cf1046910 con 0x558cf1172470 2023-08-25T09:14:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.802+0000 7f9eb7fff700 10 client.26516 handle_command_reply: tid=0 2023-08-25T09:14:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.802+0000 7f9ef280db80 1 client.26516 shutdown 2023-08-25T09:14:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.802+0000 7f9ef280db80 1 -- 172.21.15.99:0/459405506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1100f40 msgr2=0x558cf10fcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.802+0000 7f9ef280db80 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1100f40 0x558cf10fcdc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc000c00 tx=0x7f9ecc0096d0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9ef280db80 1 -- 172.21.15.99:0/459405506 shutdown_connections 2023-08-25T09:14:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9ef280db80 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cf1101860 0x558cf10fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9ef280db80 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cf10fb990 0x558cf10fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9ef280db80 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558cf1172470 0x558cf1174890 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9ef280db80 1 --2- 172.21.15.99:0/459405506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cf1100f40 0x558cf10fcdc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9eb7fff700 0 client.26516 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9ef280db80 1 -- 172.21.15.99:0/459405506 >> 172.21.15.99:0/459405506 conn(0x558cf1070080 msgr2=0x558cf10feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9ef280db80 1 -- 172.21.15.99:0/459405506 shutdown_connections 2023-08-25T09:14:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.803+0000 7f9ef280db80 1 -- 172.21.15.99:0/459405506 wait complete. 2023-08-25T09:14:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.804+0000 7f9ef280db80 20 client.26516 trim_cache size 0 max 16384 2023-08-25T09:14:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:14:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:14:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:14:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:14:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.804+0000 7f9ededff700 1 -- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ec80a01c0 msgr2=0x7f9ec80a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.804+0000 7f9ededff700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ec80a01c0 0x7f9ec80a2680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ed001dba0 tx=0x7f9ed000bf10 comp rx=0 tx=0).stop 2023-08-25T09:14:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.804+0000 7f9ededff700 1 -- 172.21.15.99:0/2393329141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed81068b0 msgr2=0x7f9ed8198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.804+0000 7f9ededff700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed81068b0 0x7f9ed8198b30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009fd0 tx=0x7f9ed000bce0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.805+0000 7f9ededff700 1 -- 172.21.15.99:0/2393329141 shutdown_connections 2023-08-25T09:14:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.805+0000 7f9ededff700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f9ec80a01c0 0x7f9ec80a2680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.805+0000 7f9ededff700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed81068b0 0x7f9ed8198b30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.805+0000 7f9ededff700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed81008a0 0x7f9ed819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.805+0000 7f9ededff700 1 --2- 172.21.15.99:0/2393329141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ed8101260 0x7f9ed819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.805+0000 7f9ededff700 1 -- 172.21.15.99:0/2393329141 >> 172.21.15.99:0/2393329141 conn(0x7f9ed80fc370 msgr2=0x7f9ed80fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.807+0000 7f9ededff700 1 -- 172.21.15.99:0/2393329141 shutdown_connections 2023-08-25T09:14:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:36.808+0000 7f9ededff700 1 -- 172.21.15.99:0/2393329141 wait complete. 2023-08-25T09:14:37.257 INFO:tasks.workunit.client.0.smithi099.stdout: 10 100449 33.67 MB/sec execute 42 sec latency 108.209 ms 2023-08-25T09:14:37.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:37 smithi181 ceph-mon[131296]: pgmap v485: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 1.12k op/s 2023-08-25T09:14:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:37 smithi099 ceph-mon[97641]: pgmap v485: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 1.12k op/s 2023-08-25T09:14:37.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:37 smithi157 ceph-mon[130879]: pgmap v485: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 1.12k op/s 2023-08-25T09:14:38.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 101156 33.67 MB/sec execute 43 sec latency 36.508 ms 2023-08-25T09:14:39.257 INFO:tasks.workunit.client.0.smithi099.stdout: 10 101757 33.55 MB/sec execute 44 sec latency 67.707 ms 2023-08-25T09:14:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:39 smithi181 ceph-mon[131296]: pgmap v486: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 796 op/s 2023-08-25T09:14:39.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:39 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:14:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:39 smithi099 ceph-mon[97641]: pgmap v486: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 796 op/s 2023-08-25T09:14:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:39 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:14:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:39 smithi157 ceph-mon[130879]: pgmap v486: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 796 op/s 2023-08-25T09:14:39.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:39 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:14:40.257 INFO:tasks.workunit.client.0.smithi099.stdout: 10 102536 33.78 MB/sec execute 45 sec latency 106.132 ms 2023-08-25T09:14:41.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 103320 33.99 MB/sec execute 46 sec latency 68.979 ms 2023-08-25T09:14:41.788 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:41 smithi181 ceph-mon[131296]: pgmap v487: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 796 op/s 2023-08-25T09:14:41.789 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:41 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:14:41.789 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:41 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:14:41.789 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:41 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:14:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:41 smithi099 ceph-mon[97641]: pgmap v487: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 796 op/s 2023-08-25T09:14:41.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:41 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:14:41.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:41 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:14:41.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:41 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:14:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:41 smithi157 ceph-mon[130879]: pgmap v487: 97 pgs: 97 active+clean; 878 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 796 op/s 2023-08-25T09:14:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:41 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:14:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:41 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:14:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:41 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:14:42.257 INFO:tasks.workunit.client.0.smithi099.stdout: 10 104152 34.15 MB/sec execute 47 sec latency 48.875 ms 2023-08-25T09:14:43.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 104803 34.31 MB/sec execute 48 sec latency 139.701 ms 2023-08-25T09:14:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:43 smithi099 ceph-mon[97641]: pgmap v488: 97 pgs: 97 active+clean; 897 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2023-08-25T09:14:43.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:43 smithi157 ceph-mon[130879]: pgmap v488: 97 pgs: 97 active+clean; 897 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2023-08-25T09:14:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:43 smithi181 ceph-mon[131296]: pgmap v488: 97 pgs: 97 active+clean; 897 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2023-08-25T09:14:44.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 105384 34.24 MB/sec execute 49 sec latency 146.478 ms 2023-08-25T09:14:45.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 106223 34.33 MB/sec execute 50 sec latency 47.230 ms 2023-08-25T09:14:45.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:45 smithi157 ceph-mon[130879]: pgmap v489: 97 pgs: 97 active+clean; 897 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.8 MiB/s wr, 593 op/s 2023-08-25T09:14:45.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:45 smithi181 ceph-mon[131296]: pgmap v489: 97 pgs: 97 active+clean; 897 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.8 MiB/s wr, 593 op/s 2023-08-25T09:14:45.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:45 smithi099 ceph-mon[97641]: pgmap v489: 97 pgs: 97 active+clean; 897 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.8 MiB/s wr, 593 op/s 2023-08-25T09:14:46.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 107132 34.66 MB/sec execute 51 sec latency 73.579 ms 2023-08-25T09:14:47.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 107931 34.84 MB/sec execute 52 sec latency 80.474 ms 2023-08-25T09:14:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:47 smithi181 ceph-mon[131296]: pgmap v490: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 823 op/s 2023-08-25T09:14:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:47 smithi099 ceph-mon[97641]: pgmap v490: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 823 op/s 2023-08-25T09:14:47.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:47 smithi157 ceph-mon[130879]: pgmap v490: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 823 op/s 2023-08-25T09:14:48.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 108752 35.11 MB/sec execute 53 sec latency 51.614 ms 2023-08-25T09:14:49.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 109325 35.00 MB/sec execute 54 sec latency 150.470 ms 2023-08-25T09:14:49.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:49 smithi181 ceph-mon[131296]: pgmap v491: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2023-08-25T09:14:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:49 smithi099 ceph-mon[97641]: pgmap v491: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2023-08-25T09:14:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:49 smithi157 ceph-mon[130879]: pgmap v491: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2023-08-25T09:14:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.008+0000 7f748dfff700 1 -- 172.21.15.99:0/576207805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7488102b00 msgr2=0x7f7488102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.008+0000 7f748dfff700 1 --2- 172.21.15.99:0/576207805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7488102b00 0x7f7488102f80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f747c009990 tx=0x7f747c009ca0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.009+0000 7f748dfff700 1 -- 172.21.15.99:0/576207805 shutdown_connections 2023-08-25T09:14:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.009+0000 7f748dfff700 1 --2- 172.21.15.99:0/576207805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74881034c0 0x7f748810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.009+0000 7f748dfff700 1 --2- 172.21.15.99:0/576207805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7488108ba0 0x7f7488108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.009+0000 7f748dfff700 1 --2- 172.21.15.99:0/576207805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7488102b00 0x7f7488102f80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.009+0000 7f748dfff700 1 -- 172.21.15.99:0/576207805 >> 172.21.15.99:0/576207805 conn(0x7f74880fe5d0 msgr2=0x7f74881009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.009+0000 7f748dfff700 1 -- 172.21.15.99:0/576207805 shutdown_connections 2023-08-25T09:14:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.009+0000 7f748dfff700 1 -- 172.21.15.99:0/576207805 wait complete. 2023-08-25T09:14:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.010+0000 7f748dfff700 1 Processor -- start 2023-08-25T09:14:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.010+0000 7f748dfff700 1 -- start start 2023-08-25T09:14:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f748dfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7488102b00 0x7f748819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f748dfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74881034c0 0x7f748819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f748dfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7488108ba0 0x7f74881989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f748dfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7488199060 con 0x7f7488102b00 2023-08-25T09:14:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f748dfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74881991d0 con 0x7f7488108ba0 2023-08-25T09:14:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f748dfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7488199390 con 0x7f74881034c0 2023-08-25T09:14:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f7486ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74881034c0 0x7f748819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f7486ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74881034c0 0x7f748819f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:53516/0 (socket says 172.21.15.99:53516) 2023-08-25T09:14:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f7486ffd700 1 -- 172.21.15.99:0/698895753 learned_addr learned my addr 172.21.15.99:0/698895753 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f7487fff700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7488108ba0 0x7f74881989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f7486ffd700 1 -- 172.21.15.99:0/698895753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7488108ba0 msgr2=0x7f74881989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.011+0000 7f7486ffd700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7488108ba0 0x7f74881989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f7486ffd700 1 -- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7488102b00 msgr2=0x7f748819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f7486ffd700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7488102b00 0x7f748819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f7486ffd700 1 -- 172.21.15.99:0/698895753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0095f0 con 0x7f74881034c0 2023-08-25T09:14:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f7486ffd700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74881034c0 0x7f748819f0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f747c005910 tx=0x7f747c00b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f7484ff9700 1 -- 172.21.15.99:0/698895753 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c01d070 con 0x7f74881034c0 2023-08-25T09:14:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f748dfff700 1 -- 172.21.15.99:0/698895753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7488199610 con 0x7f74881034c0 2023-08-25T09:14:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f748dfff700 1 -- 172.21.15.99:0/698895753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74881a5b80 con 0x7f74881034c0 2023-08-25T09:14:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f7484ff9700 1 -- 172.21.15.99:0/698895753 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747c00fb80 con 0x7f74881034c0 2023-08-25T09:14:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.012+0000 7f7484ff9700 1 -- 172.21.15.99:0/698895753 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c017650 con 0x7f74881034c0 2023-08-25T09:14:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.017+0000 7f74a1862b80 1 Processor -- start 2023-08-25T09:14:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.017+0000 7f74a1862b80 1 -- start start 2023-08-25T09:14:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f22f40 0x55ce25f23320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 0x55ce25f1d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f1d990 0x55ce25f1ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f74a1862b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25e85750 con 0x55ce25f1d990 2023-08-25T09:14:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f74a1862b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25e76720 con 0x55ce25f22f40 2023-08-25T09:14:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f74a1862b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25e74040 con 0x55ce25f23860 2023-08-25T09:14:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f7487fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f22f40 0x55ce25f23320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f74877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 0x55ce25f1d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f7487fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f22f40 0x55ce25f23320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57892/0 (socket says 172.21.15.99:57892) 2023-08-25T09:14:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f7487fff700 1 -- 172.21.15.99:0/2752669641 learned_addr learned my addr 172.21.15.99:0/2752669641 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f7487fff700 1 -- 172.21.15.99:0/2752669641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 msgr2=0x55ce25f1d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f7487fff700 1 --2- 172.21.15.99:0/2752669641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 0x55ce25f1d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f7487fff700 1 -- 172.21.15.99:0/2752669641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f1d990 msgr2=0x55ce25f1ddd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T09:14:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f7487fff700 1 --2- 172.21.15.99:0/2752669641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f1d990 0x55ce25f1ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.018+0000 7f7487fff700 1 -- 172.21.15.99:0/2752669641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce25cef5d0 con 0x55ce25f22f40 2023-08-25T09:14:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.019+0000 7f7487fff700 1 --2- 172.21.15.99:0/2752669641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f22f40 0x55ce25f23320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f74800093b0 tx=0x7f7480009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d934fe14e77d873 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.019+0000 7f7484ff9700 1 -- 172.21.15.99:0/698895753 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f747c022470 con 0x7f74881034c0 2023-08-25T09:14:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.019+0000 7f746dffb700 1 -- 172.21.15.99:0/2752669641 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748000e070 con 0x55ce25f22f40 2023-08-25T09:14:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.019+0000 7f746dffb700 1 -- 172.21.15.99:0/2752669641 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748000f040 con 0x55ce25f22f40 2023-08-25T09:14:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.019+0000 7f746dffb700 1 -- 172.21.15.99:0/2752669641 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480014650 con 0x55ce25f22f40 2023-08-25T09:14:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.020+0000 7f74a1862b80 1 -- 172.21.15.99:0/2752669641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f22f40 msgr2=0x55ce25f23320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.020+0000 7f74a1862b80 1 --2- 172.21.15.99:0/2752669641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f22f40 0x55ce25f23320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f74800093b0 tx=0x7f7480009920 comp rx=0 tx=0).stop 2023-08-25T09:14:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.020+0000 7f74a1862b80 1 -- 172.21.15.99:0/2752669641 shutdown_connections 2023-08-25T09:14:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.020+0000 7f74a1862b80 1 --2- 172.21.15.99:0/2752669641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f1d990 0x55ce25f1ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.020+0000 7f74a1862b80 1 --2- 172.21.15.99:0/2752669641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f22f40 0x55ce25f23320 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.020+0000 7f74a1862b80 1 --2- 172.21.15.99:0/2752669641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 0x55ce25f1d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.020+0000 7f74a1862b80 1 -- 172.21.15.99:0/2752669641 >> 172.21.15.99:0/2752669641 conn(0x55ce25e92080 msgr2=0x55ce25f17d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.020+0000 7f74a1862b80 1 -- 172.21.15.99:0/2752669641 shutdown_connections 2023-08-25T09:14:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.021+0000 7f74a1862b80 1 -- 172.21.15.99:0/2752669641 wait complete. 2023-08-25T09:14:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.021+0000 7f7484ff9700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f74700a0010 0x7f74700a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.021+0000 7f7484ff9700 1 -- 172.21.15.99:0/698895753 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f747c0ccab0 con 0x7f74881034c0 2023-08-25T09:14:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.021+0000 7f74a1862b80 1 -- start start 2023-08-25T09:14:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.021+0000 7f74877fe700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f74700a0010 0x7f74700a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.022+0000 7f74a1862b80 1 Processor -- start 2023-08-25T09:14:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.022+0000 7f74877fe700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f74700a0010 0x7f74700a24d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7478005950 tx=0x7f747800b390 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.022+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f29690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.022+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f26910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.022+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f26f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.022+0000 7f74a1862b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25e6f930 con 0x55ce25f23860 2023-08-25T09:14:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.022+0000 7f74a1862b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25e0db50 con 0x55ce25f1d990 2023-08-25T09:14:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.022+0000 7f74a1862b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25e0d850 con 0x55ce25f22f40 2023-08-25T09:14:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74a1862b80 10 client.0 _ll_register_callbacks cb 0x55ce25d57a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:14:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f29690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f7487fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f26f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f29690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57898/0 (socket says 172.21.15.99:57898) 2023-08-25T09:14:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 -- 172.21.15.99:0/1540766861 learned_addr learned my addr 172.21.15.99:0/1540766861 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f7486ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f26910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 -- 172.21.15.99:0/1540766861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 msgr2=0x55ce25f26910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f26910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 -- 172.21.15.99:0/1540766861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 msgr2=0x55ce25f26f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f26f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 -- 172.21.15.99:0/1540766861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7480009090 con 0x55ce25f1d990 2023-08-25T09:14:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.023+0000 7f74877fe700 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f29690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7478003b70 tx=0x7f7478004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9e2f83f80c103a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.024+0000 7f7466ffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:14:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.024+0000 7f7466ffd700 1 -- 172.21.15.99:0/1540766861 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478011ce0 con 0x55ce25f1d990 2023-08-25T09:14:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.024+0000 7f74a1862b80 1 -- 172.21.15.99:0/1540766861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce25eed160 con 0x55ce25f1d990 2023-08-25T09:14:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.024+0000 7f7466ffd700 1 -- 172.21.15.99:0/1540766861 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f7478017070 con 0x55ce25f1d990 2023-08-25T09:14:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.024+0000 7f7466ffd700 1 -- 172.21.15.99:0/1540766861 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7478011e40 con 0x55ce25f1d990 2023-08-25T09:14:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.024+0000 7f74a1862b80 10 client.26528 fetch_fsmap learned FSMap version 48 2023-08-25T09:14:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.024+0000 7f74a1862b80 1 -- 172.21.15.99:0/1540766861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55ce25c9bd80 con 0x55ce25f1d990 2023-08-25T09:14:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.024+0000 7f7466ffd700 1 -- 172.21.15.99:0/1540766861 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce25eed160 con 0x55ce25f1d990 2023-08-25T09:14:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f7466ffd700 1 -- 172.21.15.99:0/1540766861 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 48) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7478026410 con 0x55ce25f1d990 2023-08-25T09:14:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f74a1862b80 10 client.26528 fetch_fsmap finished waiting for FSMap version 48 2023-08-25T09:14:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f74a1862b80 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:14:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f74a1862b80 20 client.26528 populate_metadata read hostname 'smithi099' 2023-08-25T09:14:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f74a1862b80 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ce25f372b0 0x55ce25f396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f74a1862b80 4 client.26528 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:14:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f74a1862b80 1 -- 172.21.15.99:0/1540766861 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce25e15b40 con 0x55ce25f372b0 2023-08-25T09:14:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f7486ffd700 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ce25f372b0 0x55ce25f396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.025+0000 7f7486ffd700 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ce25f372b0 0x55ce25f396d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620cfc1b7cc49f45 server_cookie=242deafa7484c452 in_seq=0 out_seq=0 2023-08-25T09:14:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.027+0000 7f7466ffd700 10 client.26528 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f7466ffd700 1 -- 172.21.15.99:0/1540766861 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ce25e15b40 con 0x55ce25f372b0 2023-08-25T09:14:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f7466ffd700 10 client.26528 handle_command_reply: tid=0 2023-08-25T09:14:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 client.26528 shutdown 2023-08-25T09:14:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 -- 172.21.15.99:0/1540766861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 msgr2=0x55ce25f29690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f29690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7478003b70 tx=0x7f7478004750 comp rx=0 tx=0).stop 2023-08-25T09:14:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 -- 172.21.15.99:0/1540766861 shutdown_connections 2023-08-25T09:14:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f26f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ce25f372b0 0x55ce25f396d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f29690 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f7466ffd700 0 client.26528 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 --2- 172.21.15.99:0/1540766861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f26910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 -- 172.21.15.99:0/1540766861 >> 172.21.15.99:0/1540766861 conn(0x55ce25e92080 msgr2=0x55ce25f12820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.028+0000 7f74a1862b80 1 -- 172.21.15.99:0/1540766861 shutdown_connections 2023-08-25T09:14:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.029+0000 7f74a1862b80 1 -- 172.21.15.99:0/1540766861 wait complete. 2023-08-25T09:14:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.029+0000 7f74a1862b80 20 client.26528 trim_cache size 0 max 16384 2023-08-25T09:14:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.051+0000 7f74a1862b80 1 Processor -- start 2023-08-25T09:14:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.052+0000 7f74a1862b80 1 -- start start 2023-08-25T09:14:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f22f40 0x55ce25f396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 0x55ce25f2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74a1862b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25cf18c0 con 0x55ce25f22f40 2023-08-25T09:14:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74a1862b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25f2e810 con 0x55ce25f1d990 2023-08-25T09:14:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74a1862b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25f215a0 con 0x55ce25f23860 2023-08-25T09:14:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74877fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f7487fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 0x55ce25f2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74877fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:57912/0 (socket says 172.21.15.99:57912) 2023-08-25T09:14:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f74877fe700 1 -- 172.21.15.99:0/2505080510 learned_addr learned my addr 172.21.15.99:0/2505080510 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.053+0000 7f7486ffd700 1 --2- 172.21.15.99:0/2505080510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f22f40 0x55ce25f396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.054+0000 7f74877fe700 1 -- 172.21.15.99:0/2505080510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 msgr2=0x55ce25f2e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.054+0000 7f74877fe700 1 --2- 172.21.15.99:0/2505080510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 0x55ce25f2e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.054+0000 7f74877fe700 1 -- 172.21.15.99:0/2505080510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f22f40 msgr2=0x55ce25f396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.054+0000 7f74877fe700 1 --2- 172.21.15.99:0/2505080510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f22f40 0x55ce25f396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.054+0000 7f74877fe700 1 -- 172.21.15.99:0/2505080510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce25c84c00 con 0x55ce25f1d990 2023-08-25T09:14:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.054+0000 7f74877fe700 1 --2- 172.21.15.99:0/2505080510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1d450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7478005900 tx=0x7f747800bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7df0dc82ac4c6b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.055+0000 7f746e7fc700 1 -- 172.21.15.99:0/2505080510 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780265f0 con 0x55ce25f1d990 2023-08-25T09:14:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.055+0000 7f746e7fc700 1 -- 172.21.15.99:0/2505080510 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7478015070 con 0x55ce25f1d990 2023-08-25T09:14:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.055+0000 7f746e7fc700 1 -- 172.21.15.99:0/2505080510 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478004c50 con 0x55ce25f1d990 2023-08-25T09:14:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.056+0000 7f74a1862b80 1 -- 172.21.15.99:0/2505080510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 msgr2=0x55ce25f1d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.056+0000 7f74a1862b80 1 --2- 172.21.15.99:0/2505080510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1d450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7478005900 tx=0x7f747800bf00 comp rx=0 tx=0).stop 2023-08-25T09:14:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.057+0000 7f74a1862b80 1 -- 172.21.15.99:0/2505080510 shutdown_connections 2023-08-25T09:14:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.057+0000 7f74a1862b80 1 --2- 172.21.15.99:0/2505080510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f23860 0x55ce25f2e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.057+0000 7f74a1862b80 1 --2- 172.21.15.99:0/2505080510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1d450 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.057+0000 7f74a1862b80 1 --2- 172.21.15.99:0/2505080510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f22f40 0x55ce25f396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.057+0000 7f74a1862b80 1 -- 172.21.15.99:0/2505080510 >> 172.21.15.99:0/2505080510 conn(0x55ce25e92080 msgr2=0x55ce25f17c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.057+0000 7f74a1862b80 1 -- 172.21.15.99:0/2505080510 shutdown_connections 2023-08-25T09:14:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.057+0000 7f74a1862b80 1 -- 172.21.15.99:0/2505080510 wait complete. 2023-08-25T09:14:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.058+0000 7f74a1862b80 1 -- start start 2023-08-25T09:14:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.058+0000 7f74a1862b80 1 Processor -- start 2023-08-25T09:14:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f200d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f74a1862b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f1e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f74a1862b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25f1efc0 con 0x55ce25f23860 2023-08-25T09:14:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f74a1862b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25f1f130 con 0x55ce25f1d990 2023-08-25T09:14:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f74a1862b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce25f1f2f0 con 0x55ce25f22f40 2023-08-25T09:14:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f74a1862b80 10 client.0 _ll_register_callbacks cb 0x7f7478003750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:14:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f7486ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f7486ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f200d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:53550/0 (socket says 172.21.15.99:53550) 2023-08-25T09:14:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f7486ffd700 1 -- 172.21.15.99:0/3326384152 learned_addr learned my addr 172.21.15.99:0/3326384152 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:14:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f74877fe700 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f7487fff700 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f1e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f7486ffd700 1 -- 172.21.15.99:0/3326384152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 msgr2=0x55ce25f1fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.059+0000 7f7486ffd700 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.060+0000 7f7486ffd700 1 -- 172.21.15.99:0/3326384152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 msgr2=0x55ce25f1e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:14:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.060+0000 7f7486ffd700 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f1e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.060+0000 7f7486ffd700 1 -- 172.21.15.99:0/3326384152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce25c9bd80 con 0x55ce25f22f40 2023-08-25T09:14:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.060+0000 7f7487fff700 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f1e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:14:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.060+0000 7f7486ffd700 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f200d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f747c096c50 tx=0x7f747c022bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f029ac86bf60afa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:14:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.060+0000 7f7466ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:14:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.061+0000 7f7466ffd700 1 -- 172.21.15.99:0/3326384152 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c098580 con 0x55ce25f22f40 2023-08-25T09:14:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.061+0000 7f7466ffd700 1 -- 172.21.15.99:0/3326384152 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f747c00fcf0 con 0x55ce25f22f40 2023-08-25T09:14:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.061+0000 7f74a1862b80 1 -- 172.21.15.99:0/3326384152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce25f10600 con 0x55ce25f22f40 2023-08-25T09:14:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.062+0000 7f7466ffd700 1 -- 172.21.15.99:0/3326384152 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747c09a440 con 0x55ce25f22f40 2023-08-25T09:14:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.062+0000 7f74a1862b80 10 client.26503 fetch_fsmap learned FSMap version 48 2023-08-25T09:14:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.062+0000 7f74a1862b80 1 -- 172.21.15.99:0/3326384152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55ce25cc21f0 con 0x55ce25f22f40 2023-08-25T09:14:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.062+0000 7f7466ffd700 1 -- 172.21.15.99:0/3326384152 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce25f10600 con 0x55ce25f22f40 2023-08-25T09:14:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.063+0000 7f7466ffd700 1 -- 172.21.15.99:0/3326384152 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f747c0a1ad0 con 0x55ce25f22f40 2023-08-25T09:14:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.063+0000 7f74a1862b80 10 client.26503 fetch_fsmap finished waiting for FSMap version 48 2023-08-25T09:14:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.063+0000 7f74a1862b80 10 client.26503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:14:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.063+0000 7f74a1862b80 20 client.26503 populate_metadata read hostname 'smithi099' 2023-08-25T09:14:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.063+0000 7f74a1862b80 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ce25f94670 0x55ce25f96a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:14:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.063+0000 7f74a1862b80 4 client.26503 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:14:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.063+0000 7f74a1862b80 1 -- 172.21.15.99:0/3326384152 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ce25e68910 con 0x55ce25f94670 2023-08-25T09:14:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.064+0000 7f74877fe700 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ce25f94670 0x55ce25f96a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:14:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.065+0000 7f74877fe700 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ce25f94670 0x55ce25f96a60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfda1c8951d1d1e server_cookie=a9ec03e7e43b948d in_seq=0 out_seq=0 2023-08-25T09:14:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.065+0000 7f7466ffd700 10 client.26503 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:50.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 109882 34.78 MB/sec execute 55 sec latency 116.950 ms 2023-08-25T09:14:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.282+0000 7f7466ffd700 1 -- 172.21.15.99:0/3326384152 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ce25e68910 con 0x55ce25f94670 2023-08-25T09:14:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.282+0000 7f7466ffd700 10 client.26503 handle_command_reply: tid=0 2023-08-25T09:14:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.282+0000 7f74a1862b80 1 client.26503 shutdown 2023-08-25T09:14:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 -- 172.21.15.99:0/3326384152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 msgr2=0x55ce25f200d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f200d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f747c096c50 tx=0x7f747c022bf0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 -- 172.21.15.99:0/3326384152 shutdown_connections 2023-08-25T09:14:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce25f23860 0x55ce25f1e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55ce25f94670 0x55ce25f96a60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ce25f1d990 0x55ce25f1fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 --2- 172.21.15.99:0/3326384152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce25f22f40 0x55ce25f200d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 -- 172.21.15.99:0/3326384152 >> 172.21.15.99:0/3326384152 conn(0x55ce25e92080 msgr2=0x55ce25f20af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f7466ffd700 0 client.26503 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:14:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 -- 172.21.15.99:0/3326384152 shutdown_connections 2023-08-25T09:14:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.284+0000 7f74a1862b80 1 -- 172.21.15.99:0/3326384152 wait complete. 2023-08-25T09:14:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.285+0000 7f74a1862b80 20 client.26503 trim_cache size 0 max 16384 2023-08-25T09:14:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:14:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:14:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:14:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:14:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 -- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f74700a0010 msgr2=0x7f74700a24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f74700a0010 0x7f74700a24d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7478005950 tx=0x7f747800b390 comp rx=0 tx=0).stop 2023-08-25T09:14:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 -- 172.21.15.99:0/698895753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74881034c0 msgr2=0x7f748819f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:14:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74881034c0 0x7f748819f0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f747c005910 tx=0x7f747c00b610 comp rx=0 tx=0).stop 2023-08-25T09:14:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 -- 172.21.15.99:0/698895753 shutdown_connections 2023-08-25T09:14:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f74700a0010 0x7f74700a24d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7488108ba0 0x7f74881989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7488102b00 0x7f748819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 --2- 172.21.15.99:0/698895753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74881034c0 0x7f748819f0d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:14:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 -- 172.21.15.99:0/698895753 >> 172.21.15.99:0/698895753 conn(0x7f74880fe5d0 msgr2=0x7f74880feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:14:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.290+0000 7f748dfff700 1 -- 172.21.15.99:0/698895753 shutdown_connections 2023-08-25T09:14:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:14:50.291+0000 7f748dfff700 1 -- 172.21.15.99:0/698895753 wait complete. 2023-08-25T09:14:51.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 110846 34.98 MB/sec execute 56 sec latency 131.236 ms 2023-08-25T09:14:51.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:51 smithi181 ceph-mon[131296]: pgmap v492: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2023-08-25T09:14:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:51 smithi099 ceph-mon[97641]: pgmap v492: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2023-08-25T09:14:51.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:51 smithi157 ceph-mon[130879]: pgmap v492: 97 pgs: 97 active+clean; 930 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2023-08-25T09:14:52.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 111802 35.57 MB/sec execute 57 sec latency 31.940 ms 2023-08-25T09:14:53.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 112545 35.53 MB/sec execute 58 sec latency 55.004 ms 2023-08-25T09:14:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:53 smithi099 ceph-mon[97641]: pgmap v493: 97 pgs: 97 active+clean; 949 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 800 op/s 2023-08-25T09:14:53.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:53 smithi157 ceph-mon[130879]: pgmap v493: 97 pgs: 97 active+clean; 949 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 800 op/s 2023-08-25T09:14:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:53 smithi181 ceph-mon[131296]: pgmap v493: 97 pgs: 97 active+clean; 949 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 800 op/s 2023-08-25T09:14:54.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 113191 35.47 MB/sec execute 59 sec latency 93.713 ms 2023-08-25T09:14:55.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 114030 35.74 MB/sec execute 60 sec latency 58.852 ms 2023-08-25T09:14:55.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:55 smithi157 ceph-mon[130879]: pgmap v494: 97 pgs: 97 active+clean; 949 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.6 MiB/s wr, 532 op/s 2023-08-25T09:14:55.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:55 smithi181 ceph-mon[131296]: pgmap v494: 97 pgs: 97 active+clean; 949 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.6 MiB/s wr, 532 op/s 2023-08-25T09:14:55.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:55 smithi099 ceph-mon[97641]: pgmap v494: 97 pgs: 97 active+clean; 949 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.6 MiB/s wr, 532 op/s 2023-08-25T09:14:56.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 114916 35.77 MB/sec execute 61 sec latency 38.507 ms 2023-08-25T09:14:57.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 115658 35.92 MB/sec execute 62 sec latency 61.238 ms 2023-08-25T09:14:57.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:57 smithi181 ceph-mon[131296]: pgmap v495: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-08-25T09:14:57.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:57 smithi099 ceph-mon[97641]: pgmap v495: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-08-25T09:14:57.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:57 smithi157 ceph-mon[130879]: pgmap v495: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-08-25T09:14:58.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 116515 36.09 MB/sec execute 63 sec latency 75.355 ms 2023-08-25T09:14:59.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 117306 36.23 MB/sec execute 64 sec latency 59.325 ms 2023-08-25T09:14:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:14:59 smithi181 ceph-mon[131296]: pgmap v496: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 522 op/s 2023-08-25T09:14:59.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:14:59 smithi099 ceph-mon[97641]: pgmap v496: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 522 op/s 2023-08-25T09:14:59.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:14:59 smithi157 ceph-mon[130879]: pgmap v496: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 522 op/s 2023-08-25T09:15:00.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 117582 35.92 MB/sec execute 65 sec latency 470.233 ms 2023-08-25T09:15:01.258 INFO:tasks.workunit.client.0.smithi099.stdout: 10 117754 35.49 MB/sec execute 66 sec latency 1053.758 ms 2023-08-25T09:15:01.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:01 smithi181 ceph-mon[131296]: pgmap v497: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 522 op/s 2023-08-25T09:15:01.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:01 smithi099 ceph-mon[97641]: pgmap v497: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 522 op/s 2023-08-25T09:15:01.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:01 smithi157 ceph-mon[130879]: pgmap v497: 97 pgs: 97 active+clean; 981 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 522 op/s 2023-08-25T09:15:02.259 INFO:tasks.workunit.client.0.smithi099.stdout: 10 118244 35.40 MB/sec execute 67 sec latency 107.997 ms 2023-08-25T09:15:03.259 INFO:tasks.workunit.client.0.smithi099.stdout: 10 119077 35.43 MB/sec execute 68 sec latency 47.585 ms 2023-08-25T09:15:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/4057600184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84067d10 msgr2=0x7f4e84068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/4057600184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84067d10 0x7f4e84068190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4e740099c0 tx=0x7f4e74009cd0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/4057600184 shutdown_connections 2023-08-25T09:15:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/4057600184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e840686d0 0x7f4e8410d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/4057600184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e841068d0 0x7f4e84106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/4057600184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84067d10 0x7f4e84068190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/4057600184 >> 172.21.15.99:0/4057600184 conn(0x7f4e84075250 msgr2=0x7f4e84075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/4057600184 shutdown_connections 2023-08-25T09:15:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.464+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/4057600184 wait complete. 2023-08-25T09:15:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.465+0000 7f4e8c7ff700 1 Processor -- start 2023-08-25T09:15:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.465+0000 7f4e8c7ff700 1 -- start start 2023-08-25T09:15:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.465+0000 7f4e8c7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84067d10 0x7f4e8419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.465+0000 7f4e8c7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e840686d0 0x7f4e8419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.465+0000 7f4e8c7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e841068d0 0x7f4e84198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.465+0000 7f4e8c7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e841990b0 con 0x7f4e84067d10 2023-08-25T09:15:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.465+0000 7f4e8c7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84199220 con 0x7f4e840686d0 2023-08-25T09:15:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.465+0000 7f4e8c7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e841993e0 con 0x7f4e841068d0 2023-08-25T09:15:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e89d9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e840686d0 0x7f4e8419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e8ad9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e841068d0 0x7f4e84198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e8ad9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e841068d0 0x7f4e84198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57062/0 (socket says 172.21.15.99:57062) 2023-08-25T09:15:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e89d9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e840686d0 0x7f4e8419f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:43524/0 (socket says 172.21.15.99:43524) 2023-08-25T09:15:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e8ad9c700 1 -- 172.21.15.99:0/221470299 learned_addr learned my addr 172.21.15.99:0/221470299 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84067d10 0x7f4e8419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e89d9a700 1 -- 172.21.15.99:0/221470299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e841068d0 msgr2=0x7f4e84198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e841068d0 0x7f4e84198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e89d9a700 1 -- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84067d10 msgr2=0x7f4e8419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84067d10 0x7f4e8419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.466+0000 7f4e89d9a700 1 -- 172.21.15.99:0/221470299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740095f0 con 0x7f4e840686d0 2023-08-25T09:15:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.467+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e840686d0 0x7f4e8419f120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009fd0 tx=0x7f4e7400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.467+0000 7f4e7b7fe700 1 -- 172.21.15.99:0/221470299 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7401d070 con 0x7f4e840686d0 2023-08-25T09:15:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.467+0000 7f4e7b7fe700 1 -- 172.21.15.99:0/221470299 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e74022470 con 0x7f4e840686d0 2023-08-25T09:15:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.467+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/221470299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e84199660 con 0x7f4e840686d0 2023-08-25T09:15:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.467+0000 7f4e7b7fe700 1 -- 172.21.15.99:0/221470299 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74017650 con 0x7f4e840686d0 2023-08-25T09:15:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.467+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/221470299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e841a5bd0 con 0x7f4e840686d0 2023-08-25T09:15:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.470+0000 7f4e7b7fe700 1 -- 172.21.15.99:0/221470299 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f4e74022850 con 0x7f4e840686d0 2023-08-25T09:15:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4ea007fb80 1 Processor -- start 2023-08-25T09:15:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4ea007fb80 1 -- start start 2023-08-25T09:15:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4e7b7fe700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4e700a00c0 0x7f4e700a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd539320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4e7b7fe700 1 -- 172.21.15.99:0/221470299 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4e740cc780 con 0x7f4e840686d0 2023-08-25T09:15:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd539860 0x55f7cd533450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd533990 0x55f7cd533dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd49b750 con 0x55f7cd538f40 2023-08-25T09:15:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd48c720 con 0x55f7cd533990 2023-08-25T09:15:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.472+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd48a040 con 0x55f7cd539860 2023-08-25T09:15:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8ad9c700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4e700a00c0 0x7f4e700a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd539320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e89d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd539860 0x55f7cd533450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd539320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44714/0 (socket says 172.21.15.99:44714) 2023-08-25T09:15:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8a59b700 1 -- 172.21.15.99:0/3952354625 learned_addr learned my addr 172.21.15.99:0/3952354625 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8a59b700 1 -- 172.21.15.99:0/3952354625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd539860 msgr2=0x55f7cd533450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3952354625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd539860 0x55f7cd533450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8a59b700 1 -- 172.21.15.99:0/3952354625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd533990 msgr2=0x55f7cd533dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3952354625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd533990 0x55f7cd533dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8a59b700 1 -- 172.21.15.99:0/3952354625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7cd3055d0 con 0x55f7cd538f40 2023-08-25T09:15:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.473+0000 7f4e8ad9c700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4e700a00c0 0x7f4e700a2580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009c10 tx=0x7f4e7c009920 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.474+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3952354625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd539320 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009d00 tx=0x7f4e80011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9f3b47775f2457 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.474+0000 7f4e78ff9700 1 -- 172.21.15.99:0/3952354625 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80004230 con 0x55f7cd538f40 2023-08-25T09:15:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.474+0000 7f4e78ff9700 1 -- 172.21.15.99:0/3952354625 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e80004390 con 0x55f7cd538f40 2023-08-25T09:15:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.474+0000 7f4e78ff9700 1 -- 172.21.15.99:0/3952354625 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80019750 con 0x55f7cd538f40 2023-08-25T09:15:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.475+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3952354625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 msgr2=0x55f7cd539320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.475+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3952354625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd539320 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009d00 tx=0x7f4e80011720 comp rx=0 tx=0).stop 2023-08-25T09:15:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.476+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3952354625 shutdown_connections 2023-08-25T09:15:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.476+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3952354625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd533990 0x55f7cd533dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.476+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3952354625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd539320 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.476+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3952354625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd539860 0x55f7cd533450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.476+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3952354625 >> 172.21.15.99:0/3952354625 conn(0x55f7cd4a8080 msgr2=0x55f7cd52dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.476+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3952354625 shutdown_connections 2023-08-25T09:15:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.476+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3952354625 wait complete. 2023-08-25T09:15:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.476+0000 7f4ea007fb80 1 -- start start 2023-08-25T09:15:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.477+0000 7f4ea007fb80 1 Processor -- start 2023-08-25T09:15:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.477+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd53f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.477+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd538f40 0x55f7cd53c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.477+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd539860 0x55f7cd53ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.477+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd485930 con 0x55f7cd539860 2023-08-25T09:15:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.477+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd423b50 con 0x55f7cd538f40 2023-08-25T09:15:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.477+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd423850 con 0x55f7cd533990 2023-08-25T09:15:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.477+0000 7f4ea007fb80 10 client.0 _ll_register_callbacks cb 0x55f7cd36da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:15:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8ad9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd539860 0x55f7cd53ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8ad9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd539860 0x55f7cd53ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44728/0 (socket says 172.21.15.99:44728) 2023-08-25T09:15:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8ad9c700 1 -- 172.21.15.99:0/3356339717 learned_addr learned my addr 172.21.15.99:0/3356339717 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd53f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8ad9c700 1 -- 172.21.15.99:0/3356339717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 msgr2=0x55f7cd53f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8ad9c700 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd53f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8ad9c700 1 -- 172.21.15.99:0/3356339717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd538f40 msgr2=0x55f7cd53c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8ad9c700 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd538f40 0x55f7cd53c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.478+0000 7f4e8ad9c700 1 -- 172.21.15.99:0/3356339717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e80009960 con 0x55f7cd539860 2023-08-25T09:15:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.479+0000 7f4e8ad9c700 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd539860 0x55f7cd53ce70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0045f0 tx=0x7f4e7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928caee6da2d25a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.479+0000 7f4e6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:15:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.479+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3356339717 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00e040 con 0x55f7cd539860 2023-08-25T09:15:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.479+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3356339717 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4e7c019c50 con 0x55f7cd539860 2023-08-25T09:15:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.479+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3356339717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7cd503160 con 0x55f7cd539860 2023-08-25T09:15:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.480+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3356339717 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e7c019e10 con 0x55f7cd539860 2023-08-25T09:15:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.480+0000 7f4ea007fb80 10 client.16461 fetch_fsmap learned FSMap version 48 2023-08-25T09:15:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.480+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3356339717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55f7cd2b1d80 con 0x55f7cd539860 2023-08-25T09:15:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.480+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3356339717 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7cd503160 con 0x55f7cd539860 2023-08-25T09:15:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.481+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3356339717 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4e7c017570 con 0x55f7cd539860 2023-08-25T09:15:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.481+0000 7f4ea007fb80 10 client.16461 fetch_fsmap finished waiting for FSMap version 48 2023-08-25T09:15:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.481+0000 7f4ea007fb80 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:15:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.481+0000 7f4ea007fb80 20 client.16461 populate_metadata read hostname 'smithi099' 2023-08-25T09:15:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.481+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f7cd555a10 0x55f7cd557e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.481+0000 7f4ea007fb80 4 client.16461 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:15:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.481+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3356339717 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7cd42bb40 con 0x55f7cd555a10 2023-08-25T09:15:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.481+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f7cd555a10 0x55f7cd557e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.482+0000 7f4e6dffb700 10 client.16461 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.482+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f7cd555a10 0x55f7cd557e30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0362d1e88f7aec server_cookie=69cabaddd23b1149 in_seq=0 out_seq=0 2023-08-25T09:15:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3356339717 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f7cd42bb40 con 0x55f7cd555a10 2023-08-25T09:15:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4e6dffb700 10 client.16461 handle_command_reply: tid=0 2023-08-25T09:15:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 client.16461 shutdown 2023-08-25T09:15:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3356339717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd539860 msgr2=0x55f7cd53ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd539860 0x55f7cd53ce70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0045f0 tx=0x7f4e7c004ad0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3356339717 shutdown_connections 2023-08-25T09:15:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd539860 0x55f7cd53ce70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd53f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f7cd555a10 0x55f7cd557e30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3356339717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd538f40 0x55f7cd53c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.483+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3356339717 >> 172.21.15.99:0/3356339717 conn(0x55f7cd4a8080 msgr2=0x55f7cd528820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.484+0000 7f4e6dffb700 0 client.16461 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.484+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3356339717 shutdown_connections 2023-08-25T09:15:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.484+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3356339717 wait complete. 2023-08-25T09:15:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.485+0000 7f4ea007fb80 20 client.16461 trim_cache size 0 max 16384 2023-08-25T09:15:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.509+0000 7f4ea007fb80 1 Processor -- start 2023-08-25T09:15:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.510+0000 7f4ea007fb80 1 -- start start 2023-08-25T09:15:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.511+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd533450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.511+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd54cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.511+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 0x55f7cd53f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.511+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd3078c0 con 0x55f7cd538f40 2023-08-25T09:15:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.511+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd52f7a0 con 0x55f7cd539860 2023-08-25T09:15:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.511+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd52e280 con 0x55f7cd533990 2023-08-25T09:15:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.512+0000 7f4e8a59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd533450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.512+0000 7f4e8a59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd533450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:57082/0 (socket says 172.21.15.99:57082) 2023-08-25T09:15:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.512+0000 7f4e8a59b700 1 -- 172.21.15.99:0/2657189649 learned_addr learned my addr 172.21.15.99:0/2657189649 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.512+0000 7f4e8ad9c700 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 0x55f7cd53f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.513+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd54cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.514+0000 7f4e89d9a700 1 -- 172.21.15.99:0/2657189649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 msgr2=0x55f7cd533450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.514+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd533450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.514+0000 7f4e89d9a700 1 -- 172.21.15.99:0/2657189649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 msgr2=0x55f7cd53f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.514+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 0x55f7cd53f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.514+0000 7f4e89d9a700 1 -- 172.21.15.99:0/2657189649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7cd527b40 con 0x55f7cd538f40 2023-08-25T09:15:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.516+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd54cb00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4e740c9e00 tx=0x7f4e740962b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fc7facf253269b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.517+0000 7f4e797fa700 1 -- 172.21.15.99:0/2657189649 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7409a550 con 0x55f7cd538f40 2023-08-25T09:15:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.517+0000 7f4e797fa700 1 -- 172.21.15.99:0/2657189649 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e74099e50 con 0x55f7cd538f40 2023-08-25T09:15:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.517+0000 7f4e797fa700 1 -- 172.21.15.99:0/2657189649 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7409a0b0 con 0x55f7cd538f40 2023-08-25T09:15:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.523+0000 7f4ea007fb80 1 -- 172.21.15.99:0/2657189649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 msgr2=0x55f7cd54cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.523+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd54cb00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4e740c9e00 tx=0x7f4e740962b0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.523+0000 7f4ea007fb80 1 -- 172.21.15.99:0/2657189649 shutdown_connections 2023-08-25T09:15:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.523+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 0x55f7cd53f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.523+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd533990 0x55f7cd533450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.523+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/2657189649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd538f40 0x55f7cd54cb00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.523+0000 7f4ea007fb80 1 -- 172.21.15.99:0/2657189649 >> 172.21.15.99:0/2657189649 conn(0x55f7cd4a8080 msgr2=0x55f7cd536af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.525+0000 7f4ea007fb80 1 -- 172.21.15.99:0/2657189649 shutdown_connections 2023-08-25T09:15:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.525+0000 7f4ea007fb80 1 -- 172.21.15.99:0/2657189649 wait complete. 2023-08-25T09:15:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.526+0000 7f4ea007fb80 1 -- start start 2023-08-25T09:15:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.526+0000 7f4ea007fb80 1 Processor -- start 2023-08-25T09:15:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.527+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd533990 0x55f7cd535ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.527+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd538f40 0x55f7cd536000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.527+0000 7f4ea007fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 0x55f7cd534880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.527+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd534dc0 con 0x55f7cd533990 2023-08-25T09:15:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.527+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd534f30 con 0x55f7cd539860 2023-08-25T09:15:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.527+0000 7f4ea007fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cd535070 con 0x55f7cd538f40 2023-08-25T09:15:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.527+0000 7f4ea007fb80 10 client.0 _ll_register_callbacks cb 0x7f4e7c003e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:15:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.528+0000 7f4e8a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd533990 0x55f7cd535ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.528+0000 7f4e8a59b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd533990 0x55f7cd535ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44750/0 (socket says 172.21.15.99:44750) 2023-08-25T09:15:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.528+0000 7f4e8a59b700 1 -- 172.21.15.99:0/3434119026 learned_addr learned my addr 172.21.15.99:0/3434119026 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.528+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd538f40 0x55f7cd536000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.529+0000 7f4e8ad9c700 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 0x55f7cd534880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.530+0000 7f4e8a59b700 1 -- 172.21.15.99:0/3434119026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd538f40 msgr2=0x55f7cd536000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.530+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd538f40 0x55f7cd536000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.530+0000 7f4e8a59b700 1 -- 172.21.15.99:0/3434119026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 msgr2=0x55f7cd534880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.530+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 0x55f7cd534880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.530+0000 7f4e8a59b700 1 -- 172.21.15.99:0/3434119026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e841a5bd0 con 0x55f7cd533990 2023-08-25T09:15:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.532+0000 7f4e8a59b700 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd533990 0x55f7cd535ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4e80011a70 tx=0x7f4e80009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da76ef3202cb2b08 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.532+0000 7f4e6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-08-25T09:15:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.534+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3434119026 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80028b00 con 0x55f7cd533990 2023-08-25T09:15:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.534+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3434119026 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f4e80003e40 con 0x55f7cd533990 2023-08-25T09:15:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.534+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3434119026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7cd526600 con 0x55f7cd533990 2023-08-25T09:15:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.538+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3434119026 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e80028c60 con 0x55f7cd533990 2023-08-25T09:15:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.538+0000 7f4ea007fb80 10 client.16467 fetch_fsmap learned FSMap version 48 2023-08-25T09:15:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.538+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3434119026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55f7cd29ac00 con 0x55f7cd533990 2023-08-25T09:15:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.539+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3434119026 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7cd526600 con 0x55f7cd533990 2023-08-25T09:15:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.539+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3434119026 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4e80005500 con 0x55f7cd533990 2023-08-25T09:15:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.540+0000 7f4ea007fb80 10 client.16467 fetch_fsmap finished waiting for FSMap version 48 2023-08-25T09:15:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.541+0000 7f4ea007fb80 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:15:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.541+0000 7f4ea007fb80 20 client.16467 populate_metadata read hostname 'smithi099' 2023-08-25T09:15:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.541+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f7cd541120 0x55f7cd543540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.541+0000 7f4ea007fb80 4 client.16467 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:15:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.541+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3434119026 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f7cd47e910 con 0x55f7cd541120 2023-08-25T09:15:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.541+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f7cd541120 0x55f7cd543540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.549+0000 7f4e89d9a700 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f7cd541120 0x55f7cd543540 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3be2d2a4650bc6 server_cookie=876c2bb37fa4563 in_seq=0 out_seq=0 2023-08-25T09:15:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.549+0000 7f4e6dffb700 10 client.16467 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:03.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:03 smithi181 ceph-mon[131296]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 885 op/s 2023-08-25T09:15:03.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:03 smithi099 ceph-mon[97641]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 885 op/s 2023-08-25T09:15:03.885 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-mon[130879]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 885 op/s 2023-08-25T09:15:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.934+0000 7f4e6dffb700 1 -- 172.21.15.99:0/3434119026 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f7cd47e910 con 0x55f7cd541120 2023-08-25T09:15:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.934+0000 7f4e6dffb700 10 client.16467 handle_command_reply: tid=0 2023-08-25T09:15:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.934+0000 7f4ea007fb80 1 client.16467 shutdown 2023-08-25T09:15:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.939+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3434119026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd533990 msgr2=0x55f7cd535ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.939+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd533990 0x55f7cd535ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4e80011a70 tx=0x7f4e80009290 comp rx=0 tx=0).stop 2023-08-25T09:15:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.940+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3434119026 shutdown_connections 2023-08-25T09:15:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.940+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x55f7cd541120 0x55f7cd543540 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.940+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7cd539860 0x55f7cd534880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.940+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7cd533990 0x55f7cd535ac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.940+0000 7f4ea007fb80 1 --2- 172.21.15.99:0/3434119026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7cd538f40 0x55f7cd536000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.940+0000 7f4e6dffb700 0 client.16467 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.940+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3434119026 >> 172.21.15.99:0/3434119026 conn(0x55f7cd4a8080 msgr2=0x55f7cd539c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.941+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3434119026 shutdown_connections 2023-08-25T09:15:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.941+0000 7f4ea007fb80 1 -- 172.21.15.99:0/3434119026 wait complete. 2023-08-25T09:15:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.942+0000 7f4ea007fb80 20 client.16467 trim_cache size 0 max 16384 2023-08-25T09:15:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:15:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:15:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:15:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:15:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4e700a00c0 msgr2=0x7f4e700a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4e700a00c0 0x7f4e700a2580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009c10 tx=0x7f4e7c009920 comp rx=0 tx=0).stop 2023-08-25T09:15:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/221470299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e840686d0 msgr2=0x7f4e8419f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e840686d0 0x7f4e8419f120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009fd0 tx=0x7f4e7400f690 comp rx=0 tx=0).stop 2023-08-25T09:15:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/221470299 shutdown_connections 2023-08-25T09:15:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f4e700a00c0 0x7f4e700a2580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e841068d0 0x7f4e84198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84067d10 0x7f4e8419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 --2- 172.21.15.99:0/221470299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e840686d0 0x7f4e8419f120 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.943+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/221470299 >> 172.21.15.99:0/221470299 conn(0x7f4e84075250 msgr2=0x7f4e8410ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.944+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/221470299 shutdown_connections 2023-08-25T09:15:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:03.944+0000 7f4e8c7ff700 1 -- 172.21.15.99:0/221470299 wait complete. 2023-08-25T09:15:04.259 INFO:tasks.workunit.client.0.smithi099.stdout: 10 119517 35.25 MB/sec execute 69 sec latency 284.675 ms 2023-08-25T09:15:04.383 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-5703-gba6b39e4/rpm/el8/BUILD/ceph-18.0.0-5703-gba6b39e4/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f692f530700 time 2023-08-25T09:15:03.936679+0000 2023-08-25T09:15:04.384 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-5703-gba6b39e4/rpm/el8/BUILD/ceph-18.0.0-5703-gba6b39e4/src/mds/MDLog.cc: 1384: FAILED ceph_assert(journaler->is_readable() || mds->is_daemon_stopping()) 2023-08-25T09:15:04.385 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T09:15:04.385 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6940f14ed5] 2023-08-25T09:15:04.385 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ae09b) [0x7f6940f1509b] 2023-08-25T09:15:04.386 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 3: (MDLog::_replay_thread()+0x1a4e) [0x561f713530ce] 2023-08-25T09:15:04.386 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 4: (MDLog::ReplayThread::entry()+0x11) [0x561f7135b231] 2023-08-25T09:15:04.386 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f693fcb71ca] 2023-08-25T09:15:04.387 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 6: clone() 2023-08-25T09:15:04.387 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:03.935+0000 7f692f530700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-5703-gba6b39e4/rpm/el8/BUILD/ceph-18.0.0-5703-gba6b39e4/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f692f530700 time 2023-08-25T09:15:03.936679+0000 2023-08-25T09:15:04.387 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-5703-gba6b39e4/rpm/el8/BUILD/ceph-18.0.0-5703-gba6b39e4/src/mds/MDLog.cc: 1384: FAILED ceph_assert(journaler->is_readable() || mds->is_daemon_stopping()) 2023-08-25T09:15:04.388 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.388 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T09:15:04.388 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6940f14ed5] 2023-08-25T09:15:04.389 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ae09b) [0x7f6940f1509b] 2023-08-25T09:15:04.389 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 3: (MDLog::_replay_thread()+0x1a4e) [0x561f713530ce] 2023-08-25T09:15:04.389 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 4: (MDLog::ReplayThread::entry()+0x11) [0x561f7135b231] 2023-08-25T09:15:04.390 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f693fcb71ca] 2023-08-25T09:15:04.390 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 6: clone() 2023-08-25T09:15:04.390 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.391 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: *** Caught signal (Aborted) ** 2023-08-25T09:15:04.391 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: in thread 7f692f530700 thread_name:md_log_replay 2023-08-25T09:15:04.392 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T09:15:04.393 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f693fcc1cf0] 2023-08-25T09:15:04.393 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2: gsignal() 2023-08-25T09:15:04.393 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 3: abort() 2023-08-25T09:15:04.393 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6940f14f2f] 2023-08-25T09:15:04.393 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ae09b) [0x7f6940f1509b] 2023-08-25T09:15:04.393 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 6: (MDLog::_replay_thread()+0x1a4e) [0x561f713530ce] 2023-08-25T09:15:04.394 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 7: (MDLog::ReplayThread::entry()+0x11) [0x561f7135b231] 2023-08-25T09:15:04.394 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f693fcb71ca] 2023-08-25T09:15:04.394 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 9: clone() 2023-08-25T09:15:04.394 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:03.936+0000 7f692f530700 -1 *** Caught signal (Aborted) ** 2023-08-25T09:15:04.394 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: in thread 7f692f530700 thread_name:md_log_replay 2023-08-25T09:15:04.394 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.395 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T09:15:04.395 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f693fcc1cf0] 2023-08-25T09:15:04.395 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2: gsignal() 2023-08-25T09:15:04.395 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 3: abort() 2023-08-25T09:15:04.395 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6940f14f2f] 2023-08-25T09:15:04.395 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ae09b) [0x7f6940f1509b] 2023-08-25T09:15:04.395 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 6: (MDLog::_replay_thread()+0x1a4e) [0x561f713530ce] 2023-08-25T09:15:04.396 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 7: (MDLog::ReplayThread::entry()+0x11) [0x561f7135b231] 2023-08-25T09:15:04.396 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f693fcb71ca] 2023-08-25T09:15:04.396 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 9: clone() 2023-08-25T09:15:04.396 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-08-25T09:15:04.396 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.396 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: -1> 2023-08-25T09:15:03.935+0000 7f692f530700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-5703-gba6b39e4/rpm/el8/BUILD/ceph-18.0.0-5703-gba6b39e4/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f692f530700 time 2023-08-25T09:15:03.936679+0000 2023-08-25T09:15:04.397 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-5703-gba6b39e4/rpm/el8/BUILD/ceph-18.0.0-5703-gba6b39e4/src/mds/MDLog.cc: 1384: FAILED ceph_assert(journaler->is_readable() || mds->is_daemon_stopping()) 2023-08-25T09:15:04.397 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.397 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T09:15:04.398 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6940f14ed5] 2023-08-25T09:15:04.398 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ae09b) [0x7f6940f1509b] 2023-08-25T09:15:04.398 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 3: (MDLog::_replay_thread()+0x1a4e) [0x561f713530ce] 2023-08-25T09:15:04.398 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 4: (MDLog::ReplayThread::entry()+0x11) [0x561f7135b231] 2023-08-25T09:15:04.398 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f693fcb71ca] 2023-08-25T09:15:04.398 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 6: clone() 2023-08-25T09:15:04.398 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.399 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 0> 2023-08-25T09:15:03.936+0000 7f692f530700 -1 *** Caught signal (Aborted) ** 2023-08-25T09:15:04.399 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: in thread 7f692f530700 thread_name:md_log_replay 2023-08-25T09:15:04.399 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.399 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T09:15:04.399 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f693fcc1cf0] 2023-08-25T09:15:04.399 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2: gsignal() 2023-08-25T09:15:04.400 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 3: abort() 2023-08-25T09:15:04.400 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6940f14f2f] 2023-08-25T09:15:04.400 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ae09b) [0x7f6940f1509b] 2023-08-25T09:15:04.400 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 6: (MDLog::_replay_thread()+0x1a4e) [0x561f713530ce] 2023-08-25T09:15:04.400 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 7: (MDLog::ReplayThread::entry()+0x11) [0x561f7135b231] 2023-08-25T09:15:04.401 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f693fcb71ca] 2023-08-25T09:15:04.401 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 9: clone() 2023-08-25T09:15:04.401 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-08-25T09:15:04.401 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.401 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: -1> 2023-08-25T09:15:03.935+0000 7f692f530700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-5703-gba6b39e4/rpm/el8/BUILD/ceph-18.0.0-5703-gba6b39e4/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f692f530700 time 2023-08-25T09:15:03.936679+0000 2023-08-25T09:15:04.401 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-5703-gba6b39e4/rpm/el8/BUILD/ceph-18.0.0-5703-gba6b39e4/src/mds/MDLog.cc: 1384: FAILED ceph_assert(journaler->is_readable() || mds->is_daemon_stopping()) 2023-08-25T09:15:04.402 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.402 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T09:15:04.402 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6940f14ed5] 2023-08-25T09:15:04.402 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ae09b) [0x7f6940f1509b] 2023-08-25T09:15:04.402 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 3: (MDLog::_replay_thread()+0x1a4e) [0x561f713530ce] 2023-08-25T09:15:04.402 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 4: (MDLog::ReplayThread::entry()+0x11) [0x561f7135b231] 2023-08-25T09:15:04.403 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f693fcb71ca] 2023-08-25T09:15:04.403 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 6: clone() 2023-08-25T09:15:04.403 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.403 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 0> 2023-08-25T09:15:03.936+0000 7f692f530700 -1 *** Caught signal (Aborted) ** 2023-08-25T09:15:04.403 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: in thread 7f692f530700 thread_name:md_log_replay 2023-08-25T09:15:04.403 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.404 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T09:15:04.404 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f693fcc1cf0] 2023-08-25T09:15:04.404 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2: gsignal() 2023-08-25T09:15:04.404 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 3: abort() 2023-08-25T09:15:04.404 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6940f14f2f] 2023-08-25T09:15:04.404 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ae09b) [0x7f6940f1509b] 2023-08-25T09:15:04.405 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 6: (MDLog::_replay_thread()+0x1a4e) [0x561f713530ce] 2023-08-25T09:15:04.405 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 7: (MDLog::ReplayThread::entry()+0x11) [0x561f7135b231] 2023-08-25T09:15:04.405 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f693fcb71ca] 2023-08-25T09:15:04.405 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 9: clone() 2023-08-25T09:15:04.405 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-08-25T09:15:04.405 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[155565]: 2023-08-25T09:15:04.510 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:04 smithi157 systemd[1]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.h.service: Main process exited, code=exited, status=134/n/a 2023-08-25T09:15:04.789 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:04 smithi157 ceph-mon[130879]: osdmap e108: 12 total, 12 up, 12 in 2023-08-25T09:15:04.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:04 smithi099 ceph-mon[97641]: osdmap e108: 12 total, 12 up, 12 in 2023-08-25T09:15:04.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:04 smithi181 ceph-mon[131296]: osdmap e108: 12 total, 12 up, 12 in 2023-08-25T09:15:05.130 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:04 smithi157 systemd[1]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.h.service: Failed with result 'exit-code'. 2023-08-25T09:15:05.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 119880 35.04 MB/sec execute 70 sec latency 99.702 ms 2023-08-25T09:15:05.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:05 smithi099 ceph-mon[97641]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 699 op/s 2023-08-25T09:15:05.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:05 smithi157 ceph-mon[130879]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 699 op/s 2023-08-25T09:15:05.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:05 smithi181 ceph-mon[131296]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 699 op/s 2023-08-25T09:15:06.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 119989 34.70 MB/sec execute 71 sec latency 744.850 ms 2023-08-25T09:15:06.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:06 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:06.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:06 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:07.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:06 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:07.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 120220 34.38 MB/sec execute 72 sec latency 816.530 ms 2023-08-25T09:15:07.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:07 smithi157 ceph-mon[130879]: pgmap v501: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:07.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:07 smithi181 ceph-mon[131296]: pgmap v501: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:08.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:07 smithi099 ceph-mon[97641]: pgmap v501: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:08.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 120706 34.38 MB/sec execute 73 sec latency 1147.165 ms 2023-08-25T09:15:09.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 121221 34.17 MB/sec execute 74 sec latency 2147.205 ms 2023-08-25T09:15:09.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:09 smithi157 ceph-mon[130879]: pgmap v502: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:09.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:09 smithi181 ceph-mon[131296]: pgmap v502: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:10.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:09 smithi099 ceph-mon[97641]: pgmap v502: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:10.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 121756 34.10 MB/sec execute 75 sec latency 3147.244 ms 2023-08-25T09:15:11.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 122404 34.17 MB/sec execute 76 sec latency 4147.298 ms 2023-08-25T09:15:11.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:11 smithi157 ceph-mon[130879]: pgmap v503: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:11.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:11 smithi181 ceph-mon[131296]: pgmap v503: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:12.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:11 smithi099 ceph-mon[97641]: pgmap v503: 97 pgs: 25 active+clean+snaptrim_wait, 61 active+clean, 11 active+clean+snaptrim; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 994 op/s 2023-08-25T09:15:12.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 122907 34.08 MB/sec execute 77 sec latency 5147.339 ms 2023-08-25T09:15:13.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 123451 33.97 MB/sec execute 78 sec latency 6147.392 ms 2023-08-25T09:15:13.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:13 smithi157 ceph-mon[130879]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-08-25T09:15:13.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:13 smithi181 ceph-mon[131296]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-08-25T09:15:14.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:13 smithi099 ceph-mon[97641]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-08-25T09:15:14.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 124105 33.98 MB/sec execute 79 sec latency 7147.472 ms 2023-08-25T09:15:15.214 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:14 smithi157 systemd[1]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.h.service: Service RestartSec=10s expired, scheduling restart. 2023-08-25T09:15:15.214 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:14 smithi157 systemd[1]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.h.service: Scheduled restart job, restart counter is at 1. 2023-08-25T09:15:15.214 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:14 smithi157 systemd[1]: Stopped Ceph mds.h for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T09:15:15.214 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:14 smithi157 systemd[1]: Starting Ceph mds.h for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:15:15.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 124789 34.08 MB/sec execute 80 sec latency 8147.542 ms 2023-08-25T09:15:15.489 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:15 smithi157 ceph-mon[130879]: osdmap e109: 12 total, 12 up, 12 in 2023-08-25T09:15:15.489 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:15 smithi157 ceph-mon[130879]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-08-25T09:15:15.489 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:15 smithi157 podman[162803]: 2023-08-25T09:15:15.490 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:15 smithi157 bash[162803]: 919f074458a8f17a460b0cda3ea5611be2cb8c12fb72eb5069f5cd967a00d70b 2023-08-25T09:15:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:15 smithi181 ceph-mon[131296]: osdmap e109: 12 total, 12 up, 12 in 2023-08-25T09:15:15.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:15 smithi181 ceph-mon[131296]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-08-25T09:15:15.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:15 smithi099 ceph-mon[97641]: osdmap e109: 12 total, 12 up, 12 in 2023-08-25T09:15:15.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:15 smithi099 ceph-mon[97641]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-08-25T09:15:15.880 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:15 smithi157 ceph-mds[162844]: -- 172.21.15.157:0/2261825609 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b209191e0 con 0x559b208eec00 2023-08-25T09:15:15.880 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:15 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[162818]: starting mds.h at 2023-08-25T09:15:15.880 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:15:15 smithi157 systemd[1]: Started Ceph mds.h for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T09:15:16.260 INFO:tasks.workunit.client.0.smithi099.stdout: 10 125226 33.91 MB/sec execute 81 sec latency 9147.593 ms 2023-08-25T09:15:16.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:16 smithi157 ceph-mon[130879]: daemon mds.h restarted 2023-08-25T09:15:16.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:16 smithi181 ceph-mon[131296]: daemon mds.h restarted 2023-08-25T09:15:16.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:16 smithi099 ceph-mon[97641]: daemon mds.h restarted 2023-08-25T09:15:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.129+0000 7fe21efff700 1 -- 172.21.15.99:0/2761777641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2180735f0 msgr2=0x7fe21810cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.129+0000 7fe21efff700 1 --2- 172.21.15.99:0/2761777641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2180735f0 0x7fe21810cc30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe208009990 tx=0x7fe208009ca0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.129+0000 7fe21efff700 1 -- 172.21.15.99:0/2761777641 shutdown_connections 2023-08-25T09:15:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.129+0000 7fe21efff700 1 --2- 172.21.15.99:0/2761777641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21810d170 0x7fe21810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.129+0000 7fe21efff700 1 --2- 172.21.15.99:0/2761777641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe218072cd0 0x7fe2180730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.129+0000 7fe21efff700 1 --2- 172.21.15.99:0/2761777641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2180735f0 0x7fe21810cc30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.129+0000 7fe21efff700 1 -- 172.21.15.99:0/2761777641 >> 172.21.15.99:0/2761777641 conn(0x7fe2180fc380 msgr2=0x7fe2180fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.131+0000 7fe21efff700 1 -- 172.21.15.99:0/2761777641 shutdown_connections 2023-08-25T09:15:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.131+0000 7fe21efff700 1 -- 172.21.15.99:0/2761777641 wait complete. 2023-08-25T09:15:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.132+0000 7fe21efff700 1 Processor -- start 2023-08-25T09:15:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.132+0000 7fe21efff700 1 -- start start 2023-08-25T09:15:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21efff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe218072cd0 0x7fe21819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21efff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2180735f0 0x7fe21819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21efff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe21810d170 0x7fe218198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21efff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2181990b0 con 0x7fe2180735f0 2023-08-25T09:15:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21efff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe218199220 con 0x7fe218072cd0 2023-08-25T09:15:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21efff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2181993e0 con 0x7fe21810d170 2023-08-25T09:15:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe218072cd0 0x7fe21819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe218072cd0 0x7fe21819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47208/0 (socket says 172.21.15.99:47208) 2023-08-25T09:15:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21cd9b700 1 -- 172.21.15.99:0/1605529994 learned_addr learned my addr 172.21.15.99:0/1605529994 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe21d59c700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe21810d170 0x7fe218198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.133+0000 7fe20ffff700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2180735f0 0x7fe21819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe21cd9b700 1 -- 172.21.15.99:0/1605529994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe21810d170 msgr2=0x7fe218198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe21810d170 0x7fe218198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe21cd9b700 1 -- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2180735f0 msgr2=0x7fe21819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2180735f0 0x7fe21819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe21cd9b700 1 -- 172.21.15.99:0/1605529994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080095f0 con 0x7fe218072cd0 2023-08-25T09:15:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe20ffff700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2180735f0 0x7fe21819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:15:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe218072cd0 0x7fe21819ebe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe21400e9a0 tx=0x7fe21400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe20dffb700 1 -- 172.21.15.99:0/1605529994 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214009790 con 0x7fe218072cd0 2023-08-25T09:15:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe21efff700 1 -- 172.21.15.99:0/1605529994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2181996c0 con 0x7fe218072cd0 2023-08-25T09:15:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe20dffb700 1 -- 172.21.15.99:0/1605529994 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21400cbd0 con 0x7fe218072cd0 2023-08-25T09:15:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe20dffb700 1 -- 172.21.15.99:0/1605529994 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214018680 con 0x7fe218072cd0 2023-08-25T09:15:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.134+0000 7fe21efff700 1 -- 172.21.15.99:0/1605529994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2181a5bd0 con 0x7fe218072cd0 2023-08-25T09:15:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.139+0000 7fe20dffb700 1 -- 172.21.15.99:0/1605529994 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fe21400cd40 con 0x7fe218072cd0 2023-08-25T09:15:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.140+0000 7fe23296db80 1 Processor -- start 2023-08-25T09:15:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe23296db80 1 -- start start 2023-08-25T09:15:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe20dffb700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe2000a0140 0x7fe2000a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe23296db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe23296db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b495450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe23296db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b495990 0x56177b495dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe23296db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b3fd750 con 0x56177b495990 2023-08-25T09:15:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe23296db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b3ee720 con 0x56177b49af40 2023-08-25T09:15:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe23296db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b3ec040 con 0x56177b49b860 2023-08-25T09:15:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.141+0000 7fe20dffb700 1 -- 172.21.15.99:0/1605529994 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe214014070 con 0x7fe218072cd0 2023-08-25T09:15:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b495450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47224/0 (socket says 172.21.15.99:47224) 2023-08-25T09:15:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 -- 172.21.15.99:0/1985802708 learned_addr learned my addr 172.21.15.99:0/1985802708 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b495450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:49270/0 (socket says 172.21.15.99:49270) 2023-08-25T09:15:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21d59c700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe2000a0140 0x7fe2000a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 -- 172.21.15.99:0/1985802708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 msgr2=0x56177b495450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21d59c700 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b495990 0x56177b495dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b495450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 -- 172.21.15.99:0/1985802708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b495990 msgr2=0x56177b495dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b495990 0x56177b495dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 -- 172.21.15.99:0/1985802708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56177b2675d0 con 0x56177b49af40 2023-08-25T09:15:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21d59c700 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b495990 0x56177b495dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:15:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.142+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49b320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2140a0870 tx=0x7fe214098ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36edfacaa8211b53 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.143+0000 7fe21d59c700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe2000a0140 0x7fe2000a2600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe210006130 tx=0x7fe210006060 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.143+0000 7fe1feffd700 1 -- 172.21.15.99:0/1985802708 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214098f90 con 0x56177b49af40 2023-08-25T09:15:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.143+0000 7fe1feffd700 1 -- 172.21.15.99:0/1985802708 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21409cdc0 con 0x56177b49af40 2023-08-25T09:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.143+0000 7fe1feffd700 1 -- 172.21.15.99:0/1985802708 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214098480 con 0x56177b49af40 2023-08-25T09:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 -- 172.21.15.99:0/1985802708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 msgr2=0x56177b49b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49b320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2140a0870 tx=0x7fe214098ad0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 -- 172.21.15.99:0/1985802708 shutdown_connections 2023-08-25T09:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b495990 0x56177b495dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49b320 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 --2- 172.21.15.99:0/1985802708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b495450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 -- 172.21.15.99:0/1985802708 >> 172.21.15.99:0/1985802708 conn(0x56177b40a080 msgr2=0x56177b48fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 -- 172.21.15.99:0/1985802708 shutdown_connections 2023-08-25T09:15:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.144+0000 7fe23296db80 1 -- 172.21.15.99:0/1985802708 wait complete. 2023-08-25T09:15:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.145+0000 7fe23296db80 1 -- start start 2023-08-25T09:15:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.145+0000 7fe23296db80 1 Processor -- start 2023-08-25T09:15:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe23296db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b495990 0x56177b4a1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe23296db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe23296db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe23296db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b3e7930 con 0x56177b49b860 2023-08-25T09:15:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe23296db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b385b50 con 0x56177b49af40 2023-08-25T09:15:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe23296db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b385850 con 0x56177b495990 2023-08-25T09:15:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe23296db80 10 client.0 _ll_register_callbacks cb 0x56177b2cfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:15:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe21cd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b495990 0x56177b4a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47230/0 (socket says 172.21.15.99:47230) 2023-08-25T09:15:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.146+0000 7fe20ffff700 1 -- 172.21.15.99:0/3088202326 learned_addr learned my addr 172.21.15.99:0/3088202326 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.147+0000 7fe21d59c700 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.147+0000 7fe21cd9b700 1 -- 172.21.15.99:0/3088202326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 msgr2=0x56177b49e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.147+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.147+0000 7fe21cd9b700 1 -- 172.21.15.99:0/3088202326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 msgr2=0x56177b49ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.147+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.147+0000 7fe21cd9b700 1 -- 172.21.15.99:0/3088202326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2181a5bd0 con 0x56177b495990 2023-08-25T09:15:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b495990 0x56177b4a1650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe214010bc0 tx=0x7fe214099420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4ca744dbc1ffdc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe1f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:15:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3088202326 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21409cdc0 con 0x56177b495990 2023-08-25T09:15:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe23296db80 1 -- 172.21.15.99:0/3088202326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56177b465160 con 0x56177b495990 2023-08-25T09:15:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3088202326 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe214098f90 con 0x56177b495990 2023-08-25T09:15:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3088202326 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2140b1150 con 0x56177b495990 2023-08-25T09:15:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe23296db80 10 client.26536 fetch_fsmap learned FSMap version 51 2023-08-25T09:15:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe23296db80 1 -- 172.21.15.99:0/3088202326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x56177b213d80 con 0x56177b495990 2023-08-25T09:15:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3088202326 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56177b465160 con 0x56177b495990 2023-08-25T09:15:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.148+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3088202326 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 51) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe214094e10 con 0x56177b495990 2023-08-25T09:15:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.149+0000 7fe23296db80 10 client.26536 fetch_fsmap finished waiting for FSMap version 51 2023-08-25T09:15:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.149+0000 7fe23296db80 10 client.26536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:15:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.149+0000 7fe23296db80 20 client.26536 populate_metadata read hostname 'smithi099' 2023-08-25T09:15:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.149+0000 7fe23296db80 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56177b4aeda0 0x56177b4b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.149+0000 7fe23296db80 4 client.26536 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:15:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.149+0000 7fe23296db80 1 -- 172.21.15.99:0/3088202326 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56177b38db40 con 0x56177b4aeda0 2023-08-25T09:15:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.149+0000 7fe20ffff700 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56177b4aeda0 0x56177b4b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.150+0000 7fe20ffff700 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56177b4aeda0 0x56177b4b11c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8965bb9ac81039 server_cookie=81eec2303c6502b2 in_seq=0 out_seq=0 2023-08-25T09:15:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.150+0000 7fe1f7fff700 10 client.26536 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.151+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3088202326 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56177b38db40 con 0x56177b4aeda0 2023-08-25T09:15:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.151+0000 7fe1f7fff700 10 client.26536 handle_command_reply: tid=0 2023-08-25T09:15:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.151+0000 7fe23296db80 1 client.26536 shutdown 2023-08-25T09:15:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 -- 172.21.15.99:0/3088202326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b495990 msgr2=0x56177b4a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b495990 0x56177b4a1650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe214010bc0 tx=0x7fe214099420 comp rx=0 tx=0).stop 2023-08-25T09:15:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 -- 172.21.15.99:0/3088202326 shutdown_connections 2023-08-25T09:15:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b495990 0x56177b4a1650 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56177b4aeda0 0x56177b4b11c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 --2- 172.21.15.99:0/3088202326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b49af40 0x56177b49e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 -- 172.21.15.99:0/3088202326 >> 172.21.15.99:0/3088202326 conn(0x56177b40a080 msgr2=0x56177b48a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe1f7fff700 0 client.26536 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 -- 172.21.15.99:0/3088202326 shutdown_connections 2023-08-25T09:15:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.152+0000 7fe23296db80 1 -- 172.21.15.99:0/3088202326 wait complete. 2023-08-25T09:15:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.153+0000 7fe23296db80 20 client.26536 trim_cache size 0 max 16384 2023-08-25T09:15:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe23296db80 1 Processor -- start 2023-08-25T09:15:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe23296db80 1 -- start start 2023-08-25T09:15:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe23296db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b495450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe23296db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49af40 0x56177b4a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe23296db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe23296db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b2698c0 con 0x56177b49b860 2023-08-25T09:15:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe23296db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b49feb0 con 0x56177b495990 2023-08-25T09:15:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe23296db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b49b320 con 0x56177b49af40 2023-08-25T09:15:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49af40 0x56177b4a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe21cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b495450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49af40 0x56177b4a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:49280/0 (socket says 172.21.15.99:49280) 2023-08-25T09:15:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe20ffff700 1 -- 172.21.15.99:0/4213806441 learned_addr learned my addr 172.21.15.99:0/4213806441 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.172+0000 7fe21d59c700 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe21d59c700 1 -- 172.21.15.99:0/4213806441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49af40 msgr2=0x56177b4a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe21d59c700 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49af40 0x56177b4a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe21d59c700 1 -- 172.21.15.99:0/4213806441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 msgr2=0x56177b495450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe21d59c700 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b495450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe21d59c700 1 -- 172.21.15.99:0/4213806441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56177b1fcc00 con 0x56177b49b860 2023-08-25T09:15:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b495450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:15:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe20ffff700 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49af40 0x56177b4a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:15:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe21d59c700 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49e090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe21819f8e0 tx=0x7fe21000c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155b7e8d7e3554f6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.173+0000 7fe1ff7fe700 1 -- 172.21.15.99:0/4213806441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210017800 con 0x56177b49b860 2023-08-25T09:15:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe1ff7fe700 1 -- 172.21.15.99:0/4213806441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21001d040 con 0x56177b49b860 2023-08-25T09:15:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe1ff7fe700 1 -- 172.21.15.99:0/4213806441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210023dc0 con 0x56177b49b860 2023-08-25T09:15:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 -- 172.21.15.99:0/4213806441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 msgr2=0x56177b49e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49e090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe21819f8e0 tx=0x7fe21000c070 comp rx=0 tx=0).stop 2023-08-25T09:15:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 -- 172.21.15.99:0/4213806441 shutdown_connections 2023-08-25T09:15:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49b860 0x56177b49e090 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b495450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 --2- 172.21.15.99:0/4213806441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49af40 0x56177b4a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 -- 172.21.15.99:0/4213806441 >> 172.21.15.99:0/4213806441 conn(0x56177b40a080 msgr2=0x56177b48fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 -- 172.21.15.99:0/4213806441 shutdown_connections 2023-08-25T09:15:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.174+0000 7fe23296db80 1 -- 172.21.15.99:0/4213806441 wait complete. 2023-08-25T09:15:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.175+0000 7fe23296db80 1 -- start start 2023-08-25T09:15:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.175+0000 7fe23296db80 1 Processor -- start 2023-08-25T09:15:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe23296db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b497ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe23296db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49af40 0x56177b496880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe23296db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b496dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe23296db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b497300 con 0x56177b49af40 2023-08-25T09:15:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe23296db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b497440 con 0x56177b495990 2023-08-25T09:15:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe23296db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177b497600 con 0x56177b49b860 2023-08-25T09:15:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe23296db80 10 client.0 _ll_register_callbacks cb 0x7fe208009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:15:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b497ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21cd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b497ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:47254/0 (socket says 172.21.15.99:47254) 2023-08-25T09:15:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21cd9b700 1 -- 172.21.15.99:0/3118248064 learned_addr learned my addr 172.21.15.99:0/3118248064 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21d59c700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b496dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe20ffff700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49af40 0x56177b496880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21cd9b700 1 -- 172.21.15.99:0/3118248064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 msgr2=0x56177b496dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b496dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21cd9b700 1 -- 172.21.15.99:0/3118248064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49af40 msgr2=0x56177b496880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49af40 0x56177b496880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21cd9b700 1 -- 172.21.15.99:0/3118248064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56177b213d80 con 0x56177b495990 2023-08-25T09:15:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe21d59c700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b496dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:15:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.176+0000 7fe20ffff700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49af40 0x56177b496880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:15:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.177+0000 7fe21cd9b700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b497ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe214095490 tx=0x7fe21400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685b1364c541bbd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.177+0000 7fe1f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:15:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.177+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3118248064 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140b1150 con 0x56177b495990 2023-08-25T09:15:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.177+0000 7fe23296db80 1 -- 172.21.15.99:0/3118248064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56177b488600 con 0x56177b495990 2023-08-25T09:15:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.177+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3118248064 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe214098f90 con 0x56177b495990 2023-08-25T09:15:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.177+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3118248064 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe214099490 con 0x56177b495990 2023-08-25T09:15:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.177+0000 7fe23296db80 10 client.26570 fetch_fsmap learned FSMap version 51 2023-08-25T09:15:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.177+0000 7fe23296db80 1 -- 172.21.15.99:0/3118248064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fe218110220 con 0x56177b495990 2023-08-25T09:15:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3118248064 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56177b488600 con 0x56177b495990 2023-08-25T09:15:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3118248064 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 51) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe2140b80b0 con 0x56177b495990 2023-08-25T09:15:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe23296db80 10 client.26570 fetch_fsmap finished waiting for FSMap version 51 2023-08-25T09:15:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe23296db80 10 client.26570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:15:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe23296db80 20 client.26570 populate_metadata read hostname 'smithi099' 2023-08-25T09:15:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe23296db80 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56177b503bc0 0x56177b505fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe23296db80 4 client.26570 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:15:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe23296db80 1 -- 172.21.15.99:0/3118248064 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56177b3e0910 con 0x56177b503bc0 2023-08-25T09:15:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.178+0000 7fe20ffff700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56177b503bc0 0x56177b505fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.179+0000 7fe20ffff700 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56177b503bc0 0x56177b505fb0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682f92fdf5fa2a89 server_cookie=12db3c1d23228c60 in_seq=0 out_seq=0 2023-08-25T09:15:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:17.179+0000 7fe1f7fff700 10 client.26570 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:17.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 125711 33.72 MB/sec execute 82 sec latency 10148.188 ms 2023-08-25T09:15:17.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:17 smithi157 ceph-mon[130879]: osdmap e110: 12 total, 12 up, 12 in 2023-08-25T09:15:17.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:17 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6836/985582767,v1:172.21.15.157:6837/985582767] up:boot 2023-08-25T09:15:17.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:17 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T09:15:17.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:17 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:15:17.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:17 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:17.631 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:17 smithi157 ceph-mon[130879]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.8 MiB/s wr, 734 op/s 2023-08-25T09:15:17.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:17 smithi181 ceph-mon[131296]: osdmap e110: 12 total, 12 up, 12 in 2023-08-25T09:15:17.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:17 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6836/985582767,v1:172.21.15.157:6837/985582767] up:boot 2023-08-25T09:15:17.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:17 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T09:15:17.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:17 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:15:17.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:17 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:17.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:17 smithi181 ceph-mon[131296]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.8 MiB/s wr, 734 op/s 2023-08-25T09:15:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:17 smithi099 ceph-mon[97641]: osdmap e110: 12 total, 12 up, 12 in 2023-08-25T09:15:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:17 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6836/985582767,v1:172.21.15.157:6837/985582767] up:boot 2023-08-25T09:15:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:17 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-08-25T09:15:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:17 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:15:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:17 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:17.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:17 smithi099 ceph-mon[97641]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.8 MiB/s wr, 734 op/s 2023-08-25T09:15:18.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 126489 33.88 MB/sec execute 83 sec latency 10891.605 ms 2023-08-25T09:15:19.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 127174 33.98 MB/sec execute 84 sec latency 8278.443 ms 2023-08-25T09:15:19.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:19 smithi181 ceph-mon[131296]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.8 MiB/s wr, 734 op/s 2023-08-25T09:15:19.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:19 smithi099 ceph-mon[97641]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.8 MiB/s wr, 734 op/s 2023-08-25T09:15:19.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:19 smithi157 ceph-mon[130879]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.8 MiB/s wr, 734 op/s 2023-08-25T09:15:20.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 127624 33.86 MB/sec execute 85 sec latency 8749.198 ms 2023-08-25T09:15:21.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 128094 33.68 MB/sec execute 86 sec latency 7220.822 ms 2023-08-25T09:15:21.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:21 smithi181 ceph-mon[131296]: pgmap v510: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 296 op/s 2023-08-25T09:15:21.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:21 smithi099 ceph-mon[97641]: pgmap v510: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 296 op/s 2023-08-25T09:15:21.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:21 smithi157 ceph-mon[130879]: pgmap v510: 97 pgs: 97 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 296 op/s 2023-08-25T09:15:22.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 128873 33.68 MB/sec execute 87 sec latency 8220.864 ms 2023-08-25T09:15:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.914+0000 7fe1f7fff700 1 -- 172.21.15.99:0/3118248064 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56177b3e0910 con 0x56177b503bc0 2023-08-25T09:15:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.914+0000 7fe1f7fff700 10 client.26570 handle_command_reply: tid=0 2023-08-25T09:15:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.914+0000 7fe23296db80 1 client.26570 shutdown 2023-08-25T09:15:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 -- 172.21.15.99:0/3118248064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 msgr2=0x56177b497ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b497ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe214095490 tx=0x7fe21400c6e0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 -- 172.21.15.99:0/3118248064 shutdown_connections 2023-08-25T09:15:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56177b49b860 0x56177b496dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x56177b503bc0 0x56177b505fb0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177b495990 0x56177b497ef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 --2- 172.21.15.99:0/3118248064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177b49af40 0x56177b496880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 -- 172.21.15.99:0/3118248064 >> 172.21.15.99:0/3118248064 conn(0x56177b40a080 msgr2=0x56177b498a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe1f7fff700 0 client.26570 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 -- 172.21.15.99:0/3118248064 shutdown_connections 2023-08-25T09:15:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 1 -- 172.21.15.99:0/3118248064 wait complete. 2023-08-25T09:15:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.915+0000 7fe23296db80 20 client.26570 trim_cache size 0 max 16384 2023-08-25T09:15:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:15:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:15:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:15:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:15:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.919+0000 7fe21efff700 1 -- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe2000a0140 msgr2=0x7fe2000a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.919+0000 7fe21efff700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe2000a0140 0x7fe2000a2600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe210006130 tx=0x7fe210006060 comp rx=0 tx=0).stop 2023-08-25T09:15:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.919+0000 7fe21efff700 1 -- 172.21.15.99:0/1605529994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe218072cd0 msgr2=0x7fe21819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.919+0000 7fe21efff700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe218072cd0 0x7fe21819ebe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe21400e9a0 tx=0x7fe21400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.920+0000 7fe21efff700 1 -- 172.21.15.99:0/1605529994 shutdown_connections 2023-08-25T09:15:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.920+0000 7fe21efff700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe2000a0140 0x7fe2000a2600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.920+0000 7fe21efff700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe21810d170 0x7fe218198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.920+0000 7fe21efff700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe218072cd0 0x7fe21819ebe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.920+0000 7fe21efff700 1 --2- 172.21.15.99:0/1605529994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2180735f0 0x7fe21819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.920+0000 7fe21efff700 1 -- 172.21.15.99:0/1605529994 >> 172.21.15.99:0/1605529994 conn(0x7fe2180fc380 msgr2=0x7fe21810f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.920+0000 7fe21efff700 1 -- 172.21.15.99:0/1605529994 shutdown_connections 2023-08-25T09:15:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:22.920+0000 7fe21efff700 1 -- 172.21.15.99:0/1605529994 wait complete. 2023-08-25T09:15:23.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 129685 33.94 MB/sec execute 88 sec latency 8856.020 ms 2023-08-25T09:15:23.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:23 smithi099 ceph-mon[97641]: pgmap v511: 97 pgs: 97 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 677 op/s 2023-08-25T09:15:23.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:23 smithi157 ceph-mon[130879]: pgmap v511: 97 pgs: 97 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 677 op/s 2023-08-25T09:15:23.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:23 smithi181 ceph-mon[131296]: pgmap v511: 97 pgs: 97 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 677 op/s 2023-08-25T09:15:24.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 130228 33.92 MB/sec execute 89 sec latency 65.587 ms 2023-08-25T09:15:24.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:24 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:24.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:24 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:24.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:24 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:25.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 130749 33.78 MB/sec execute 90 sec latency 122.468 ms 2023-08-25T09:15:25.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:25 smithi099 ceph-mon[97641]: pgmap v512: 97 pgs: 97 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 548 op/s 2023-08-25T09:15:25.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:25 smithi157 ceph-mon[130879]: pgmap v512: 97 pgs: 97 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 548 op/s 2023-08-25T09:15:25.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:25 smithi181 ceph-mon[131296]: pgmap v512: 97 pgs: 97 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 548 op/s 2023-08-25T09:15:26.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 131409 33.89 MB/sec execute 91 sec latency 139.394 ms 2023-08-25T09:15:27.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 131743 33.64 MB/sec execute 92 sec latency 477.180 ms 2023-08-25T09:15:27.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:27 smithi099 ceph-mon[97641]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 540 op/s 2023-08-25T09:15:27.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:27 smithi157 ceph-mon[130879]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 540 op/s 2023-08-25T09:15:27.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:27 smithi181 ceph-mon[131296]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 540 op/s 2023-08-25T09:15:28.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 132313 33.56 MB/sec execute 93 sec latency 142.766 ms 2023-08-25T09:15:29.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 133062 33.68 MB/sec execute 94 sec latency 136.672 ms 2023-08-25T09:15:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:29 smithi099 ceph-mon[97641]: pgmap v514: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-08-25T09:15:29.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:29 smithi157 ceph-mon[130879]: pgmap v514: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-08-25T09:15:29.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:29 smithi181 ceph-mon[131296]: pgmap v514: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-08-25T09:15:30.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 133806 33.81 MB/sec execute 95 sec latency 159.253 ms 2023-08-25T09:15:31.262 INFO:tasks.workunit.client.0.smithi099.stdout: 10 134336 33.68 MB/sec execute 96 sec latency 86.174 ms 2023-08-25T09:15:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:31 smithi099 ceph-mon[97641]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-08-25T09:15:31.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:31 smithi157 ceph-mon[130879]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-08-25T09:15:31.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:31 smithi181 ceph-mon[131296]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-08-25T09:15:32.261 INFO:tasks.workunit.client.0.smithi099.stdout: 10 134530 33.43 MB/sec execute 97 sec latency 587.110 ms 2023-08-25T09:15:33.262 INFO:tasks.workunit.client.0.smithi099.stdout: 10 134804 33.24 MB/sec execute 98 sec latency 947.523 ms 2023-08-25T09:15:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:33 smithi099 ceph-mon[97641]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 702 op/s 2023-08-25T09:15:33.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:33 smithi157 ceph-mon[130879]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 702 op/s 2023-08-25T09:15:33.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:33 smithi181 ceph-mon[131296]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 702 op/s 2023-08-25T09:15:34.262 INFO:tasks.workunit.client.0.smithi099.stdout: 10 135457 33.28 MB/sec execute 99 sec latency 107.280 ms 2023-08-25T09:15:35.262 INFO:tasks.workunit.client.0.smithi099.stdout: 10 136191 33.37 MB/sec execute 100 sec latency 138.197 ms 2023-08-25T09:15:35.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:35 smithi157 ceph-mon[130879]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 442 op/s 2023-08-25T09:15:35.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:35 smithi181 ceph-mon[131296]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 442 op/s 2023-08-25T09:15:35.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:35 smithi099 ceph-mon[97641]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 442 op/s 2023-08-25T09:15:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 -- 172.21.15.99:0/655592289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 msgr2=0x7fd9d8102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/655592289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 0x7fd9d8102f40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009990 tx=0x7fd9cc009ca0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 -- 172.21.15.99:0/655592289 shutdown_connections 2023-08-25T09:15:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/655592289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9d8103480 0x7fd9d810f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/655592289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9d8108ad0 0x7fd9d8108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/655592289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 0x7fd9d8102f40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 -- 172.21.15.99:0/655592289 >> 172.21.15.99:0/655592289 conn(0x7fd9d80fe590 msgr2=0x7fd9d81009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 -- 172.21.15.99:0/655592289 shutdown_connections 2023-08-25T09:15:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.099+0000 7fd9de7ff700 1 -- 172.21.15.99:0/655592289 wait complete. 2023-08-25T09:15:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.100+0000 7fd9de7ff700 1 Processor -- start 2023-08-25T09:15:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.100+0000 7fd9de7ff700 1 -- start start 2023-08-25T09:15:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9de7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 0x7fd9d819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9de7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9d8103480 0x7fd9d819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9de7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9d8108ad0 0x7fd9d81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9de7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8199060 con 0x7fd9d8108ad0 2023-08-25T09:15:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9de7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81991d0 con 0x7fd9d8102ac0 2023-08-25T09:15:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9de7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8199390 con 0x7fd9d8103480 2023-08-25T09:15:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9d8103480 0x7fd9d819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 0x7fd9d819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 0x7fd9d819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:56120/0 (socket says 172.21.15.99:56120) 2023-08-25T09:15:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9d8103480 0x7fd9d819f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:42554/0 (socket says 172.21.15.99:42554) 2023-08-25T09:15:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d7fff700 1 -- 172.21.15.99:0/2582072312 learned_addr learned my addr 172.21.15.99:0/2582072312 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9dcd9c700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9d8108ad0 0x7fd9d81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d7fff700 1 -- 172.21.15.99:0/2582072312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9d8103480 msgr2=0x7fd9d819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9d8103480 0x7fd9d819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d7fff700 1 -- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9d8108ad0 msgr2=0x7fd9d81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9d8108ad0 0x7fd9d81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.101+0000 7fd9d7fff700 1 -- 172.21.15.99:0/2582072312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0095f0 con 0x7fd9d8102ac0 2023-08-25T09:15:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.102+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 0x7fd9d819eb90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800e9a0 tx=0x7fd9c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.102+0000 7fd9d57fa700 1 -- 172.21.15.99:0/2582072312 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8009790 con 0x7fd9d8102ac0 2023-08-25T09:15:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.102+0000 7fd9de7ff700 1 -- 172.21.15.99:0/2582072312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d8199670 con 0x7fd9d8102ac0 2023-08-25T09:15:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.102+0000 7fd9d57fa700 1 -- 172.21.15.99:0/2582072312 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9c8010430 con 0x7fd9d8102ac0 2023-08-25T09:15:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.102+0000 7fd9d57fa700 1 -- 172.21.15.99:0/2582072312 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8018680 con 0x7fd9d8102ac0 2023-08-25T09:15:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.102+0000 7fd9de7ff700 1 -- 172.21.15.99:0/2582072312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d81a5b80 con 0x7fd9d8102ac0 2023-08-25T09:15:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.106+0000 7fd9d57fa700 1 -- 172.21.15.99:0/2582072312 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fd9c80105a0 con 0x7fd9d8102ac0 2023-08-25T09:15:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.107+0000 7fd9f2071b80 1 Processor -- start 2023-08-25T09:15:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.107+0000 7fd9f2071b80 1 -- start start 2023-08-25T09:15:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1111320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f110b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f110bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f1073750 con 0x5587f1111860 2023-08-25T09:15:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f1064720 con 0x5587f1110f40 2023-08-25T09:15:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f1062040 con 0x5587f110b990 2023-08-25T09:15:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9dcd9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1111320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f110bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f110b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f110bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:42568/0 (socket says 172.21.15.99:42568) 2023-08-25T09:15:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d77fe700 1 -- 172.21.15.99:0/1057295287 learned_addr learned my addr 172.21.15.99:0/1057295287 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d77fe700 1 -- 172.21.15.99:0/1057295287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 msgr2=0x5587f1111320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/1057295287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1111320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d77fe700 1 -- 172.21.15.99:0/1057295287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 msgr2=0x5587f110b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/1057295287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f110b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.108+0000 7fd9d77fe700 1 -- 172.21.15.99:0/1057295287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587f0edd5d0 con 0x5587f110b990 2023-08-25T09:15:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.109+0000 7fd9d57fa700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd9c00a00c0 0x7fd9c00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.109+0000 7fd9d57fa700 1 -- 172.21.15.99:0/2582072312 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd9c8014070 con 0x7fd9d8102ac0 2023-08-25T09:15:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.109+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/1057295287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f110bdd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc015000 tx=0x7fd9cc005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b175dabe9b292d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.109+0000 7fd9dcd9c700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd9c00a00c0 0x7fd9c00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.109+0000 7fd9be7fc700 1 -- 172.21.15.99:0/1057295287 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc01d070 con 0x5587f110b990 2023-08-25T09:15:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.109+0000 7fd9be7fc700 1 -- 172.21.15.99:0/1057295287 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9cc01e040 con 0x5587f110b990 2023-08-25T09:15:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.109+0000 7fd9be7fc700 1 -- 172.21.15.99:0/1057295287 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc004880 con 0x5587f110b990 2023-08-25T09:15:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9dcd9c700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd9c00a00c0 0x7fd9c00a2580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9d819f890 tx=0x7fd9d000b040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1057295287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 msgr2=0x5587f110bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/1057295287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f110bdd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc015000 tx=0x7fd9cc005ea0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1057295287 shutdown_connections 2023-08-25T09:15:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/1057295287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f110bdd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/1057295287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/1057295287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f110b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1057295287 >> 172.21.15.99:0/1057295287 conn(0x5587f1080080 msgr2=0x5587f1105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1057295287 shutdown_connections 2023-08-25T09:15:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.110+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1057295287 wait complete. 2023-08-25T09:15:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.111+0000 7fd9f2071b80 1 -- start start 2023-08-25T09:15:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.111+0000 7fd9f2071b80 1 Processor -- start 2023-08-25T09:15:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1117570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f11147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1114d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f105d930 con 0x5587f1111860 2023-08-25T09:15:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f0ffbb50 con 0x5587f1110f40 2023-08-25T09:15:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f0ffb850 con 0x5587f110b990 2023-08-25T09:15:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9f2071b80 10 client.0 _ll_register_callbacks cb 0x5587f0f45a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:15:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1117570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f11147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1117570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:42570/0 (socket says 172.21.15.99:42570) 2023-08-25T09:15:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d7fff700 1 -- 172.21.15.99:0/2552235410 learned_addr learned my addr 172.21.15.99:0/2552235410 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9dcd9c700 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1114d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d7fff700 1 -- 172.21.15.99:0/2552235410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 msgr2=0x5587f11147f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f11147f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d7fff700 1 -- 172.21.15.99:0/2552235410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 msgr2=0x5587f1114d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1114d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.112+0000 7fd9d7fff700 1 -- 172.21.15.99:0/2552235410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d8110290 con 0x5587f110b990 2023-08-25T09:15:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1117570 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8095900 tx=0x7fd9c80c9430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2fca492c5423467 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:15:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9b77fe700 1 -- 172.21.15.99:0/2552235410 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8098580 con 0x5587f110b990 2023-08-25T09:15:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9b77fe700 1 -- 172.21.15.99:0/2552235410 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd9c8099590 con 0x5587f110b990 2023-08-25T09:15:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9f2071b80 1 -- 172.21.15.99:0/2552235410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587f10db160 con 0x5587f110b990 2023-08-25T09:15:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9b77fe700 1 -- 172.21.15.99:0/2552235410 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9c80989d0 con 0x5587f110b990 2023-08-25T09:15:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9f2071b80 10 client.26554 fetch_fsmap learned FSMap version 52 2023-08-25T09:15:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9f2071b80 1 -- 172.21.15.99:0/2552235410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5587f0e89d80 con 0x5587f110b990 2023-08-25T09:15:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.113+0000 7fd9b77fe700 1 -- 172.21.15.99:0/2552235410 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587f10db160 con 0x5587f110b990 2023-08-25T09:15:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.114+0000 7fd9b77fe700 1 -- 172.21.15.99:0/2552235410 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd9c80b81d0 con 0x5587f110b990 2023-08-25T09:15:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.114+0000 7fd9f2071b80 10 client.26554 fetch_fsmap finished waiting for FSMap version 52 2023-08-25T09:15:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.114+0000 7fd9f2071b80 10 client.26554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:15:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.114+0000 7fd9f2071b80 20 client.26554 populate_metadata read hostname 'smithi099' 2023-08-25T09:15:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.114+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5587f1125060 0x5587f1127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.114+0000 7fd9f2071b80 4 client.26554 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:15:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.114+0000 7fd9f2071b80 1 -- 172.21.15.99:0/2552235410 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587f1003b40 con 0x5587f1125060 2023-08-25T09:15:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.114+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5587f1125060 0x5587f1127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.115+0000 7fd9b77fe700 10 client.26554 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.115+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5587f1125060 0x5587f1127450 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a9d77a825ca1d1 server_cookie=1f8fc2fd305677ce in_seq=0 out_seq=0 2023-08-25T09:15:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9b77fe700 1 -- 172.21.15.99:0/2552235410 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5587f1003b40 con 0x5587f1125060 2023-08-25T09:15:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9b77fe700 10 client.26554 handle_command_reply: tid=0 2023-08-25T09:15:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 client.26554 shutdown 2023-08-25T09:15:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 -- 172.21.15.99:0/2552235410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 msgr2=0x5587f1117570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1117570 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8095900 tx=0x7fd9c80c9430 comp rx=0 tx=0).stop 2023-08-25T09:15:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 -- 172.21.15.99:0/2552235410 shutdown_connections 2023-08-25T09:15:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1114d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1117570 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5587f1125060 0x5587f1127450 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/2552235410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f11147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 -- 172.21.15.99:0/2552235410 >> 172.21.15.99:0/2552235410 conn(0x5587f1080080 msgr2=0x5587f1100820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9b77fe700 0 client.26554 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 -- 172.21.15.99:0/2552235410 shutdown_connections 2023-08-25T09:15:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.118+0000 7fd9f2071b80 1 -- 172.21.15.99:0/2552235410 wait complete. 2023-08-25T09:15:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.119+0000 7fd9f2071b80 20 client.26554 trim_cache size 0 max 16384 2023-08-25T09:15:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.137+0000 7fd9f2071b80 1 Processor -- start 2023-08-25T09:15:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.137+0000 7fd9f2071b80 1 -- start start 2023-08-25T09:15:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.137+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f110b990 0x5587f110b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.137+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f1110f40 0x5587f111c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.137+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1117110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.137+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f0edf8c0 con 0x5587f1111860 2023-08-25T09:15:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f111c780 con 0x5587f110b990 2023-08-25T09:15:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f1106f70 con 0x5587f1110f40 2023-08-25T09:15:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f1110f40 0x5587f111c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f1110f40 0x5587f111c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:42582/0 (socket says 172.21.15.99:42582) 2023-08-25T09:15:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 -- 172.21.15.99:0/1673774720 learned_addr learned my addr 172.21.15.99:0/1673774720 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9dcd9c700 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1117110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 -- 172.21.15.99:0/1673774720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f110b990 msgr2=0x5587f110b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f110b990 0x5587f110b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 -- 172.21.15.99:0/1673774720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 msgr2=0x5587f1117110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1117110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 -- 172.21.15.99:0/1673774720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587f10ffb40 con 0x5587f1110f40 2023-08-25T09:15:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9dcd9c700 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:15:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.138+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f1110f40 0x5587f111c240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc000c00 tx=0x7fd9cc01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd1ba1388d8038c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9beffd700 1 -- 172.21.15.99:0/1673774720 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc032070 con 0x5587f1110f40 2023-08-25T09:15:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9beffd700 1 -- 172.21.15.99:0/1673774720 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9cc01e040 con 0x5587f1110f40 2023-08-25T09:15:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9beffd700 1 -- 172.21.15.99:0/1673774720 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc01d040 con 0x5587f1110f40 2023-08-25T09:15:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1673774720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f1110f40 msgr2=0x5587f111c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f1110f40 0x5587f111c240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc000c00 tx=0x7fd9cc01a040 comp rx=0 tx=0).stop 2023-08-25T09:15:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1673774720 shutdown_connections 2023-08-25T09:15:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f110b990 0x5587f110b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/1673774720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f1110f40 0x5587f111c240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.139+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1673774720 >> 172.21.15.99:0/1673774720 conn(0x5587f1080080 msgr2=0x5587f110e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.140+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1673774720 shutdown_connections 2023-08-25T09:15:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.140+0000 7fd9f2071b80 1 -- 172.21.15.99:0/1673774720 wait complete. 2023-08-25T09:15:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.140+0000 7fd9f2071b80 1 -- start start 2023-08-25T09:15:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.140+0000 7fd9f2071b80 1 Processor -- start 2023-08-25T09:15:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1118220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1118760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9f2071b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1177e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f1118f60 con 0x5587f1111860 2023-08-25T09:15:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f11190d0 con 0x5587f1110f40 2023-08-25T09:15:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9f2071b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f1178410 con 0x5587f110b990 2023-08-25T09:15:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9f2071b80 10 client.0 _ll_register_callbacks cb 0x5587f0f45a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:15:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1118220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1118760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1118220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:42598/0 (socket says 172.21.15.99:42598) 2023-08-25T09:15:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9d7fff700 1 -- 172.21.15.99:0/544218954 learned_addr learned my addr 172.21.15.99:0/544218954 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1118760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:56140/0 (socket says 172.21.15.99:56140) 2023-08-25T09:15:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.141+0000 7fd9dcd9c700 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1177e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9d7fff700 1 -- 172.21.15.99:0/544218954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 msgr2=0x5587f1118760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1118760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9d7fff700 1 -- 172.21.15.99:0/544218954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 msgr2=0x5587f1177e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1177e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9d7fff700 1 -- 172.21.15.99:0/544218954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587f0edd5d0 con 0x5587f110b990 2023-08-25T09:15:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1118760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:15:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9d7fff700 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1118220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8095390 tx=0x7fd9c8094eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634c9736658605b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:15:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9b77fe700 1 -- 172.21.15.99:0/544218954 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8099ea0 con 0x5587f110b990 2023-08-25T09:15:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9f2071b80 1 -- 172.21.15.99:0/544218954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587f10fe600 con 0x5587f110b990 2023-08-25T09:15:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9b77fe700 1 -- 172.21.15.99:0/544218954 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd9c80b8320 con 0x5587f110b990 2023-08-25T09:15:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9b77fe700 1 -- 172.21.15.99:0/544218954 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9c80989d0 con 0x5587f110b990 2023-08-25T09:15:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9f2071b80 10 client.26560 fetch_fsmap learned FSMap version 52 2023-08-25T09:15:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.142+0000 7fd9f2071b80 1 -- 172.21.15.99:0/544218954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5587f0e72c00 con 0x5587f110b990 2023-08-25T09:15:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9b77fe700 1 -- 172.21.15.99:0/544218954 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587f10fe600 con 0x5587f110b990 2023-08-25T09:15:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9b77fe700 1 -- 172.21.15.99:0/544218954 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd9c8097ff0 con 0x5587f110b990 2023-08-25T09:15:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9f2071b80 10 client.26560 fetch_fsmap finished waiting for FSMap version 52 2023-08-25T09:15:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9f2071b80 10 client.26560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:15:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9f2071b80 20 client.26560 populate_metadata read hostname 'smithi099' 2023-08-25T09:15:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5587f1179860 0x5587f117bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9f2071b80 4 client.26560 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:15:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9f2071b80 1 -- 172.21.15.99:0/544218954 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5587f1056910 con 0x5587f1179860 2023-08-25T09:15:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.143+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5587f1179860 0x5587f117bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.144+0000 7fd9d77fe700 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5587f1179860 0x5587f117bc50 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44c2a6eb7bb00fb4 server_cookie=d34a0dad786b25df in_seq=0 out_seq=0 2023-08-25T09:15:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.144+0000 7fd9b77fe700 10 client.26560 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:36.264 INFO:tasks.workunit.client.0.smithi099.stdout: 10 136740 33.31 MB/sec execute 101 sec latency 152.892 ms 2023-08-25T09:15:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9b77fe700 1 -- 172.21.15.99:0/544218954 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5587f1056910 con 0x5587f1179860 2023-08-25T09:15:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9b77fe700 10 client.26560 handle_command_reply: tid=0 2023-08-25T09:15:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 client.26560 shutdown 2023-08-25T09:15:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 -- 172.21.15.99:0/544218954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 msgr2=0x5587f1118220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1118220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8095390 tx=0x7fd9c8094eb0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 -- 172.21.15.99:0/544218954 shutdown_connections 2023-08-25T09:15:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587f1111860 0x5587f1177e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f110b990 0x5587f1118220 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5587f1179860 0x5587f117bc50 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 --2- 172.21.15.99:0/544218954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587f1110f40 0x5587f1118760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 -- 172.21.15.99:0/544218954 >> 172.21.15.99:0/544218954 conn(0x5587f1080080 msgr2=0x5587f10fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9b77fe700 0 client.26560 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 -- 172.21.15.99:0/544218954 shutdown_connections 2023-08-25T09:15:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.379+0000 7fd9f2071b80 1 -- 172.21.15.99:0/544218954 wait complete. 2023-08-25T09:15:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9f2071b80 20 client.26560 trim_cache size 0 max 16384 2023-08-25T09:15:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:15:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:15:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:15:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:15:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 -- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd9c00a00c0 msgr2=0x7fd9c00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd9c00a00c0 0x7fd9c00a2580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9d819f890 tx=0x7fd9d000b040 comp rx=0 tx=0).stop 2023-08-25T09:15:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 -- 172.21.15.99:0/2582072312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 msgr2=0x7fd9d819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 0x7fd9d819eb90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800e9a0 tx=0x7fd9c800ecb0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 -- 172.21.15.99:0/2582072312 shutdown_connections 2023-08-25T09:15:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fd9c00a00c0 0x7fd9c00a2580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9d8108ad0 0x7fd9d81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8102ac0 0x7fd9d819eb90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 --2- 172.21.15.99:0/2582072312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9d8103480 0x7fd9d819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.380+0000 7fd9de7ff700 1 -- 172.21.15.99:0/2582072312 >> 172.21.15.99:0/2582072312 conn(0x7fd9d80fe590 msgr2=0x7fd9d81065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.381+0000 7fd9de7ff700 1 -- 172.21.15.99:0/2582072312 shutdown_connections 2023-08-25T09:15:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:36.381+0000 7fd9de7ff700 1 -- 172.21.15.99:0/2582072312 wait complete. 2023-08-25T09:15:37.264 INFO:tasks.workunit.client.0.smithi099.stdout: 10 137341 33.26 MB/sec execute 102 sec latency 279.684 ms 2023-08-25T09:15:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:37 smithi099 ceph-mon[97641]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 893 op/s 2023-08-25T09:15:37.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:37 smithi157 ceph-mon[130879]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 893 op/s 2023-08-25T09:15:37.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:37 smithi181 ceph-mon[131296]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 893 op/s 2023-08-25T09:15:38.264 INFO:tasks.workunit.client.0.smithi099.stdout: 10 137979 33.32 MB/sec execute 103 sec latency 147.132 ms 2023-08-25T09:15:39.264 INFO:tasks.workunit.client.0.smithi099.stdout: 10 138682 33.34 MB/sec execute 104 sec latency 114.695 ms 2023-08-25T09:15:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:39 smithi099 ceph-mon[97641]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-08-25T09:15:39.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:39 smithi157 ceph-mon[130879]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-08-25T09:15:39.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:39 smithi181 ceph-mon[131296]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-08-25T09:15:40.264 INFO:tasks.workunit.client.0.smithi099.stdout: 10 139249 33.29 MB/sec execute 105 sec latency 189.332 ms 2023-08-25T09:15:41.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 140036 33.46 MB/sec execute 106 sec latency 103.211 ms 2023-08-25T09:15:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:41 smithi099 ceph-mon[97641]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-08-25T09:15:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:41 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:41 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:15:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:41 smithi157 ceph-mon[130879]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-08-25T09:15:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:41 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:41 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:15:41.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:41 smithi181 ceph-mon[131296]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-08-25T09:15:41.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:41 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:41.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:41 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:15:42.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 140464 33.32 MB/sec execute 107 sec latency 137.409 ms 2023-08-25T09:15:43.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 141006 33.25 MB/sec execute 108 sec latency 153.360 ms 2023-08-25T09:15:43.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:43 smithi099 ceph-mon[97641]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 945 op/s 2023-08-25T09:15:43.903 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:43 smithi157 ceph-mon[130879]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 945 op/s 2023-08-25T09:15:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:43 smithi181 ceph-mon[131296]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 945 op/s 2023-08-25T09:15:44.264 INFO:tasks.workunit.client.0.smithi099.stdout: 10 141729 33.35 MB/sec execute 109 sec latency 93.708 ms 2023-08-25T09:15:45.264 INFO:tasks.workunit.client.0.smithi099.stdout: 10 142364 33.33 MB/sec execute 110 sec latency 129.383 ms 2023-08-25T09:15:45.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:45 smithi157 ceph-mon[130879]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 697 op/s 2023-08-25T09:15:45.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:45 smithi181 ceph-mon[131296]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 697 op/s 2023-08-25T09:15:45.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:45 smithi099 ceph-mon[97641]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 697 op/s 2023-08-25T09:15:46.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 143034 33.41 MB/sec execute 111 sec latency 124.399 ms 2023-08-25T09:15:46.629 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:46 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:15:46.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:46 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:15:46.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:46 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:15:46.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:46 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:15:46.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:46 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:15:46.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:46 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:15:46.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:46 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:15:46.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:46 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:15:46.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:46 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:15:47.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 143580 33.32 MB/sec execute 112 sec latency 186.507 ms 2023-08-25T09:15:47.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:47 smithi157 ceph-mon[130879]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 875 op/s 2023-08-25T09:15:47.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:47 smithi181 ceph-mon[131296]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 875 op/s 2023-08-25T09:15:47.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:47 smithi099 ceph-mon[97641]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 875 op/s 2023-08-25T09:15:48.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 144250 33.33 MB/sec execute 113 sec latency 102.811 ms 2023-08-25T09:15:49.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 144994 33.46 MB/sec execute 114 sec latency 160.880 ms 2023-08-25T09:15:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 -- 172.21.15.99:0/3430103674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380067d10 msgr2=0x7fa380068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3430103674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380067d10 0x7fa380068190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa374009920 tx=0x7fa374009c30 comp rx=0 tx=0).stop 2023-08-25T09:15:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 -- 172.21.15.99:0/3430103674 shutdown_connections 2023-08-25T09:15:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3430103674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3800686d0 0x7fa38010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3430103674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3801068a0 0x7fa380106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3430103674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380067d10 0x7fa380068190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 -- 172.21.15.99:0/3430103674 >> 172.21.15.99:0/3430103674 conn(0x7fa380075050 msgr2=0x7fa380075460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 -- 172.21.15.99:0/3430103674 shutdown_connections 2023-08-25T09:15:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.553+0000 7fa384f9d700 1 -- 172.21.15.99:0/3430103674 wait complete. 2023-08-25T09:15:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.555+0000 7fa384f9d700 1 Processor -- start 2023-08-25T09:15:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.556+0000 7fa384f9d700 1 -- start start 2023-08-25T09:15:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.556+0000 7fa384f9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380067d10 0x7fa38006aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.556+0000 7fa384f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800686d0 0x7fa38006b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.556+0000 7fa384f9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3801068a0 0x7fa3801aac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.556+0000 7fa384f9d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800ff850 con 0x7fa3801068a0 2023-08-25T09:15:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.556+0000 7fa384f9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800ff9c0 con 0x7fa380067d10 2023-08-25T09:15:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.556+0000 7fa384f9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800ffb80 con 0x7fa3800686d0 2023-08-25T09:15:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.557+0000 7fa37ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3801068a0 0x7fa3801aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.557+0000 7fa37ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3801068a0 0x7fa3801aac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39016/0 (socket says 172.21.15.99:39016) 2023-08-25T09:15:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.557+0000 7fa37ffff700 1 -- 172.21.15.99:0/3672829786 learned_addr learned my addr 172.21.15.99:0/3672829786 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.557+0000 7fa37ffff700 1 -- 172.21.15.99:0/3672829786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800686d0 msgr2=0x7fa38006b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.557+0000 7fa37ffff700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800686d0 0x7fa38006b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.557+0000 7fa37ffff700 1 -- 172.21.15.99:0/3672829786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380067d10 msgr2=0x7fa38006aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.557+0000 7fa37ffff700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380067d10 0x7fa38006aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.557+0000 7fa37ffff700 1 -- 172.21.15.99:0/3672829786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374009520 con 0x7fa3801068a0 2023-08-25T09:15:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.558+0000 7fa37ffff700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3801068a0 0x7fa3801aac90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa37800ef30 tx=0x7fa37800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.558+0000 7fa37cff9700 1 -- 172.21.15.99:0/3672829786 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378018070 con 0x7fa3801068a0 2023-08-25T09:15:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.558+0000 7fa37cff9700 1 -- 172.21.15.99:0/3672829786 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37800f040 con 0x7fa3801068a0 2023-08-25T09:15:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.558+0000 7fa37cff9700 1 -- 172.21.15.99:0/3672829786 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378004020 con 0x7fa3801068a0 2023-08-25T09:15:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.558+0000 7fa384f9d700 1 -- 172.21.15.99:0/3672829786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3800ffe00 con 0x7fa3801068a0 2023-08-25T09:15:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.559+0000 7fa384f9d700 1 -- 172.21.15.99:0/3672829786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3801002c0 con 0x7fa3801068a0 2023-08-25T09:15:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.564+0000 7fa37cff9700 1 -- 172.21.15.99:0/3672829786 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa378004180 con 0x7fa3801068a0 2023-08-25T09:15:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.565+0000 7fa37cff9700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa3680a0140 0x7fa3680a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.565+0000 7fa37cff9700 1 -- 172.21.15.99:0/3672829786 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa3780cb150 con 0x7fa3801068a0 2023-08-25T09:15:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.566+0000 7fa399a7fb80 1 Processor -- start 2023-08-25T09:15:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.567+0000 7fa399a7fb80 1 -- start start 2023-08-25T09:15:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.567+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b7010 0x558c9e9b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.567+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9e9b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.567+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b1990 0x558c9e9b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.567+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e919750 con 0x558c9e9b7010 2023-08-25T09:15:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.567+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e90a720 con 0x558c9e9b1990 2023-08-25T09:15:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.567+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e908040 con 0x558c9e9b7930 2023-08-25T09:15:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.568+0000 7fa37effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9e9b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.568+0000 7fa37effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9e9b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:44736/0 (socket says 172.21.15.99:44736) 2023-08-25T09:15:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.568+0000 7fa37effd700 1 -- 172.21.15.99:0/2427919690 learned_addr learned my addr 172.21.15.99:0/2427919690 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.569+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa3680a0140 0x7fa3680a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.569+0000 7fa37effd700 1 -- 172.21.15.99:0/2427919690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b1990 msgr2=0x558c9e9b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.569+0000 7fa37effd700 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b1990 0x558c9e9b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.569+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b7010 0x558c9e9b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.569+0000 7fa37effd700 1 -- 172.21.15.99:0/2427919690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b7010 msgr2=0x558c9e9b73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.570+0000 7fa37effd700 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b7010 0x558c9e9b73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.570+0000 7fa37effd700 1 -- 172.21.15.99:0/2427919690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c9e7835d0 con 0x558c9e9b7930 2023-08-25T09:15:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.570+0000 7fa37effd700 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9e9b1450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa3740094f0 tx=0x7fa3740058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29c8ad4dc4b2341 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.570+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b7010 0x558c9e9b73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:15:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.570+0000 7fa365ffb700 1 -- 172.21.15.99:0/2427919690 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37401c070 con 0x558c9e9b7930 2023-08-25T09:15:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.571+0000 7fa365ffb700 1 -- 172.21.15.99:0/2427919690 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37400f920 con 0x558c9e9b7930 2023-08-25T09:15:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.571+0000 7fa365ffb700 1 -- 172.21.15.99:0/2427919690 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740053b0 con 0x558c9e9b7930 2023-08-25T09:15:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.572+0000 7fa399a7fb80 1 -- 172.21.15.99:0/2427919690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 msgr2=0x558c9e9b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.572+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9e9b1450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa3740094f0 tx=0x7fa3740058e0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.572+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa3680a0140 0x7fa3680a2600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa3700095a0 tx=0x7fa370009530 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.573+0000 7fa399a7fb80 1 -- 172.21.15.99:0/2427919690 shutdown_connections 2023-08-25T09:15:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.573+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b1990 0x558c9e9b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.573+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b7010 0x558c9e9b73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.573+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/2427919690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9e9b1450 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.573+0000 7fa399a7fb80 1 -- 172.21.15.99:0/2427919690 >> 172.21.15.99:0/2427919690 conn(0x558c9e926080 msgr2=0x558c9e9abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.573+0000 7fa399a7fb80 1 -- 172.21.15.99:0/2427919690 shutdown_connections 2023-08-25T09:15:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.573+0000 7fa399a7fb80 1 -- 172.21.15.99:0/2427919690 wait complete. 2023-08-25T09:15:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.574+0000 7fa399a7fb80 1 -- start start 2023-08-25T09:15:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.574+0000 7fa399a7fb80 1 Processor -- start 2023-08-25T09:15:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9bd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 0x558c9e9bb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e903930 con 0x558c9e9b1990 2023-08-25T09:15:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e8a1b50 con 0x558c9e9b7930 2023-08-25T09:15:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e8a1850 con 0x558c9e9b7010 2023-08-25T09:15:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa399a7fb80 10 client.0 _ll_register_callbacks cb 0x558c9e7eba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:15:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa37effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa37effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:44744/0 (socket says 172.21.15.99:44744) 2023-08-25T09:15:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.575+0000 7fa37effd700 1 -- 172.21.15.99:0/102307588 learned_addr learned my addr 172.21.15.99:0/102307588 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.576+0000 7fa37effd700 1 -- 172.21.15.99:0/102307588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 msgr2=0x558c9e9bb2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T09:15:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.576+0000 7fa37effd700 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 0x558c9e9bb2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.576+0000 7fa37effd700 1 -- 172.21.15.99:0/102307588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 msgr2=0x558c9e9bd8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T09:15:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.576+0000 7fa37effd700 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9bd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.576+0000 7fa37effd700 1 -- 172.21.15.99:0/102307588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38010de90 con 0x558c9e9b7010 2023-08-25T09:15:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.577+0000 7fa37effd700 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bab70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa374005e90 tx=0x7fa37400f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6163a8ee266903e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.577+0000 7fa35effd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:15:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.578+0000 7fa35effd700 1 -- 172.21.15.99:0/102307588 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37401c040 con 0x558c9e9b7010 2023-08-25T09:15:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.578+0000 7fa35effd700 1 -- 172.21.15.99:0/102307588 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa3740054f0 con 0x558c9e9b7010 2023-08-25T09:15:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.578+0000 7fa399a7fb80 1 -- 172.21.15.99:0/102307588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c9e981160 con 0x558c9e9b7010 2023-08-25T09:15:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.578+0000 7fa35effd700 1 -- 172.21.15.99:0/102307588 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa37400ba00 con 0x558c9e9b7010 2023-08-25T09:15:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.578+0000 7fa399a7fb80 10 client.26569 fetch_fsmap learned FSMap version 53 2023-08-25T09:15:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa399a7fb80 1 -- 172.21.15.99:0/102307588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x558c9e72fd80 con 0x558c9e9b7010 2023-08-25T09:15:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa35effd700 1 -- 172.21.15.99:0/102307588 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c9e981160 con 0x558c9e9b7010 2023-08-25T09:15:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa35effd700 1 -- 172.21.15.99:0/102307588 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa374030070 con 0x558c9e9b7010 2023-08-25T09:15:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa399a7fb80 10 client.26569 fetch_fsmap finished waiting for FSMap version 53 2023-08-25T09:15:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa399a7fb80 10 client.26569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:15:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa399a7fb80 20 client.26569 populate_metadata read hostname 'smithi099' 2023-08-25T09:15:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558c9e9dbe40 0x558c9e9de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa399a7fb80 4 client.26569 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:15:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa399a7fb80 1 -- 172.21.15.99:0/102307588 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c9e8a9b40 con 0x558c9e9dbe40 2023-08-25T09:15:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.579+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558c9e9dbe40 0x558c9e9de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.580+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558c9e9dbe40 0x558c9e9de260 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33801aab0a server_cookie=f539c5bf72e7f588 in_seq=0 out_seq=0 2023-08-25T09:15:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.580+0000 7fa35effd700 10 client.26569 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa35effd700 1 -- 172.21.15.99:0/102307588 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558c9e8a9b40 con 0x558c9e9dbe40 2023-08-25T09:15:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa35effd700 10 client.26569 handle_command_reply: tid=0 2023-08-25T09:15:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 client.26569 shutdown 2023-08-25T09:15:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 -- 172.21.15.99:0/102307588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 msgr2=0x558c9e9bab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bab70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa374005e90 tx=0x7fa37400f720 comp rx=0 tx=0).stop 2023-08-25T09:15:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 -- 172.21.15.99:0/102307588 shutdown_connections 2023-08-25T09:15:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558c9e9dbe40 0x558c9e9de260 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 0x558c9e9bb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9bd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/102307588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bab70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 -- 172.21.15.99:0/102307588 >> 172.21.15.99:0/102307588 conn(0x558c9e926080 msgr2=0x558c9e9b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa35effd700 0 client.26569 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 -- 172.21.15.99:0/102307588 shutdown_connections 2023-08-25T09:15:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.581+0000 7fa399a7fb80 1 -- 172.21.15.99:0/102307588 wait complete. 2023-08-25T09:15:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.582+0000 7fa399a7fb80 20 client.26569 trim_cache size 0 max 16384 2023-08-25T09:15:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.598+0000 7fa399a7fb80 1 Processor -- start 2023-08-25T09:15:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.598+0000 7fa399a7fb80 1 -- start start 2023-08-25T09:15:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 0x558c9e9abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e7858c0 con 0x558c9e9b1990 2023-08-25T09:15:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e9acad0 con 0x558c9e9b7930 2023-08-25T09:15:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9e9ad270 con 0x558c9e9b7010 2023-08-25T09:15:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 0x558c9e9abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:44760/0 (socket says 172.21.15.99:44760) 2023-08-25T09:15:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39032/0 (socket says 172.21.15.99:39032) 2023-08-25T09:15:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37effd700 1 -- 172.21.15.99:0/1561050435 learned_addr learned my addr 172.21.15.99:0/1561050435 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37ffff700 1 -- 172.21.15.99:0/1561050435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 msgr2=0x558c9e9bd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37ffff700 1 --2- 172.21.15.99:0/1561050435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37ffff700 1 -- 172.21.15.99:0/1561050435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 msgr2=0x558c9e9d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37ffff700 1 --2- 172.21.15.99:0/1561050435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37ffff700 1 -- 172.21.15.99:0/1561050435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c9e718c00 con 0x558c9e9b7930 2023-08-25T09:15:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.599+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/1561050435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:15:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.600+0000 7fa37ffff700 1 --2- 172.21.15.99:0/1561050435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 0x558c9e9abbd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa3780c8900 tx=0x7fa3780982b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e141adcf38f0e407 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.600+0000 7fa3667fc700 1 -- 172.21.15.99:0/1561050435 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378098950 con 0x558c9e9b7930 2023-08-25T09:15:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.600+0000 7fa3667fc700 1 -- 172.21.15.99:0/1561050435 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37809c780 con 0x558c9e9b7930 2023-08-25T09:15:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.600+0000 7fa3667fc700 1 -- 172.21.15.99:0/1561050435 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378097e40 con 0x558c9e9b7930 2023-08-25T09:15:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1561050435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 msgr2=0x558c9e9abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1561050435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 0x558c9e9abbd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa3780c8900 tx=0x7fa3780982b0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1561050435 shutdown_connections 2023-08-25T09:15:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1561050435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7930 0x558c9e9abbd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1561050435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1561050435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7010 0x558c9e9bd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1561050435 >> 172.21.15.99:0/1561050435 conn(0x558c9e926080 msgr2=0x558c9e9d7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1561050435 shutdown_connections 2023-08-25T09:15:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1561050435 wait complete. 2023-08-25T09:15:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 -- start start 2023-08-25T09:15:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.601+0000 7fa399a7fb80 1 Processor -- start 2023-08-25T09:15:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7010 0x558c9e9d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa399a7fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9ea2ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9ea2f1d0 con 0x558c9e9b1990 2023-08-25T09:15:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9ea2f310 con 0x558c9e9b7010 2023-08-25T09:15:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa399a7fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9ea2f480 con 0x558c9e9b7930 2023-08-25T09:15:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa399a7fb80 10 client.0 _ll_register_callbacks cb 0x7fa3780097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:15:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa37effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7010 0x558c9e9d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa37effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7010 0x558c9e9d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:40086/0 (socket says 172.21.15.99:40086) 2023-08-25T09:15:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa37effd700 1 -- 172.21.15.99:0/1217176435 learned_addr learned my addr 172.21.15.99:0/1217176435 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:15:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.602+0000 7fa37ffff700 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9ea2ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa37effd700 1 -- 172.21.15.99:0/1217176435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 msgr2=0x558c9ea2ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa37effd700 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9ea2ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa37effd700 1 -- 172.21.15.99:0/1217176435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 msgr2=0x558c9e9d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:15:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa37effd700 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa37effd700 1 -- 172.21.15.99:0/1217176435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa370009ba0 con 0x558c9e9b7010 2023-08-25T09:15:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa37effd700 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7010 0x558c9e9d1af0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa37402b040 tx=0x7fa3740207d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c297391fa2886e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:15:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa35effd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:15:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa35effd700 1 -- 172.21.15.99:0/1217176435 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37401c070 con 0x558c9e9b7010 2023-08-25T09:15:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa35effd700 1 -- 172.21.15.99:0/1217176435 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa374000e40 con 0x558c9e9b7010 2023-08-25T09:15:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1217176435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c9e9a4600 con 0x558c9e9b7010 2023-08-25T09:15:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa35effd700 1 -- 172.21.15.99:0/1217176435 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa374005700 con 0x558c9e9b7010 2023-08-25T09:15:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa399a7fb80 10 client.26594 fetch_fsmap learned FSMap version 53 2023-08-25T09:15:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.603+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1217176435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7fa3801002c0 con 0x558c9e9b7010 2023-08-25T09:15:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa35effd700 1 -- 172.21.15.99:0/1217176435 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c9e9a4600 con 0x558c9e9b7010 2023-08-25T09:15:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa35effd700 1 -- 172.21.15.99:0/1217176435 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa374009950 con 0x558c9e9b7010 2023-08-25T09:15:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa399a7fb80 10 client.26594 fetch_fsmap finished waiting for FSMap version 53 2023-08-25T09:15:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa399a7fb80 10 client.26594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:15:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa399a7fb80 20 client.26594 populate_metadata read hostname 'smithi099' 2023-08-25T09:15:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558c9ea30760 0x558c9ea32b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:15:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa399a7fb80 4 client.26594 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:15:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1217176435 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558c9e8fc910 con 0x558c9ea30760 2023-08-25T09:15:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.604+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558c9ea30760 0x558c9ea32b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:15:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.605+0000 7fa37f7fe700 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558c9ea30760 0x558c9ea32b80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e96599b7f50b58bb server_cookie=451131c8c3734ae6 in_seq=0 out_seq=0 2023-08-25T09:15:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.605+0000 7fa35effd700 10 client.26594 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:49 smithi099 ceph-mon[97641]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2023-08-25T09:15:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.830+0000 7fa35effd700 1 -- 172.21.15.99:0/1217176435 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558c9e8fc910 con 0x558c9ea30760 2023-08-25T09:15:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.830+0000 7fa35effd700 10 client.26594 handle_command_reply: tid=0 2023-08-25T09:15:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.830+0000 7fa399a7fb80 1 client.26594 shutdown 2023-08-25T09:15:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.837+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1217176435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7010 msgr2=0x558c9e9d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.837+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7010 0x558c9e9d1af0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa37402b040 tx=0x7fa3740207d0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.838+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1217176435 shutdown_connections 2023-08-25T09:15:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.838+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c9e9b7930 0x558c9ea2ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.838+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9e9b1990 0x558c9e9d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.838+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x558c9ea30760 0x558c9ea32b80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.838+0000 7fa399a7fb80 1 --2- 172.21.15.99:0/1217176435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c9e9b7010 0x558c9e9d1af0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.838+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1217176435 >> 172.21.15.99:0/1217176435 conn(0x558c9e926080 msgr2=0x558c9e9abbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.839+0000 7fa35effd700 0 client.26594 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:15:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.840+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1217176435 shutdown_connections 2023-08-25T09:15:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.840+0000 7fa399a7fb80 1 -- 172.21.15.99:0/1217176435 wait complete. 2023-08-25T09:15:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.840+0000 7fa399a7fb80 20 client.26594 trim_cache size 0 max 16384 2023-08-25T09:15:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:15:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:15:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:15:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:15:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 -- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa3680a0140 msgr2=0x7fa3680a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa3680a0140 0x7fa3680a2600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa3700095a0 tx=0x7fa370009530 comp rx=0 tx=0).stop 2023-08-25T09:15:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 -- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3801068a0 msgr2=0x7fa3801aac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:15:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3801068a0 0x7fa3801aac90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa37800ef30 tx=0x7fa37800c450 comp rx=0 tx=0).stop 2023-08-25T09:15:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 -- 172.21.15.99:0/3672829786 shutdown_connections 2023-08-25T09:15:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fa3680a0140 0x7fa3680a2600 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa3700095a0 tx=0x7fa370009530 comp rx=0 tx=0).stop 2023-08-25T09:15:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3801068a0 0x7fa3801aac90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380067d10 0x7fa38006aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 --2- 172.21.15.99:0/3672829786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800686d0 0x7fa38006b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:15:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.841+0000 7fa384f9d700 1 -- 172.21.15.99:0/3672829786 >> 172.21.15.99:0/3672829786 conn(0x7fa380075050 msgr2=0x7fa3801043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:15:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.843+0000 7fa384f9d700 1 -- 172.21.15.99:0/3672829786 shutdown_connections 2023-08-25T09:15:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:15:49.844+0000 7fa384f9d700 1 -- 172.21.15.99:0/3672829786 wait complete. 2023-08-25T09:15:49.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:49 smithi157 ceph-mon[130879]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2023-08-25T09:15:49.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:49 smithi181 ceph-mon[131296]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2023-08-25T09:15:50.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 145372 33.32 MB/sec execute 115 sec latency 189.773 ms 2023-08-25T09:15:51.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 146067 33.28 MB/sec execute 116 sec latency 116.905 ms 2023-08-25T09:15:51.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:51 smithi099 ceph-mon[97641]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2023-08-25T09:15:51.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:51 smithi157 ceph-mon[130879]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2023-08-25T09:15:51.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:51 smithi181 ceph-mon[131296]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2023-08-25T09:15:52.266 INFO:tasks.workunit.client.0.smithi099.stdout: 10 146879 33.47 MB/sec execute 117 sec latency 97.277 ms 2023-08-25T09:15:53.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 147419 33.43 MB/sec execute 118 sec latency 168.784 ms 2023-08-25T09:15:53.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:53 smithi099 ceph-mon[97641]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 698 op/s 2023-08-25T09:15:53.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:53 smithi157 ceph-mon[130879]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 698 op/s 2023-08-25T09:15:53.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:53 smithi181 ceph-mon[131296]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 698 op/s 2023-08-25T09:15:54.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 148023 33.34 MB/sec execute 119 sec latency 130.564 ms 2023-08-25T09:15:55.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 148732 33.44 MB/sec execute 120 sec latency 153.215 ms 2023-08-25T09:15:55.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:55 smithi157 ceph-mon[130879]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 450 op/s 2023-08-25T09:15:55.747 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:55 smithi181 ceph-mon[131296]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 450 op/s 2023-08-25T09:15:55.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:55 smithi099 ceph-mon[97641]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 450 op/s 2023-08-25T09:15:56.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 149256 33.38 MB/sec execute 121 sec latency 88.769 ms 2023-08-25T09:15:56.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:56 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:56.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:56 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:56.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:56 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:15:57.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 149945 33.35 MB/sec execute 122 sec latency 87.443 ms 2023-08-25T09:15:57.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:57 smithi157 ceph-mon[130879]: pgmap v528: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-08-25T09:15:57.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:57 smithi181 ceph-mon[131296]: pgmap v528: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-08-25T09:15:57.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:57 smithi099 ceph-mon[97641]: pgmap v528: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-08-25T09:15:58.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 150226 33.21 MB/sec execute 123 sec latency 662.107 ms 2023-08-25T09:15:59.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 150496 33.10 MB/sec execute 124 sec latency 714.293 ms 2023-08-25T09:15:59.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:15:59 smithi099 ceph-mon[97641]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-08-25T09:15:59.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:15:59 smithi157 ceph-mon[130879]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-08-25T09:15:59.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:15:59 smithi181 ceph-mon[131296]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-08-25T09:16:00.265 INFO:tasks.workunit.client.0.smithi099.stdout: 10 151084 33.13 MB/sec execute 125 sec latency 181.297 ms 2023-08-25T09:16:01.266 INFO:tasks.workunit.client.0.smithi099.stdout: 10 151084 32.87 MB/sec execute 126 sec latency 1181.456 ms 2023-08-25T09:16:01.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:01 smithi099 ceph-mon[97641]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-08-25T09:16:01.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:01 smithi099 ceph-mon[97641]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:16:01.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:01 smithi157 ceph-mon[130879]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-08-25T09:16:01.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:01 smithi157 ceph-mon[130879]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:16:01.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:01 smithi181 ceph-mon[131296]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-08-25T09:16:01.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:01 smithi181 ceph-mon[131296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T09:16:02.266 INFO:tasks.workunit.client.0.smithi099.stdout: 10 151084 32.61 MB/sec execute 127 sec latency 2181.617 ms 2023-08-25T09:16:03.042 INFO:tasks.workunit.client.0.smithi099.stdout:[136925] write failed on handle 13547 (Resource temporarily unavailable) 2023-08-25T09:16:03.043 INFO:tasks.workunit.client.0.smithi099.stdout:Child failed with status 1 2023-08-25T09:16:03.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:16:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.052+0000 7fb4db1ff700 1 -- 172.21.15.99:0/37684119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d41068d0 msgr2=0x7fb4d4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.052+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/37684119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d41068d0 0x7fb4d4106cb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098f0 tx=0x7fb4c8009c00 comp rx=0 tx=0).stop 2023-08-25T09:16:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.053+0000 7fb4db1ff700 1 -- 172.21.15.99:0/37684119 shutdown_connections 2023-08-25T09:16:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.053+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/37684119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40686d0 0x7fb4d410d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.053+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/37684119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d41068d0 0x7fb4d4106cb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.053+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/37684119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4d4067d10 0x7fb4d4068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.053+0000 7fb4db1ff700 1 -- 172.21.15.99:0/37684119 >> 172.21.15.99:0/37684119 conn(0x7fb4d4075250 msgr2=0x7fb4d4075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.053+0000 7fb4db1ff700 1 -- 172.21.15.99:0/37684119 shutdown_connections 2023-08-25T09:16:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.053+0000 7fb4db1ff700 1 -- 172.21.15.99:0/37684119 wait complete. 2023-08-25T09:16:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.055+0000 7fb4db1ff700 1 Processor -- start 2023-08-25T09:16:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.055+0000 7fb4db1ff700 1 -- start start 2023-08-25T09:16:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.055+0000 7fb4db1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d4067d10 0x7fb4d41947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.055+0000 7fb4db1ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4d40686d0 0x7fb4d4194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.055+0000 7fb4db1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d41068d0 0x7fb4d4199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.055+0000 7fb4db1ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4199950 con 0x7fb4d40686d0 2023-08-25T09:16:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.055+0000 7fb4db1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4199ac0 con 0x7fb4d4067d10 2023-08-25T09:16:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.055+0000 7fb4db1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4199c80 con 0x7fb4d41068d0 2023-08-25T09:16:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d4067d10 0x7fb4d41947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d4067d10 0x7fb4d41947a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:38372/0 (socket says 172.21.15.99:38372) 2023-08-25T09:16:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/167896874 learned_addr learned my addr 172.21.15.99:0/167896874 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d979c700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d41068d0 0x7fb4d4199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/167896874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d41068d0 msgr2=0x7fb4d4199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d41068d0 0x7fb4d4199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4d40686d0 msgr2=0x7fb4d4194ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4d40686d0 0x7fb4d4194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/167896874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8009520 con 0x7fb4d4067d10 2023-08-25T09:16:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.056+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d4067d10 0x7fb4d41947a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009230 tx=0x7fb4c800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.057+0000 7fb4d1ffb700 1 -- 172.21.15.99:0/167896874 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c801c070 con 0x7fb4d4067d10 2023-08-25T09:16:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.057+0000 7fb4d1ffb700 1 -- 172.21.15.99:0/167896874 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4c80058e0 con 0x7fb4d4067d10 2023-08-25T09:16:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.057+0000 7fb4db1ff700 1 -- 172.21.15.99:0/167896874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d4199f00 con 0x7fb4d4067d10 2023-08-25T09:16:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.057+0000 7fb4d1ffb700 1 -- 172.21.15.99:0/167896874 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8003950 con 0x7fb4d4067d10 2023-08-25T09:16:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.057+0000 7fb4db1ff700 1 -- 172.21.15.99:0/167896874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d41a1810 con 0x7fb4d4067d10 2023-08-25T09:16:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.060+0000 7fb4d1ffb700 1 -- 172.21.15.99:0/167896874 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fb4c8020030 con 0x7fb4d4067d10 2023-08-25T09:16:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.062+0000 7fb4d1ffb700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb4c40a0140 0x7fb4c40a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.062+0000 7fb4eeba6b80 1 Processor -- start 2023-08-25T09:16:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.062+0000 7fb4d1ffb700 1 -- 172.21.15.99:0/167896874 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4c80cc580 con 0x7fb4d4067d10 2023-08-25T09:16:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.062+0000 7fb4eeba6b80 1 -- start start 2023-08-25T09:16:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.062+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb4c40a0140 0x7fb4c40a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490aef40 0x5565490af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556549011750 con 0x5565490a9990 2023-08-25T09:16:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556549002720 con 0x5565490aef40 2023-08-25T09:16:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556549000040 con 0x5565490af860 2023-08-25T09:16:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:43712/0 (socket says 172.21.15.99:43712) 2023-08-25T09:16:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 -- 172.21.15.99:0/645712312 learned_addr learned my addr 172.21.15.99:0/645712312 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490aef40 0x5565490af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d979c700 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb4c40a0140 0x7fb4c40a2600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005950 tx=0x7fb4c000f040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 -- 172.21.15.99:0/645712312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490aef40 msgr2=0x5565490af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:16:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490aef40 0x5565490af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 -- 172.21.15.99:0/645712312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 msgr2=0x5565490a9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:16:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.063+0000 7fb4d3fff700 1 -- 172.21.15.99:0/645712312 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556548e7b5d0 con 0x5565490af860 2023-08-25T09:16:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.064+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490a9450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000bd90 tx=0x7fb4c0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7b21f751f99b69e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.064+0000 7fb4beffd700 1 -- 172.21.15.99:0/645712312 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000be80 con 0x5565490af860 2023-08-25T09:16:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.064+0000 7fb4beffd700 1 -- 172.21.15.99:0/645712312 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4c0019df0 con 0x5565490af860 2023-08-25T09:16:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.064+0000 7fb4beffd700 1 -- 172.21.15.99:0/645712312 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00175e0 con 0x5565490af860 2023-08-25T09:16:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.065+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/645712312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 msgr2=0x5565490a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.065+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490a9450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000bd90 tx=0x7fb4c0019000 comp rx=0 tx=0).stop 2023-08-25T09:16:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.065+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/645712312 shutdown_connections 2023-08-25T09:16:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.065+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.065+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490aef40 0x5565490af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.065+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/645712312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490a9450 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.065+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/645712312 >> 172.21.15.99:0/645712312 conn(0x55654901e080 msgr2=0x5565490a3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.065+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/645712312 shutdown_connections 2023-08-25T09:16:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.066+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/645712312 wait complete. 2023-08-25T09:16:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.066+0000 7fb4eeba6b80 1 -- start start 2023-08-25T09:16:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.066+0000 7fb4eeba6b80 1 Processor -- start 2023-08-25T09:16:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490a9990 0x5565490b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556548ffb930 con 0x5565490aef40 2023-08-25T09:16:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556548f99b50 con 0x5565490a9990 2023-08-25T09:16:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556548f99850 con 0x5565490af860 2023-08-25T09:16:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4eeba6b80 10 client.0 _ll_register_callbacks cb 0x556548ee3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:16:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4d8f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490a9990 0x5565490b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4d8f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490a9990 0x5565490b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:38390/0 (socket says 172.21.15.99:38390) 2023-08-25T09:16:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/3717703582 learned_addr learned my addr 172.21.15.99:0/3717703582 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4d979c700 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/3717703582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 msgr2=0x5565490b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:16:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.067+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/3717703582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 msgr2=0x5565490b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:16:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/3717703582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c000ba70 con 0x5565490a9990 2023-08-25T09:16:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:16:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490a9990 0x5565490b2990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80966f0 tx=0x7fb4c8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e5a8af87b74376 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-08-25T09:16:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4b3fff700 1 -- 172.21.15.99:0/3717703582 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8099c80 con 0x5565490a9990 2023-08-25T09:16:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/3717703582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556549079160 con 0x5565490a9990 2023-08-25T09:16:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4b3fff700 1 -- 172.21.15.99:0/3717703582 <== mon.2 v2:172.21.15.157:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4c809dab0 con 0x5565490a9990 2023-08-25T09:16:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4b3fff700 1 -- 172.21.15.99:0/3717703582 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4c80a1a00 con 0x5565490a9990 2023-08-25T09:16:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4eeba6b80 10 client.26606 fetch_fsmap learned FSMap version 55 2023-08-25T09:16:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.068+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/3717703582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x556548e27d80 con 0x5565490a9990 2023-08-25T09:16:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4b3fff700 1 -- 172.21.15.99:0/3717703582 <== mon.2 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556549079160 con 0x5565490a9990 2023-08-25T09:16:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4b3fff700 1 -- 172.21.15.99:0/3717703582 <== mon.2 v2:172.21.15.157:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb4c80ae070 con 0x5565490a9990 2023-08-25T09:16:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4eeba6b80 10 client.26606 fetch_fsmap finished waiting for FSMap version 55 2023-08-25T09:16:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4eeba6b80 10 client.26606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:16:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4eeba6b80 20 client.26606 populate_metadata read hostname 'smithi099' 2023-08-25T09:16:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5565490c2ea0 0x5565490c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4eeba6b80 4 client.26606 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:16:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/3717703582 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556548fa1b40 con 0x5565490c2ea0 2023-08-25T09:16:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.069+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5565490c2ea0 0x5565490c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.070+0000 7fb4b3fff700 10 client.26606 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:16:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.070+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5565490c2ea0 0x5565490c52c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c68df9480d7499 server_cookie=53e1806023e19863 in_seq=0 out_seq=0 2023-08-25T09:16:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.070+0000 7fb4b3fff700 1 -- 172.21.15.99:0/3717703582 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556548fa1b40 con 0x5565490c2ea0 2023-08-25T09:16:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.070+0000 7fb4b3fff700 10 client.26606 handle_command_reply: tid=0 2023-08-25T09:16:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.070+0000 7fb4eeba6b80 1 client.26606 shutdown 2023-08-25T09:16:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.070+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/3717703582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490a9990 msgr2=0x5565490b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.070+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490a9990 0x5565490b2990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80966f0 tx=0x7fb4c8011000 comp rx=0 tx=0).stop 2023-08-25T09:16:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/3717703582 shutdown_connections 2023-08-25T09:16:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490af860 0x5565490b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x5565490c2ea0 0x5565490c52c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490a9990 0x5565490b2990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/3717703582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/3717703582 >> 172.21.15.99:0/3717703582 conn(0x55654901e080 msgr2=0x55654909e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4b3fff700 0 client.26606 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:16:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/3717703582 shutdown_connections 2023-08-25T09:16:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/3717703582 wait complete. 2023-08-25T09:16:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.071+0000 7fb4eeba6b80 20 client.26606 trim_cache size 0 max 16384 2023-08-25T09:16:03.092 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-08-25T09:16:03.093 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-08-25T09:16:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4eeba6b80 1 Processor -- start 2023-08-25T09:16:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4eeba6b80 1 -- start start 2023-08-25T09:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490aef40 0x5565490ba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556548e7d8c0 con 0x5565490a9990 2023-08-25T09:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565490a0720 con 0x5565490af860 2023-08-25T09:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565490a4ca0 con 0x5565490aef40 2023-08-25T09:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4d8f9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490aef40 0x5565490ba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490aef40 0x5565490ba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:43720/0 (socket says 172.21.15.99:43720) 2023-08-25T09:16:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4d979c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4d8f9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38678/0 (socket says 172.21.15.99:38678) 2023-08-25T09:16:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.088+0000 7fb4d3fff700 1 -- 172.21.15.99:0/2231292020 learned_addr learned my addr 172.21.15.99:0/2231292020 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4d3fff700 1 -- 172.21.15.99:0/2231292020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 msgr2=0x5565490b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:16:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4d3fff700 1 -- 172.21.15.99:0/2231292020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 msgr2=0x5565490a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:16:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4d3fff700 1 -- 172.21.15.99:0/2231292020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556548e10c00 con 0x5565490aef40 2023-08-25T09:16:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4d979c700 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490aef40 0x5565490ba240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0024040 tx=0x7fb4c000d720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d111967e6f35898 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4bf7fe700 1 -- 172.21.15.99:0/2231292020 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00313e0 con 0x5565490aef40 2023-08-25T09:16:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4bf7fe700 1 -- 172.21.15.99:0/2231292020 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4c0031540 con 0x5565490aef40 2023-08-25T09:16:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.089+0000 7fb4bf7fe700 1 -- 172.21.15.99:0/2231292020 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0031bc0 con 0x5565490aef40 2023-08-25T09:16:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/2231292020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490aef40 msgr2=0x5565490ba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490aef40 0x5565490ba240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0024040 tx=0x7fb4c000d720 comp rx=0 tx=0).stop 2023-08-25T09:16:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/2231292020 shutdown_connections 2023-08-25T09:16:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490a9990 0x5565490a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/2231292020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490aef40 0x5565490ba240 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/2231292020 >> 172.21.15.99:0/2231292020 conn(0x55654901e080 msgr2=0x5565490acad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/2231292020 shutdown_connections 2023-08-25T09:16:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/2231292020 wait complete. 2023-08-25T09:16:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.090+0000 7fb4eeba6b80 1 -- start start 2023-08-25T09:16:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4eeba6b80 1 Processor -- start 2023-08-25T09:16:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490a9990 0x5565490b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4eeba6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565490ab700 con 0x5565490aef40 2023-08-25T09:16:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565490ab870 con 0x5565490af860 2023-08-25T09:16:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4eeba6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565490aba30 con 0x5565490a9990 2023-08-25T09:16:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4eeba6b80 10 client.0 _ll_register_callbacks cb 0x7fb4c80995e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:16:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4d8f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490a9990 0x5565490b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4d8f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490a9990 0x5565490b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:43732/0 (socket says 172.21.15.99:43732) 2023-08-25T09:16:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/273274397 learned_addr learned my addr 172.21.15.99:0/273274397 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.091+0000 7fb4d979c700 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 msgr2=0x5565490ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:16:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/273274397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 msgr2=0x5565490aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:16:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490aa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4d8f9b700 1 -- 172.21.15.99:0/273274397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556548e27d80 con 0x5565490a9990 2023-08-25T09:16:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4d979c700 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:16:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4d8f9b700 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490a9990 0x5565490b35a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009fd0 tx=0x7fb4c8009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c29837a0261417 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-08-25T09:16:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4b3fff700 1 -- 172.21.15.99:0/273274397 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80ca0c0 con 0x5565490a9990 2023-08-25T09:16:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4b3fff700 1 -- 172.21.15.99:0/273274397 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4c809dab0 con 0x5565490a9990 2023-08-25T09:16:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/273274397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55654909c600 con 0x5565490a9990 2023-08-25T09:16:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4b3fff700 1 -- 172.21.15.99:0/273274397 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4c8099c80 con 0x5565490a9990 2023-08-25T09:16:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4eeba6b80 10 client.26593 fetch_fsmap learned FSMap version 55 2023-08-25T09:16:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/273274397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fb4cc003cd0 con 0x5565490a9990 2023-08-25T09:16:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.092+0000 7fb4b3fff700 1 -- 172.21.15.99:0/273274397 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55654909c600 con 0x5565490a9990 2023-08-25T09:16:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4b3fff700 1 -- 172.21.15.99:0/273274397 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb4c807e070 con 0x5565490a9990 2023-08-25T09:16:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4eeba6b80 10 client.26593 fetch_fsmap finished waiting for FSMap version 55 2023-08-25T09:16:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4eeba6b80 10 client.26593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-25T09:16:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4eeba6b80 20 client.26593 populate_metadata read hostname 'smithi099' 2023-08-25T09:16:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x556549117130 0x556549119520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4eeba6b80 4 client.26593 mds_command: new command op to 24593 tid=0[{"prefix": "flush journal"}] 2023-08-25T09:16:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/273274397 --> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556548ff4910 con 0x556549117130 2023-08-25T09:16:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x556549117130 0x556549119520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4d3fff700 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x556549117130 0x556549119520 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee1fc5011728f4a server_cookie=21d72905e2305219 in_seq=0 out_seq=0 2023-08-25T09:16:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.093+0000 7fb4b3fff700 10 client.26593 ms_handle_connect on v2:172.21.15.157:6838/1062424301 2023-08-25T09:16:03.538 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:03 smithi099 ceph-mon[97641]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 797 op/s 2023-08-25T09:16:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.753+0000 7fb4b3fff700 1 -- 172.21.15.99:0/273274397 <== mds.0 v2:172.21.15.157:6838/1062424301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556548ff4910 con 0x556549117130 2023-08-25T09:16:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.753+0000 7fb4b3fff700 10 client.26593 handle_command_reply: tid=0 2023-08-25T09:16:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.753+0000 7fb4eeba6b80 1 client.26593 shutdown 2023-08-25T09:16:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.757+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/273274397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490a9990 msgr2=0x5565490b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.757+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490a9990 0x5565490b35a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009fd0 tx=0x7fb4c8009470 comp rx=0 tx=0).stop 2023-08-25T09:16:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/273274397 shutdown_connections 2023-08-25T09:16:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:6838/1062424301,v1:172.21.15.157:6839/1062424301] conn(0x556549117130 0x556549119520 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5565490af860 0x5565490ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565490a9990 0x5565490b35a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4eeba6b80 1 --2- 172.21.15.99:0/273274397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565490aef40 0x5565490aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/273274397 >> 172.21.15.99:0/273274397 conn(0x55654901e080 msgr2=0x5565490afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4b3fff700 0 client.26593 ms_handle_reset on v2:172.21.15.157:6838/1062424301 2023-08-25T09:16:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/273274397 shutdown_connections 2023-08-25T09:16:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.758+0000 7fb4eeba6b80 1 -- 172.21.15.99:0/273274397 wait complete. 2023-08-25T09:16:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.759+0000 7fb4eeba6b80 20 client.26593 trim_cache size 0 max 16384 2023-08-25T09:16:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2023-08-25T09:16:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2023-08-25T09:16:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2023-08-25T09:16:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2023-08-25T09:16:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.763+0000 7fb4db1ff700 1 -- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb4c40a0140 msgr2=0x7fb4c40a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.763+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb4c40a0140 0x7fb4c40a2600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005950 tx=0x7fb4c000f040 comp rx=0 tx=0).stop 2023-08-25T09:16:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.764+0000 7fb4db1ff700 1 -- 172.21.15.99:0/167896874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d4067d10 msgr2=0x7fb4d41947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.764+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d4067d10 0x7fb4d41947a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009230 tx=0x7fb4c800b610 comp rx=0 tx=0).stop 2023-08-25T09:16:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.767+0000 7fb4db1ff700 1 -- 172.21.15.99:0/167896874 shutdown_connections 2023-08-25T09:16:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.767+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb4c40a0140 0x7fb4c40a2600 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005950 tx=0x7fb4c000f040 comp rx=0 tx=0).stop 2023-08-25T09:16:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.767+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d41068d0 0x7fb4d4199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.767+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d4067d10 0x7fb4d41947a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.767+0000 7fb4db1ff700 1 --2- 172.21.15.99:0/167896874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4d40686d0 0x7fb4d4194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.767+0000 7fb4db1ff700 1 -- 172.21.15.99:0/167896874 >> 172.21.15.99:0/167896874 conn(0x7fb4d4075250 msgr2=0x7fb4d40fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.772+0000 7fb4db1ff700 1 -- 172.21.15.99:0/167896874 shutdown_connections 2023-08-25T09:16:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2023-08-25T09:16:03.776+0000 7fb4db1ff700 1 -- 172.21.15.99:0/167896874 wait complete. 2023-08-25T09:16:03.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:03 smithi157 ceph-mon[130879]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 797 op/s 2023-08-25T09:16:03.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:03 smithi181 ceph-mon[131296]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 797 op/s 2023-08-25T09:16:04.480 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4b369cf18ed1391a426ab4ae86da834e9c074f81 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-08-25T09:16:05.462 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a701c7b72d714a6f8768ccd22ebc9244 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4b369cf18ed1391a426ab4ae86da834e9c074f81 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-08-25T09:16:05.473 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-08-25T09:16:05.485 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T09:16:05.500 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-08-25T09:16:05.510 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-08-25T09:16:05.510 INFO:teuthology.task.exec:Executing custom commands... 2023-08-25T09:16:05.511 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T09:16:05.512 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-08-25T09:16:05.601 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:05 smithi099 ceph-mon[97641]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 524 op/s 2023-08-25T09:16:05.613 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-08-25T09:16:05.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:05 smithi157 ceph-mon[130879]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 524 op/s 2023-08-25T09:16:05.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-08-25T09:16:05.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:05 smithi181 ceph-mon[131296]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 524 op/s 2023-08-25T09:16:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.881+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/3385037278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0100ad0 msgr2=0x7fb7c0100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:05.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.881+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/3385037278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0100ad0 0x7fb7c0100eb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009990 tx=0x7fb7bc009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:05.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.881+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/3385037278 shutdown_connections 2023-08-25T09:16:05.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.881+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/3385037278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0102bd0 0x7fb7c010d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:05.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.881+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/3385037278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0100ad0 0x7fb7c0100eb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:05.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.881+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/3385037278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7c0102210 0x7fb7c0102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:05.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.881+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/3385037278 >> 172.21.15.99:0/3385037278 conn(0x7fb7c00fc1e0 msgr2=0x7fb7c00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:05.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.882+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/3385037278 shutdown_connections 2023-08-25T09:16:05.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.882+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/3385037278 wait complete. 2023-08-25T09:16:05.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c6d9d700 1 Processor -- start 2023-08-25T09:16:05.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c6d9d700 1 -- start start 2023-08-25T09:16:05.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0100ad0 0x7fb7c019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:05.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0102210 0x7fb7c019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:05.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7c0102bd0 0x7fb7c0196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:05.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c6d9d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0196d70 con 0x7fb7c0102210 2023-08-25T09:16:05.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c6d9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0196ee0 con 0x7fb7c0102bd0 2023-08-25T09:16:05.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c6d9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c01970a0 con 0x7fb7c0100ad0 2023-08-25T09:16:05.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c559a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0102210 0x7fb7c019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:05.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c559a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0102210 0x7fb7c019cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38706/0 (socket says 172.21.15.99:38706) 2023-08-25T09:16:05.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c559a700 1 -- 172.21.15.99:0/38921841 learned_addr learned my addr 172.21.15.99:0/38921841 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:05.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c659c700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7c0102bd0 0x7fb7c0196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:05.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.883+0000 7fb7c5d9b700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0100ad0 0x7fb7c019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:05.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c559a700 1 -- 172.21.15.99:0/38921841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0100ad0 msgr2=0x7fb7c019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:05.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c559a700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0100ad0 0x7fb7c019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:05.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c559a700 1 -- 172.21.15.99:0/38921841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7c0102bd0 msgr2=0x7fb7c0196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:05.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c559a700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7c0102bd0 0x7fb7c0196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:05.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c559a700 1 -- 172.21.15.99:0/38921841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0009520 con 0x7fb7c0102210 2023-08-25T09:16:05.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c659c700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7c0102bd0 0x7fb7c0196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:05.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c559a700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0102210 0x7fb7c019cbd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000eb30 tx=0x7fb7b000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:05.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7b6ffd700 1 -- 172.21.15.99:0/38921841 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0009880 con 0x7fb7c0102210 2023-08-25T09:16:05.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc0095f0 con 0x7fb7c0102210 2023-08-25T09:16:05.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7b6ffd700 1 -- 172.21.15.99:0/38921841 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7b00044e0 con 0x7fb7c0102210 2023-08-25T09:16:05.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7b6ffd700 1 -- 172.21.15.99:0/38921841 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0003d80 con 0x7fb7c0102210 2023-08-25T09:16:05.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.884+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c01a34a0 con 0x7fb7c0102210 2023-08-25T09:16:05.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.885+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c004e620 con 0x7fb7c0102210 2023-08-25T09:16:05.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.887+0000 7fb7b6ffd700 1 -- 172.21.15.99:0/38921841 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fb7b0003f00 con 0x7fb7c0102210 2023-08-25T09:16:05.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.889+0000 7fb7b6ffd700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb7ac0a00c0 0x7fb7ac0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:05.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.889+0000 7fb7b6ffd700 1 -- 172.21.15.99:0/38921841 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb7b0014070 con 0x7fb7c0102210 2023-08-25T09:16:05.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.889+0000 7fb7c5d9b700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb7ac0a00c0 0x7fb7ac0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:05.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.890+0000 7fb7c5d9b700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb7ac0a00c0 0x7fb7ac0a2580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00b2e0 tx=0x7fb7bc005fb0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:05.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:05.892+0000 7fb7b6ffd700 1 -- 172.21.15.99:0/38921841 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb7b0095760 con 0x7fb7c0102210 2023-08-25T09:16:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.122+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 --> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7c01108a0 con 0x7fb7ac0a00c0 2023-08-25T09:16:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.133+0000 7fb7b6ffd700 1 -- 172.21.15.99:0/38921841 <== mgr.25202 v2:172.21.15.99:6800/2884087217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fb7c01108a0 con 0x7fb7ac0a00c0 2023-08-25T09:16:06.135 INFO:teuthology.orchestra.run.smithi099.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-08-25T00:00:00", "created": "2023-08-25T08:59:11", "first": "2023-08-25T09:00:00", "last": "2023-08-25T09:16:00", "last_pruned": "2023-08-25T09:16:02", "created_count": 17, "pruned_count": 11, "active": true} 2023-08-25T09:16:06.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb7ac0a00c0 msgr2=0x7fb7ac0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb7ac0a00c0 0x7fb7ac0a2580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00b2e0 tx=0x7fb7bc005fb0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0102210 msgr2=0x7fb7c019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0102210 0x7fb7c019cbd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000eb30 tx=0x7fb7b000ee40 comp rx=0 tx=0).stop 2023-08-25T09:16:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 shutdown_connections 2023-08-25T09:16:06.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb7ac0a00c0 0x7fb7ac0a2580 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7c0102bd0 0x7fb7c0196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0100ad0 0x7fb7c019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 --2- 172.21.15.99:0/38921841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c0102210 0x7fb7c019cbd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 >> 172.21.15.99:0/38921841 conn(0x7fb7c00fc1e0 msgr2=0x7fb7c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:06.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 shutdown_connections 2023-08-25T09:16:06.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.136+0000 7fb7c6d9d700 1 -- 172.21.15.99:0/38921841 wait complete. 2023-08-25T09:16:06.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-08-25T09:16:06.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 -- 172.21.15.99:0/1579666256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20281068c0 msgr2=0x7f2028106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 --2- 172.21.15.99:0/1579666256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20281068c0 0x7f2028106ca0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2018009990 tx=0x7f2018009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 -- 172.21.15.99:0/1579666256 shutdown_connections 2023-08-25T09:16:06.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 --2- 172.21.15.99:0/1579666256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028101270 0x7f202810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 --2- 172.21.15.99:0/1579666256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20281068c0 0x7f2028106ca0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 --2- 172.21.15.99:0/1579666256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20281008b0 0x7f2028100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 -- 172.21.15.99:0/1579666256 >> 172.21.15.99:0/1579666256 conn(0x7f20280fc380 msgr2=0x7f20280fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:06.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 -- 172.21.15.99:0/1579666256 shutdown_connections 2023-08-25T09:16:06.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.350+0000 7f202ddff700 1 -- 172.21.15.99:0/1579666256 wait complete. 2023-08-25T09:16:06.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.352+0000 7f202ddff700 1 Processor -- start 2023-08-25T09:16:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.352+0000 7f202ddff700 1 -- start start 2023-08-25T09:16:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.352+0000 7f202ddff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20281008b0 0x7f2028072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.352+0000 7f202ddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028101270 0x7f2028073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.352+0000 7f202ddff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20281068c0 0x7f20281a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.352+0000 7f202ddff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281a50e0 con 0x7f20281008b0 2023-08-25T09:16:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.352+0000 7f202ddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281a5220 con 0x7f2028101270 2023-08-25T09:16:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.352+0000 7f202ddff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281a5390 con 0x7f20281068c0 2023-08-25T09:16:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20281008b0 0x7f2028072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20281008b0 0x7f2028072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38716/0 (socket says 172.21.15.99:38716) 2023-08-25T09:16:06.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 -- 172.21.15.99:0/3848011326 learned_addr learned my addr 172.21.15.99:0/3848011326 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:06.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f2027fff700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20281068c0 0x7f20281a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:06.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f2026ffd700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028101270 0x7f2028073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:06.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 -- 172.21.15.99:0/3848011326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20281068c0 msgr2=0x7f20281a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20281068c0 0x7f20281a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 -- 172.21.15.99:0/3848011326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028101270 msgr2=0x7f2028073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028101270 0x7f2028073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 -- 172.21.15.99:0/3848011326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c009520 con 0x7f20281008b0 2023-08-25T09:16:06.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.353+0000 7f20277fe700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20281008b0 0x7f2028072e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2018005910 tx=0x7f2018004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:06.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.354+0000 7f2024ff9700 1 -- 172.21.15.99:0/3848011326 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201801d070 con 0x7f20281008b0 2023-08-25T09:16:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.354+0000 7f2024ff9700 1 -- 172.21.15.99:0/3848011326 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2018022470 con 0x7f20281008b0 2023-08-25T09:16:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.354+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20180095f0 con 0x7f20281008b0 2023-08-25T09:16:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.354+0000 7f2024ff9700 1 -- 172.21.15.99:0/3848011326 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018017650 con 0x7f20281008b0 2023-08-25T09:16:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.354+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20281a5960 con 0x7f20281008b0 2023-08-25T09:16:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.355+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20280ffff0 con 0x7f20281008b0 2023-08-25T09:16:06.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.357+0000 7f2024ff9700 1 -- 172.21.15.99:0/3848011326 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f20180225e0 con 0x7f20281008b0 2023-08-25T09:16:06.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.358+0000 7f2024ff9700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f20100a00c0 0x7f20100a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:06.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.358+0000 7f2024ff9700 1 -- 172.21.15.99:0/3848011326 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f20180ccd10 con 0x7f20281008b0 2023-08-25T09:16:06.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.358+0000 7f2026ffd700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f20100a00c0 0x7f20100a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:06.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.359+0000 7f2026ffd700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f20100a00c0 0x7f20100a2580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2028074620 tx=0x7f201c009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:06.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.360+0000 7f2024ff9700 1 -- 172.21.15.99:0/3848011326 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f2018096750 con 0x7f20281008b0 2023-08-25T09:16:06.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.589+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 --> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f20281a5550 con 0x7f20100a00c0 2023-08-25T09:16:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.599+0000 7f2024ff9700 1 -- 172.21.15.99:0/3848011326 <== mgr.25202 v2:172.21.15.99:6800/2884087217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f20281a5550 con 0x7f20100a00c0 2023-08-25T09:16:06.601 INFO:teuthology.orchestra.run.smithi099.stdout:/ 1M 6M3h 2023-08-25T09:16:06.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f20100a00c0 msgr2=0x7f20100a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f20100a00c0 0x7f20100a2580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2028074620 tx=0x7f201c009260 comp rx=0 tx=0).stop 2023-08-25T09:16:06.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20281008b0 msgr2=0x7f2028072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20281008b0 0x7f2028072e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2018005910 tx=0x7f2018004710 comp rx=0 tx=0).stop 2023-08-25T09:16:06.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 shutdown_connections 2023-08-25T09:16:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f20100a00c0 0x7f20100a2580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20281068c0 0x7f20281a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20281008b0 0x7f2028072e00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 --2- 172.21.15.99:0/3848011326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028101270 0x7f2028073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 >> 172.21.15.99:0/3848011326 conn(0x7f20280fc380 msgr2=0x7f20280fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 shutdown_connections 2023-08-25T09:16:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.602+0000 7f202ddff700 1 -- 172.21.15.99:0/3848011326 wait complete. 2023-08-25T09:16:06.622 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-08-25T09:16:06.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:16:06.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 -- 172.21.15.99:0/3801736757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12241008a0 msgr2=0x7f1224100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 --2- 172.21.15.99:0/3801736757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12241008a0 0x7f1224100d20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1214009990 tx=0x7f1214009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 -- 172.21.15.99:0/3801736757 shutdown_connections 2023-08-25T09:16:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 --2- 172.21.15.99:0/3801736757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1224101260 0x7f122410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 --2- 172.21.15.99:0/3801736757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12241068b0 0x7f1224106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 --2- 172.21.15.99:0/3801736757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12241008a0 0x7f1224100d20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 -- 172.21.15.99:0/3801736757 >> 172.21.15.99:0/3801736757 conn(0x7f12240fc3d0 msgr2=0x7f12240fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 -- 172.21.15.99:0/3801736757 shutdown_connections 2023-08-25T09:16:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.833+0000 7f122c3ff700 1 -- 172.21.15.99:0/3801736757 wait complete. 2023-08-25T09:16:06.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.834+0000 7f122c3ff700 1 Processor -- start 2023-08-25T09:16:06.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.834+0000 7f122c3ff700 1 -- start start 2023-08-25T09:16:06.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122c3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12241008a0 0x7f122419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:06.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122c3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224101260 0x7f122419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122c3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12241068b0 0x7f1224198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122c3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1224199140 con 0x7f1224101260 2023-08-25T09:16:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122c3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241992b0 con 0x7f12241008a0 2023-08-25T09:16:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122c3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1224199470 con 0x7f12241068b0 2023-08-25T09:16:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122999a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224101260 0x7f122419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122999a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224101260 0x7f122419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49688/0 (socket says 172.21.15.99:49688) 2023-08-25T09:16:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122999a700 1 -- 172.21.15.99:0/1851189313 learned_addr learned my addr 172.21.15.99:0/1851189313 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122a99c700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12241068b0 0x7f1224198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122a19b700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12241008a0 0x7f122419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122999a700 1 -- 172.21.15.99:0/1851189313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12241068b0 msgr2=0x7f1224198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122999a700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12241068b0 0x7f1224198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122999a700 1 -- 172.21.15.99:0/1851189313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12241008a0 msgr2=0x7f122419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:06.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122999a700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12241008a0 0x7f122419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:06.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.835+0000 7f122999a700 1 -- 172.21.15.99:0/1851189313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12140095f0 con 0x7f1224101260 2023-08-25T09:16:06.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.836+0000 7f122999a700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224101260 0x7f122419eed0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1214005910 tx=0x7f1214004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:06.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.836+0000 7f121b7fe700 1 -- 172.21.15.99:0/1851189313 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121401d070 con 0x7f1224101260 2023-08-25T09:16:06.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.836+0000 7f121b7fe700 1 -- 172.21.15.99:0/1851189313 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1214022470 con 0x7f1224101260 2023-08-25T09:16:06.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.836+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12241996f0 con 0x7f1224101260 2023-08-25T09:16:06.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.836+0000 7f121b7fe700 1 -- 172.21.15.99:0/1851189313 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214017650 con 0x7f1224101260 2023-08-25T09:16:06.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.836+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12241a5910 con 0x7f1224101260 2023-08-25T09:16:06.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.837+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1224067710 con 0x7f1224101260 2023-08-25T09:16:06.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.839+0000 7f121b7fe700 1 -- 172.21.15.99:0/1851189313 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f121400fad0 con 0x7f1224101260 2023-08-25T09:16:06.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.840+0000 7f121b7fe700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f12100a0100 0x7f12100a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:06.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.840+0000 7f121b7fe700 1 -- 172.21.15.99:0/1851189313 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f12140ccb50 con 0x7f1224101260 2023-08-25T09:16:06.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.841+0000 7f122a19b700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f12100a0100 0x7f12100a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:06.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.841+0000 7f122a19b700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f12100a0100 0x7f12100a25c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1220007b60 tx=0x7f12200073d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:06.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:06.844+0000 7f121b7fe700 1 -- 172.21.15.99:0/1851189313 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f12140965f0 con 0x7f1224101260 2023-08-25T09:16:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.044+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1224199ea0 con 0x7f1224101260 2023-08-25T09:16:07.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.046+0000 7f121b7fe700 1 -- 172.21.15.99:0/1851189313 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+23142 (secure 0 0 0) 0x7f1224199ea0 con 0x7f1224101260 2023-08-25T09:16:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T09:16:07.049 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":110,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T09:15:15.498006+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2892690228":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/3418985282":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6800/4068336524":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/90449605":"2023-08-26T08:59:10.145783+0000","172.21.15.99:6801/4068336524":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1861696476":"2023-08-26T08:59:10.145783+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:16:07.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.049+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f12100a0100 msgr2=0x7f12100a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.049+0000 7f122c3ff700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f12100a0100 0x7f12100a25c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1220007b60 tx=0x7f12200073d0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.049+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224101260 msgr2=0x7f122419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.049+0000 7f122c3ff700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224101260 0x7f122419eed0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1214005910 tx=0x7f1214004730 comp rx=0 tx=0).stop 2023-08-25T09:16:07.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.050+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 shutdown_connections 2023-08-25T09:16:07.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.050+0000 7f122c3ff700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f12100a0100 0x7f12100a25c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.050+0000 7f122c3ff700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12241068b0 0x7f1224198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.050+0000 7f122c3ff700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12241008a0 0x7f122419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.050+0000 7f122c3ff700 1 --2- 172.21.15.99:0/1851189313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224101260 0x7f122419eed0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.050+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 >> 172.21.15.99:0/1851189313 conn(0x7f12240fc3d0 msgr2=0x7f122410bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:07.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.050+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 shutdown_connections 2023-08-25T09:16:07.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.050+0000 7f122c3ff700 1 -- 172.21.15.99:0/1851189313 wait complete. 2023-08-25T09:16:07.071 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T08:57:17.389400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T08:57:18.695578+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:16:07.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:16:07.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 -- 172.21.15.99:0/55237709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b4101250 msgr2=0x7f49b410d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/55237709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b4101250 0x7f49b410d760 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00b0a0 tx=0x7f49ac00b3b0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 -- 172.21.15.99:0/55237709 shutdown_connections 2023-08-25T09:16:07.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/55237709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b4101250 0x7f49b410d760 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/55237709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b41068a0 0x7f49b4106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/55237709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49b4100890 0x7f49b4100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 -- 172.21.15.99:0/55237709 >> 172.21.15.99:0/55237709 conn(0x7f49b40fc380 msgr2=0x7f49b40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:07.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 -- 172.21.15.99:0/55237709 shutdown_connections 2023-08-25T09:16:07.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.270+0000 7f49bb7ff700 1 -- 172.21.15.99:0/55237709 wait complete. 2023-08-25T09:16:07.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.271+0000 7f49bb7ff700 1 Processor -- start 2023-08-25T09:16:07.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.271+0000 7f49bb7ff700 1 -- start start 2023-08-25T09:16:07.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.271+0000 7f49bb7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49b4100890 0x7f49b419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:07.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49bb7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b4101250 0x7f49b419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:07.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49bb7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b41068a0 0x7f49b4196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:07.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49bb7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4196dd0 con 0x7f49b41068a0 2023-08-25T09:16:07.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49bb7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4196f40 con 0x7f49b4101250 2023-08-25T09:16:07.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49bb7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4197100 con 0x7f49b4100890 2023-08-25T09:16:07.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b9d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b41068a0 0x7f49b4196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:07.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b959b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49b4100890 0x7f49b419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:07.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b959b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49b4100890 0x7f49b419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:50126/0 (socket says 172.21.15.99:50126) 2023-08-25T09:16:07.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b959b700 1 -- 172.21.15.99:0/800213734 learned_addr learned my addr 172.21.15.99:0/800213734 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:07.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b9d9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b41068a0 0x7f49b4196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49716/0 (socket says 172.21.15.99:49716) 2023-08-25T09:16:07.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b8d9a700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b4101250 0x7f49b419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:07.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b959b700 1 -- 172.21.15.99:0/800213734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b4101250 msgr2=0x7f49b419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b959b700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b4101250 0x7f49b419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b959b700 1 -- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b41068a0 msgr2=0x7f49b4196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b959b700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b41068a0 0x7f49b4196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b959b700 1 -- 172.21.15.99:0/800213734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac009d00 con 0x7f49b4100890 2023-08-25T09:16:07.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.272+0000 7f49b9d9c700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b41068a0 0x7f49b4196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:07.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.273+0000 7f49b959b700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49b4100890 0x7f49b419c6f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e9a0 tx=0x7f49b000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:07.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.273+0000 7f49aa7fc700 1 -- 172.21.15.99:0/800213734 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0009790 con 0x7f49b4100890 2023-08-25T09:16:07.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.273+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b41973e0 con 0x7f49b4100890 2023-08-25T09:16:07.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.273+0000 7f49aa7fc700 1 -- 172.21.15.99:0/800213734 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b0010430 con 0x7f49b4100890 2023-08-25T09:16:07.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.273+0000 7f49aa7fc700 1 -- 172.21.15.99:0/800213734 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0018680 con 0x7f49b4100890 2023-08-25T09:16:07.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.273+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b41a36e0 con 0x7f49b4100890 2023-08-25T09:16:07.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.274+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b40676a0 con 0x7f49b4100890 2023-08-25T09:16:07.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.276+0000 7f49aa7fc700 1 -- 172.21.15.99:0/800213734 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f49b00105a0 con 0x7f49b4100890 2023-08-25T09:16:07.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.277+0000 7f49aa7fc700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f49a00a00c0 0x7f49a00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:07.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.278+0000 7f49aa7fc700 1 -- 172.21.15.99:0/800213734 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f49b0014070 con 0x7f49b4100890 2023-08-25T09:16:07.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.278+0000 7f49b8d9a700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f49a00a00c0 0x7f49a00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:07.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.278+0000 7f49b8d9a700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f49a00a00c0 0x7f49a00a2580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f49b4197bd0 tx=0x7f49a4009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:07.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.280+0000 7f49aa7fc700 1 -- 172.21.15.99:0/800213734 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f49b0095410 con 0x7f49b4100890 2023-08-25T09:16:07.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.479+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49b41109a0 con 0x7f49b4100890 2023-08-25T09:16:07.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.480+0000 7f49aa7fc700 1 -- 172.21.15.99:0/800213734 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49b00260a0 con 0x7f49b4100890 2023-08-25T09:16:07.481 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T09:16:07.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f49a00a00c0 msgr2=0x7f49a00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f49a00a00c0 0x7f49a00a2580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f49b4197bd0 tx=0x7f49a4009260 comp rx=0 tx=0).stop 2023-08-25T09:16:07.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49b4100890 msgr2=0x7f49b419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49b4100890 0x7f49b419c6f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e9a0 tx=0x7f49b000ecb0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 shutdown_connections 2023-08-25T09:16:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f49a00a00c0 0x7f49a00a2580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b41068a0 0x7f49b4196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49b4100890 0x7f49b419c6f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 --2- 172.21.15.99:0/800213734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b4101250 0x7f49b419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 >> 172.21.15.99:0/800213734 conn(0x7f49b40fc380 msgr2=0x7f49b41043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 shutdown_connections 2023-08-25T09:16:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.482+0000 7f49bb7ff700 1 -- 172.21.15.99:0/800213734 wait complete. 2023-08-25T09:16:07.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:16:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.704+0000 7f5c883ff700 1 -- 172.21.15.99:0/2861359344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80101230 msgr2=0x7f5c8010d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.704+0000 7f5c883ff700 1 --2- 172.21.15.99:0/2861359344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80101230 0x7f5c8010d740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800b0a0 tx=0x7f5c7800b3b0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.704+0000 7f5c883ff700 1 -- 172.21.15.99:0/2861359344 shutdown_connections 2023-08-25T09:16:07.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.704+0000 7f5c883ff700 1 --2- 172.21.15.99:0/2861359344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80101230 0x7f5c8010d740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.704+0000 7f5c883ff700 1 --2- 172.21.15.99:0/2861359344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c80106880 0x7f5c80106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.704+0000 7f5c883ff700 1 --2- 172.21.15.99:0/2861359344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c80100870 0x7f5c80100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.704+0000 7f5c883ff700 1 -- 172.21.15.99:0/2861359344 >> 172.21.15.99:0/2861359344 conn(0x7f5c800fc3a0 msgr2=0x7f5c800fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:07.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.705+0000 7f5c883ff700 1 -- 172.21.15.99:0/2861359344 shutdown_connections 2023-08-25T09:16:07.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.705+0000 7f5c883ff700 1 -- 172.21.15.99:0/2861359344 wait complete. 2023-08-25T09:16:07.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.705+0000 7f5c883ff700 1 Processor -- start 2023-08-25T09:16:07.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c883ff700 1 -- start start 2023-08-25T09:16:07.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c883ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c80100870 0x7f5c8019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:07.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c883ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c80101230 0x7f5c8019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:07.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c883ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80106880 0x7f5c80198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:07.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c883ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80199000 con 0x7f5c80106880 2023-08-25T09:16:07.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c883ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80199170 con 0x7f5c80101230 2023-08-25T09:16:07.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c883ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80199330 con 0x7f5c80100870 2023-08-25T09:16:07.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c8699c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80106880 0x7f5c80198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:07.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c8619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c80100870 0x7f5c8019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:07.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c8699c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80106880 0x7f5c80198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49734/0 (socket says 172.21.15.99:49734) 2023-08-25T09:16:07.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c8699c700 1 -- 172.21.15.99:0/1609864690 learned_addr learned my addr 172.21.15.99:0/1609864690 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:07.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.706+0000 7f5c8599a700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c80101230 0x7f5c8019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:07.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c8699c700 1 -- 172.21.15.99:0/1609864690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c80100870 msgr2=0x7f5c8019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c8699c700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c80100870 0x7f5c8019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c8699c700 1 -- 172.21.15.99:0/1609864690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c80101230 msgr2=0x7f5c8019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c8699c700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c80101230 0x7f5c8019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c8699c700 1 -- 172.21.15.99:0/1609864690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78009d00 con 0x7f5c80106880 2023-08-25T09:16:07.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c8699c700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80106880 0x7f5c80198990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009850 tx=0x7f5c780077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:07.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c777fe700 1 -- 172.21.15.99:0/1609864690 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800e040 con 0x7f5c80106880 2023-08-25T09:16:07.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c777fe700 1 -- 172.21.15.99:0/1609864690 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c780079e0 con 0x7f5c80106880 2023-08-25T09:16:07.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c777fe700 1 -- 172.21.15.99:0/1609864690 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780052a0 con 0x7f5c80106880 2023-08-25T09:16:07.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c801995b0 con 0x7f5c80106880 2023-08-25T09:16:07.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.707+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c801a5910 con 0x7f5c80106880 2023-08-25T09:16:07.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.708+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c800676a0 con 0x7f5c80106880 2023-08-25T09:16:07.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.710+0000 7f5c777fe700 1 -- 172.21.15.99:0/1609864690 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f5c78004c60 con 0x7f5c80106880 2023-08-25T09:16:07.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.712+0000 7f5c777fe700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c6c0a0140 0x7f5c6c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:07.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.712+0000 7f5c777fe700 1 -- 172.21.15.99:0/1609864690 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5c780cc930 con 0x7f5c80106880 2023-08-25T09:16:07.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.712+0000 7f5c8619b700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c6c0a0140 0x7f5c6c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:07.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.712+0000 7f5c8619b700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c6c0a0140 0x7f5c6c0a2600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c005950 tx=0x7f5c7c00b410 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:07.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.715+0000 7f5c777fe700 1 -- 172.21.15.99:0/1609864690 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5c78096370 con 0x7f5c80106880 2023-08-25T09:16:07.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:07 smithi181 ceph-mon[131296]: from='client.16533 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T09:16:07.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:07 smithi181 ceph-mon[131296]: pgmap v533: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 847 op/s 2023-08-25T09:16:07.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:07 smithi181 ceph-mon[131296]: from='client.16539 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T09:16:07.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:07 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1851189313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:07.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:07 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/800213734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T09:16:07.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:07 smithi099 ceph-mon[97641]: from='client.16533 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T09:16:07.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:07 smithi099 ceph-mon[97641]: pgmap v533: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 847 op/s 2023-08-25T09:16:07.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:07 smithi099 ceph-mon[97641]: from='client.16539 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T09:16:07.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:07 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1851189313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:07.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:07 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/800213734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T09:16:07.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:07 smithi157 ceph-mon[130879]: from='client.16533 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T09:16:07.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:07 smithi157 ceph-mon[130879]: pgmap v533: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 847 op/s 2023-08-25T09:16:07.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:07 smithi157 ceph-mon[130879]: from='client.16539 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T09:16:07.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:07 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1851189313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:07.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:07 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/800213734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T09:16:07.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.927+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c801109e0 con 0x7f5c80106880 2023-08-25T09:16:07.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.928+0000 7f5c777fe700 1 -- 172.21.15.99:0/1609864690 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c78099990 con 0x7f5c80106880 2023-08-25T09:16:07.930 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-08-25T09:16:07.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c6c0a0140 msgr2=0x7f5c6c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c6c0a0140 0x7f5c6c0a2600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c005950 tx=0x7f5c7c00b410 comp rx=0 tx=0).stop 2023-08-25T09:16:07.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80106880 msgr2=0x7f5c80198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:07.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80106880 0x7f5c80198990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009850 tx=0x7f5c780077f0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 shutdown_connections 2023-08-25T09:16:07.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f5c6c0a0140 0x7f5c6c0a2600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c80106880 0x7f5c80198990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c80100870 0x7f5c8019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 --2- 172.21.15.99:0/1609864690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c80101230 0x7f5c8019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:07.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 >> 172.21.15.99:0/1609864690 conn(0x7f5c800fc3a0 msgr2=0x7f5c8010bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:07.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 shutdown_connections 2023-08-25T09:16:07.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:07.930+0000 7f5c883ff700 1 -- 172.21.15.99:0/1609864690 wait complete. 2023-08-25T09:16:07.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:16:08.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 -- 172.21.15.99:0/3996614271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 msgr2=0x7f9a0c10d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:08.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 --2- 172.21.15.99:0/3996614271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 0x7f9a0c10d740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400b0a0 tx=0x7f9a0400b3b0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 -- 172.21.15.99:0/3996614271 shutdown_connections 2023-08-25T09:16:08.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 --2- 172.21.15.99:0/3996614271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 0x7f9a0c10d740 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 --2- 172.21.15.99:0/3996614271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c106880 0x7f9a0c106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 --2- 172.21.15.99:0/3996614271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a0c100870 0x7f9a0c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 -- 172.21.15.99:0/3996614271 >> 172.21.15.99:0/3996614271 conn(0x7f9a0c0fc3a0 msgr2=0x7f9a0c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:08.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 -- 172.21.15.99:0/3996614271 shutdown_connections 2023-08-25T09:16:08.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.150+0000 7f9a119ff700 1 -- 172.21.15.99:0/3996614271 wait complete. 2023-08-25T09:16:08.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.151+0000 7f9a119ff700 1 Processor -- start 2023-08-25T09:16:08.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.151+0000 7f9a119ff700 1 -- start start 2023-08-25T09:16:08.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a119ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a0c100870 0x7f9a0c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:08.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a119ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 0x7f9a0c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:08.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a119ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c106880 0x7f9a0c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:08.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a119ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c1990c0 con 0x7f9a0c101230 2023-08-25T09:16:08.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a119ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c199230 con 0x7f9a0c106880 2023-08-25T09:16:08.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a119ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c1993f0 con 0x7f9a0c100870 2023-08-25T09:16:08.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 0x7f9a0c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:08.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 0x7f9a0c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49762/0 (socket says 172.21.15.99:49762) 2023-08-25T09:16:08.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0a7fc700 1 -- 172.21.15.99:0/959966213 learned_addr learned my addr 172.21.15.99:0/959966213 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:08.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0affd700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a0c100870 0x7f9a0c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:08.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0b7fe700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c106880 0x7f9a0c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:08.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0a7fc700 1 -- 172.21.15.99:0/959966213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a0c100870 msgr2=0x7f9a0c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:08.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0a7fc700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a0c100870 0x7f9a0c19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0a7fc700 1 -- 172.21.15.99:0/959966213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c106880 msgr2=0x7f9a0c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:08.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0a7fc700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c106880 0x7f9a0c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.152+0000 7f9a0a7fc700 1 -- 172.21.15.99:0/959966213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009d00 con 0x7f9a0c101230 2023-08-25T09:16:08.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.153+0000 7f9a0a7fc700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 0x7f9a0c19ee50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e9a0 tx=0x7f9a0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:08.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.153+0000 7f9a109fd700 1 -- 172.21.15.99:0/959966213 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00009790 con 0x7f9a0c101230 2023-08-25T09:16:08.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.153+0000 7f9a109fd700 1 -- 172.21.15.99:0/959966213 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a00010430 con 0x7f9a0c101230 2023-08-25T09:16:08.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.153+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c1996d0 con 0x7f9a0c101230 2023-08-25T09:16:08.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.153+0000 7f9a109fd700 1 -- 172.21.15.99:0/959966213 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00018680 con 0x7f9a0c101230 2023-08-25T09:16:08.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.153+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c1a5890 con 0x7f9a0c101230 2023-08-25T09:16:08.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.154+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c0676a0 con 0x7f9a0c101230 2023-08-25T09:16:08.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.156+0000 7f9a109fd700 1 -- 172.21.15.99:0/959966213 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9a000105a0 con 0x7f9a0c101230 2023-08-25T09:16:08.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.158+0000 7f9a109fd700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99f40a00c0 0x7f99f40a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:08.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.159+0000 7f9a109fd700 1 -- 172.21.15.99:0/959966213 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9a00014070 con 0x7f9a0c101230 2023-08-25T09:16:08.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.159+0000 7f9a0affd700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99f40a00c0 0x7f99f40a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:08.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.159+0000 7f9a0affd700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99f40a00c0 0x7f99f40a2580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009be0 tx=0x7f99fc009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:08.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.162+0000 7f9a109fd700 1 -- 172.21.15.99:0/959966213 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9a00051e00 con 0x7f9a0c101230 2023-08-25T09:16:08.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.363+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a0c0008d0 con 0x7f9a0c101230 2023-08-25T09:16:08.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.363+0000 7f9a109fd700 1 -- 172.21.15.99:0/959966213 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a00095260 con 0x7f9a0c101230 2023-08-25T09:16:08.366 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-08-25T09:16:08.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99f40a00c0 msgr2=0x7f99f40a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:08.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99f40a00c0 0x7f99f40a2580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009be0 tx=0x7f99fc009260 comp rx=0 tx=0).stop 2023-08-25T09:16:08.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 msgr2=0x7f9a0c19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:08.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 0x7f9a0c19ee50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e9a0 tx=0x7f9a0000ecb0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 shutdown_connections 2023-08-25T09:16:08.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f99f40a00c0 0x7f99f40a2580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c106880 0x7f9a0c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a0c100870 0x7f9a0c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 --2- 172.21.15.99:0/959966213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c101230 0x7f9a0c19ee50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 >> 172.21.15.99:0/959966213 conn(0x7f9a0c0fc3a0 msgr2=0x7f9a0c10bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:08.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 shutdown_connections 2023-08-25T09:16:08.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.366+0000 7f9a119ff700 1 -- 172.21.15.99:0/959966213 wait complete. 2023-08-25T09:16:08.386 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-08-25T09:16:08.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.641+0000 7f602b7ff700 1 -- 172.21.15.99:0/2674652204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024101260 msgr2=0x7f602410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:08.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.641+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2674652204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024101260 0x7f602410d770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f601c00b0a0 tx=0x7f601c00b3b0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.643+0000 7f602b7ff700 1 -- 172.21.15.99:0/2674652204 shutdown_connections 2023-08-25T09:16:08.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.643+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2674652204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024101260 0x7f602410d770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.643+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2674652204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60241068b0 0x7f6024106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.643+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2674652204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60241008a0 0x7f6024100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.643+0000 7f602b7ff700 1 -- 172.21.15.99:0/2674652204 >> 172.21.15.99:0/2674652204 conn(0x7f60240fc3b0 msgr2=0x7f60240fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:08.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.645+0000 7f602b7ff700 1 -- 172.21.15.99:0/2674652204 shutdown_connections 2023-08-25T09:16:08.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.647+0000 7f602b7ff700 1 -- 172.21.15.99:0/2674652204 wait complete. 2023-08-25T09:16:08.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.648+0000 7f602b7ff700 1 Processor -- start 2023-08-25T09:16:08.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.649+0000 7f602b7ff700 1 -- start start 2023-08-25T09:16:08.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.649+0000 7f602b7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241008a0 0x7f602410d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:08.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.649+0000 7f602b7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6024101260 0x7f602410d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:08.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.649+0000 7f602b7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60241068b0 0x7f602410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:08.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.649+0000 7f602b7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602410ab20 con 0x7f6024101260 2023-08-25T09:16:08.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.649+0000 7f602b7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602410ac60 con 0x7f60241068b0 2023-08-25T09:16:08.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.649+0000 7f602b7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602410ae20 con 0x7f60241008a0 2023-08-25T09:16:08.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.650+0000 7f6028d9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6024101260 0x7f602410d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:08.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.650+0000 7f6028d9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6024101260 0x7f602410d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49780/0 (socket says 172.21.15.99:49780) 2023-08-25T09:16:08.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.650+0000 7f6028d9a700 1 -- 172.21.15.99:0/2257901246 learned_addr learned my addr 172.21.15.99:0/2257901246 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:08.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.651+0000 7f6028d9a700 1 -- 172.21.15.99:0/2257901246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241008a0 msgr2=0x7f602410d1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T09:16:08.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.651+0000 7f6028d9a700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241008a0 0x7f602410d1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.651+0000 7f6028d9a700 1 -- 172.21.15.99:0/2257901246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60241068b0 msgr2=0x7f602410a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:08.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.651+0000 7f6028d9a700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60241068b0 0x7f602410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:08.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.651+0000 7f6028d9a700 1 -- 172.21.15.99:0/2257901246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c009d00 con 0x7f6024101260 2023-08-25T09:16:08.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.651+0000 7f6028d9a700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6024101260 0x7f602410d6e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9a0 tx=0x7f602000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:08.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.652+0000 7f601a7fc700 1 -- 172.21.15.99:0/2257901246 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000cbd0 con 0x7f6024101260 2023-08-25T09:16:08.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.652+0000 7f601a7fc700 1 -- 172.21.15.99:0/2257901246 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f602000cd30 con 0x7f6024101260 2023-08-25T09:16:08.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.652+0000 7f601a7fc700 1 -- 172.21.15.99:0/2257901246 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020018680 con 0x7f6024101260 2023-08-25T09:16:08.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.653+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6024071390 con 0x7f6024101260 2023-08-25T09:16:08.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.653+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6024071850 con 0x7f6024101260 2023-08-25T09:16:08.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.655+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6024067b60 con 0x7f6024101260 2023-08-25T09:16:08.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.661+0000 7f601a7fc700 1 -- 172.21.15.99:0/2257901246 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f6020010430 con 0x7f6024101260 2023-08-25T09:16:08.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.664+0000 7f601a7fc700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f60100a02a0 0x7f60100a2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:08.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.665+0000 7f602959b700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f60100a02a0 0x7f60100a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:08.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.666+0000 7f602959b700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f60100a02a0 0x7f60100a2760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6014009730 tx=0x7f6014006cb0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:08.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.673+0000 7f601a7fc700 1 -- 172.21.15.99:0/2257901246 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8673+0+0 (secure 0 0 0) 0x7f6020014070 con 0x7f6024101260 2023-08-25T09:16:08.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:08.674+0000 7f601a7fc700 1 -- 172.21.15.99:0/2257901246 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f6020094ba0 con 0x7f6024101260 2023-08-25T09:16:08.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:08 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1609864690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T09:16:08.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:08 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/959966213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T09:16:08.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:08 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1609864690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T09:16:08.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:08 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/959966213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T09:16:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:08 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1609864690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T09:16:08.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:08 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/959966213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T09:16:09.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.184+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f602410b6e0 con 0x7f6024101260 2023-08-25T09:16:09.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.210+0000 7f601a7fc700 1 -- 172.21.15.99:0/2257901246 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+728387 (secure 0 0 0) 0x7f60200981c0 con 0x7f6024101260 2023-08-25T09:16:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T09:16:09.224 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T09:16:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 26, 2023-08-25T09:16:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "active_gid": 25202, 2023-08-25T09:16:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "active_name": "x", 2023-08-25T09:16:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "active_addrs": { 2023-08-25T09:16:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-08-25T09:16:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-08-25T09:16:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-08-25T09:16:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2884087217 2023-08-25T09:16:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-08-25T09:16:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-08-25T09:16:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2884087217 2023-08-25T09:16:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "active_addr": "172.21.15.99:6801/2884087217", 2023-08-25T09:16:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "active_change": "2023-08-25T08:59:10.146026+0000", 2023-08-25T09:16:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "active_mgr_features": 4540138322906710015, 2023-08-25T09:16:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "available": true, 2023-08-25T09:16:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "standbys": [ 2023-08-25T09:16:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "gid": 25193, 2023-08-25T09:16:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "y", 2023-08-25T09:16:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr_features": 4540138322906710015, 2023-08-25T09:16:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2023-08-25T09:16:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2023-08-25T09:16:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2023-08-25T09:16:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.235 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.235 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.235 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.235 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.238 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.238 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.238 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.239 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.239 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.239 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.239 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.240 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.240 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.241 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.241 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.241 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.242 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.242 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.242 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.243 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.243 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.243 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.243 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.243 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.244 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.244 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.244 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.244 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.245 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.245 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.246 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.246 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.246 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.246 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.247 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.247 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.247 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.248 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2023-08-25T09:16:09.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2023-08-25T09:16:09.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.248 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.249 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.249 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.249 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.249 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.249 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2023-08-25T09:16:09.250 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.250 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.250 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.250 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2023-08-25T09:16:09.251 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2023-08-25T09:16:09.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.251 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.251 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.251 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2023-08-25T09:16:09.252 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.252 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.252 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.252 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2023-08-25T09:16:09.252 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.253 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2023-08-25T09:16:09.253 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2023-08-25T09:16:09.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.254 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.255 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.255 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2023-08-25T09:16:09.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.255 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.256 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2023-08-25T09:16:09.256 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2023-08-25T09:16:09.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.256 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.257 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.257 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.257 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.258 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2023-08-25T09:16:09.258 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.259 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2023-08-25T09:16:09.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2023-08-25T09:16:09.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.259 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.259 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2023-08-25T09:16:09.260 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.260 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.260 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.260 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2023-08-25T09:16:09.260 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.261 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.261 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2023-08-25T09:16:09.261 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2023-08-25T09:16:09.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.262 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.262 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.262 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.263 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2023-08-25T09:16:09.263 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.264 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.264 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2023-08-25T09:16:09.264 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2023-08-25T09:16:09.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.265 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.265 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.265 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.266 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.266 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-25T09:16:09.266 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.267 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2023-08-25T09:16:09.267 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2023-08-25T09:16:09.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.267 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.268 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.268 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.268 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.268 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.268 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2023-08-25T09:16:09.269 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.269 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.270 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2023-08-25T09:16:09.270 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.270 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.271 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.271 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2023-08-25T09:16:09.271 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2023-08-25T09:16:09.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.271 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.272 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.272 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.272 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.273 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2023-08-25T09:16:09.273 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.273 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.274 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2023-08-25T09:16:09.274 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2023-08-25T09:16:09.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.274 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.275 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2023-08-25T09:16:09.275 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.275 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.275 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2023-08-25T09:16:09.275 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T09:16:09.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.276 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.276 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2023-08-25T09:16:09.277 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2023-08-25T09:16:09.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:09.277 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:09.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2023-08-25T09:16:09.278 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2023-08-25T09:16:09.278 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.278 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-25T09:16:09.278 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T09:16:09.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.279 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.279 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2023-08-25T09:16:09.279 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2023-08-25T09:16:09.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:09.280 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.280 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2023-08-25T09:16:09.280 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:09.281 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2023-08-25T09:16:09.281 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.281 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-25T09:16:09.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.282 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.282 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2023-08-25T09:16:09.282 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2023-08-25T09:16:09.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.283 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2023-08-25T09:16:09.283 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.283 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.284 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-25T09:16:09.284 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-25T09:16:09.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.284 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.285 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2023-08-25T09:16:09.285 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2023-08-25T09:16:09.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:09.285 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.286 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2023-08-25T09:16:09.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2023-08-25T09:16:09.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2023-08-25T09:16:09.286 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.287 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2023-08-25T09:16:09.287 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-25T09:16:09.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.288 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2023-08-25T09:16:09.288 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2023-08-25T09:16:09.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2023-08-25T09:16:09.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.289 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2023-08-25T09:16:09.290 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T09:16:09.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.290 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.290 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2023-08-25T09:16:09.291 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2023-08-25T09:16:09.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:09.291 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.291 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:09.291 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2023-08-25T09:16:09.292 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2023-08-25T09:16:09.292 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.292 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-25T09:16:09.292 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T09:16:09.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.293 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.293 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.293 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.294 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.294 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.294 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.294 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.295 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.295 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.295 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.295 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.295 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.296 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.296 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.296 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.296 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.296 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.297 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.297 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.297 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.298 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.298 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.298 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.299 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.299 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.299 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.299 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.300 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.300 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.301 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.302 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.302 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.303 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.303 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.304 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.305 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.305 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.305 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.306 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.306 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.307 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.307 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.308 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.309 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.310 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.310 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.311 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.312 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.313 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.313 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.313 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.314 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.314 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.315 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2023-08-25T09:16:09.316 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2023-08-25T09:16:09.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:09.317 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.318 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:09.319 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.319 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.320 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-25T09:16:09.321 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.322 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.322 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2023-08-25T09:16:09.323 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2023-08-25T09:16:09.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.324 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.325 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2023-08-25T09:16:09.325 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.325 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.326 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.326 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2023-08-25T09:16:09.326 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2023-08-25T09:16:09.326 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap" 2023-08-25T09:16:09.326 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2023-08-25T09:16:09.327 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.327 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.328 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2023-08-25T09:16:09.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2023-08-25T09:16:09.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.328 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.329 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.329 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.329 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.329 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.330 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-25T09:16:09.330 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.330 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.331 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:09.331 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:09.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.332 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:09.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.334 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.334 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.335 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-25T09:16:09.335 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.336 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.337 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2023-08-25T09:16:09.337 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2023-08-25T09:16:09.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.338 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.339 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:09.339 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:09.340 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.340 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.340 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-25T09:16:09.341 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-25T09:16:09.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.341 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.341 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2023-08-25T09:16:09.342 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2023-08-25T09:16:09.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:09.342 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:09.343 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.343 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.343 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.343 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-25T09:16:09.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.344 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.344 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.345 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2023-08-25T09:16:09.345 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.345 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.345 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.345 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2023-08-25T09:16:09.346 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2023-08-25T09:16:09.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:09.346 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.346 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2023-08-25T09:16:09.347 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.347 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.347 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-25T09:16:09.347 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.348 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.348 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2023-08-25T09:16:09.348 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2023-08-25T09:16:09.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.349 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.349 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.349 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2023-08-25T09:16:09.349 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.350 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.350 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.350 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-25T09:16:09.350 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.350 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.351 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2023-08-25T09:16:09.351 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2023-08-25T09:16:09.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.351 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.352 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2023-08-25T09:16:09.352 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.352 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.352 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.353 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-25T09:16:09.353 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.353 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.354 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2023-08-25T09:16:09.354 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2023-08-25T09:16:09.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.354 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.354 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.355 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.355 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.355 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.355 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.355 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-25T09:16:09.356 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-25T09:16:09.356 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.356 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.356 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2023-08-25T09:16:09.357 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2023-08-25T09:16:09.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.357 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:09.358 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.358 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.358 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.358 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-25T09:16:09.358 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.359 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.359 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2023-08-25T09:16:09.359 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2023-08-25T09:16:09.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:09.360 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2023-08-25T09:16:09.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.361 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.361 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-25T09:16:09.361 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.362 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2023-08-25T09:16:09.362 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2023-08-25T09:16:09.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.362 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.363 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.363 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.363 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-25T09:16:09.364 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.364 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.364 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2023-08-25T09:16:09.365 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2023-08-25T09:16:09.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.365 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.365 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.366 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.366 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.366 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.366 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-25T09:16:09.366 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.367 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.367 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2023-08-25T09:16:09.367 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2023-08-25T09:16:09.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.368 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.368 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.368 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.369 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-25T09:16:09.369 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.369 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.370 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2023-08-25T09:16:09.370 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2023-08-25T09:16:09.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.370 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.371 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-25T09:16:09.371 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.371 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.371 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.371 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:09.372 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.372 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.372 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2023-08-25T09:16:09.373 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2023-08-25T09:16:09.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.373 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.373 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-25T09:16:09.374 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.374 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.374 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.374 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2023-08-25T09:16:09.374 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.375 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.375 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2023-08-25T09:16:09.375 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2023-08-25T09:16:09.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.376 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.376 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-25T09:16:09.376 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.376 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.377 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.377 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2023-08-25T09:16:09.377 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.377 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.378 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2023-08-25T09:16:09.378 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2023-08-25T09:16:09.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.378 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.379 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-25T09:16:09.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.379 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.379 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.379 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:09.380 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.380 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.380 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2023-08-25T09:16:09.381 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2023-08-25T09:16:09.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.381 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.381 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-25T09:16:09.382 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.382 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.382 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.382 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2023-08-25T09:16:09.382 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.383 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.383 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2023-08-25T09:16:09.383 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2023-08-25T09:16:09.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.384 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.384 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-25T09:16:09.384 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.384 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2023-08-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.386 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2023-08-25T09:16:09.386 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2023-08-25T09:16:09.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.386 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.387 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-25T09:16:09.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.387 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.387 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2023-08-25T09:16:09.388 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.388 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.389 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2023-08-25T09:16:09.389 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2023-08-25T09:16:09.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.389 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-25T09:16:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2023-08-25T09:16:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2023-08-25T09:16:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2023-08-25T09:16:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-25T09:16:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2023-08-25T09:16:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2023-08-25T09:16:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2023-08-25T09:16:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-25T09:16:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2023-08-25T09:16:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2023-08-25T09:16:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2023-08-25T09:16:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-25T09:16:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2023-08-25T09:16:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2023-08-25T09:16:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-08-25T09:16:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2023-08-25T09:16:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2023-08-25T09:16:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2023-08-25T09:16:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-25T09:16:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2023-08-25T09:16:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2023-08-25T09:16:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-25T09:16:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2023-08-25T09:16:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2023-08-25T09:16:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2023-08-25T09:16:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-25T09:16:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2023-08-25T09:16:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2023-08-25T09:16:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2023-08-25T09:16:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2023-08-25T09:16:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2023-08-25T09:16:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2023-08-25T09:16:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-25T09:16:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2023-08-25T09:16:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2023-08-25T09:16:09.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.416 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.416 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.416 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2023-08-25T09:16:09.416 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.417 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.417 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-25T09:16:09.417 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.418 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.418 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2023-08-25T09:16:09.418 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2023-08-25T09:16:09.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.419 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2023-08-25T09:16:09.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.420 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.420 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-25T09:16:09.420 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.420 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.421 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2023-08-25T09:16:09.421 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2023-08-25T09:16:09.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.421 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.422 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2023-08-25T09:16:09.422 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.422 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.422 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2023-08-25T09:16:09.423 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.423 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.423 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2023-08-25T09:16:09.424 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2023-08-25T09:16:09.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.424 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.424 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.424 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.425 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.425 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.425 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-25T09:16:09.425 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.426 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.426 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2023-08-25T09:16:09.426 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2023-08-25T09:16:09.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.427 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.427 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:09.427 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.428 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.428 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2023-08-25T09:16:09.428 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.428 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.429 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2023-08-25T09:16:09.429 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2023-08-25T09:16:09.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.429 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2023-08-25T09:16:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2023-08-25T09:16:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2023-08-25T09:16:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-25T09:16:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-25T09:16:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-25T09:16:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-25T09:16:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-25T09:16:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-25T09:16:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-25T09:16:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2023-08-25T09:16:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-25T09:16:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.453 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2023-08-25T09:16:09.453 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2023-08-25T09:16:09.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2023-08-25T09:16:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2023-08-25T09:16:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2023-08-25T09:16:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-25T09:16:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2023-08-25T09:16:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2023-08-25T09:16:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2023-08-25T09:16:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2023-08-25T09:16:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2023-08-25T09:16:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2023-08-25T09:16:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2023-08-25T09:16:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2023-08-25T09:16:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2023-08-25T09:16:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2023-08-25T09:16:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2023-08-25T09:16:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-25T09:16:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-25T09:16:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2023-08-25T09:16:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2023-08-25T09:16:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-25T09:16:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2023-08-25T09:16:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2023-08-25T09:16:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-25T09:16:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2023-08-25T09:16:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2023-08-25T09:16:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-25T09:16:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2023-08-25T09:16:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2023-08-25T09:16:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-25T09:16:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2023-08-25T09:16:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2023-08-25T09:16:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-25T09:16:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2023-08-25T09:16:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2023-08-25T09:16:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2023-08-25T09:16:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2023-08-25T09:16:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2023-08-25T09:16:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2023-08-25T09:16:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-25T09:16:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2023-08-25T09:16:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2023-08-25T09:16:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-25T09:16:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2023-08-25T09:16:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2023-08-25T09:16:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-25T09:16:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2023-08-25T09:16:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2023-08-25T09:16:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2023-08-25T09:16:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2023-08-25T09:16:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2023-08-25T09:16:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-25T09:16:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2023-08-25T09:16:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2023-08-25T09:16:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-25T09:16:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2023-08-25T09:16:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2023-08-25T09:16:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2023-08-25T09:16:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2023-08-25T09:16:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-25T09:16:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2023-08-25T09:16:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2023-08-25T09:16:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2023-08-25T09:16:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-25T09:16:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2023-08-25T09:16:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-25T09:16:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-25T09:16:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2023-08-25T09:16:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-25T09:16:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-25T09:16:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-25T09:16:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2023-08-25T09:16:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2023-08-25T09:16:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-25T09:16:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-25T09:16:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2023-08-25T09:16:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-25T09:16:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.536 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-25T09:16:09.536 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-25T09:16:09.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.536 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.537 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.537 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.537 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.537 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.537 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.538 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.538 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.538 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-25T09:16:09.539 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-25T09:16:09.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.539 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.539 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.540 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.540 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.540 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.540 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.540 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.541 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.541 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-25T09:16:09.541 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-25T09:16:09.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.542 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.542 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.542 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.542 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.543 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.543 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.543 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.543 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.544 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-25T09:16:09.544 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-25T09:16:09.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.544 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.544 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.545 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.545 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.545 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.545 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.545 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.546 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.546 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.546 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-25T09:16:09.547 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-25T09:16:09.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.547 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.547 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.547 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.548 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.548 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.548 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.548 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.548 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.549 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.549 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-25T09:16:09.549 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-25T09:16:09.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.550 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.550 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.550 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.550 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.551 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.551 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.551 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.551 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.551 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.552 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-25T09:16:09.552 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-25T09:16:09.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.552 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.553 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.553 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.553 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.553 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.553 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.554 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.554 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.554 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-25T09:16:09.555 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-25T09:16:09.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.555 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.555 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.556 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.556 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.556 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.556 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.556 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.557 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.557 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2023-08-25T09:16:09.557 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-25T09:16:09.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.558 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.558 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:09.558 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.558 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.559 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.559 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.559 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.559 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.560 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-25T09:16:09.560 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-25T09:16:09.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.560 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.561 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.561 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.561 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.561 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.561 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.562 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.562 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.562 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2023-08-25T09:16:09.563 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2023-08-25T09:16:09.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.563 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.563 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.564 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.564 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.564 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.564 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.564 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.565 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.565 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2023-08-25T09:16:09.565 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-25T09:16:09.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.566 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.566 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:09.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.566 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.567 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.567 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.567 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.567 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.568 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-25T09:16:09.568 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-25T09:16:09.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.568 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.569 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.569 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.569 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.569 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.569 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.570 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.570 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.570 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-25T09:16:09.571 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-25T09:16:09.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.571 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.571 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.572 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.572 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.572 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.572 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.572 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.573 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.573 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.573 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-25T09:16:09.573 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-25T09:16:09.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.574 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.574 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.574 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.574 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.575 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.575 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.575 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.575 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.576 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-25T09:16:09.576 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-25T09:16:09.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.576 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.577 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.577 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.577 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.577 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.577 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.578 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.578 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.578 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2023-08-25T09:16:09.579 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-25T09:16:09.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.579 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.579 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.579 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.580 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.580 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.580 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.580 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.581 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.581 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-25T09:16:09.581 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-25T09:16:09.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.582 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.582 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.582 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.582 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.583 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.583 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.583 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.583 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.584 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-25T09:16:09.584 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-25T09:16:09.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.584 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.585 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.585 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.585 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.585 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.585 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.586 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.586 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.586 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-25T09:16:09.586 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-25T09:16:09.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.587 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.587 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.587 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.588 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.588 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.588 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.588 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.589 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.589 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-25T09:16:09.589 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-25T09:16:09.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.590 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.590 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.590 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.590 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.591 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.591 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.591 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.591 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.592 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-25T09:16:09.592 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-25T09:16:09.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.592 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.593 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.593 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.593 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.593 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.593 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.594 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.594 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.594 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-25T09:16:09.595 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-25T09:16:09.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.595 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.595 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.595 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.596 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.596 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.596 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.596 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.596 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.597 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.597 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.597 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-25T09:16:09.597 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-25T09:16:09.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.598 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.598 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.599 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.599 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.599 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.599 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.600 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.600 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.600 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.600 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-25T09:16:09.601 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-25T09:16:09.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.601 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.601 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.601 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.601 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.602 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.602 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.602 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.602 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.603 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.603 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2023-08-25T09:16:09.603 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-25T09:16:09.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.604 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.604 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.604 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.604 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.604 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.605 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.605 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.605 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.606 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-25T09:16:09.606 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-25T09:16:09.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.606 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.607 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-25T09:16:09.607 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.607 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.607 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.607 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.607 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.608 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.608 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.608 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-25T09:16:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-25T09:16:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.610 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.610 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.610 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.610 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.611 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.611 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-25T09:16:09.611 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-25T09:16:09.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.612 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.612 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2023-08-25T09:16:09.612 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.612 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.612 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.613 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.613 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.613 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.613 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.614 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-25T09:16:09.614 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-25T09:16:09.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.614 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.614 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.615 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2023-08-25T09:16:09.615 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.615 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.615 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.615 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.616 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.616 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.616 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2023-08-25T09:16:09.616 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-25T09:16:09.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.617 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.617 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.617 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.618 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.618 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.618 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.618 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.618 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.619 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.619 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-25T09:16:09.619 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-25T09:16:09.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.619 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.620 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:09.620 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.620 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.620 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.621 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.621 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.621 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.622 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2023-08-25T09:16:09.622 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-25T09:16:09.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.622 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.622 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.623 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.623 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.623 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.623 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.623 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.623 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.624 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.624 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.624 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2023-08-25T09:16:09.625 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-25T09:16:09.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.625 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.625 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.625 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.625 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.626 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.626 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.626 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.626 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.627 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.627 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-25T09:16:09.627 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-25T09:16:09.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.628 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.628 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.628 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:09.628 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.628 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.629 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.629 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.629 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.629 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.630 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-25T09:16:09.630 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-25T09:16:09.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.630 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.631 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:09.631 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.631 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.631 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.631 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.632 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.632 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.632 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-25T09:16:09.633 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-25T09:16:09.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.633 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.633 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:09.633 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.634 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.634 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.634 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.634 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.634 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.635 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.635 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2023-08-25T09:16:09.635 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2023-08-25T09:16:09.635 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.636 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.636 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.636 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.636 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.636 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.637 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.637 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.637 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.637 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.638 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2023-08-25T09:16:09.638 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2023-08-25T09:16:09.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.638 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.639 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.639 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.639 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.639 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.639 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.640 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.640 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.640 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2023-08-25T09:16:09.640 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2023-08-25T09:16:09.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.641 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.641 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.641 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.642 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.642 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.642 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2023-08-25T09:16:09.642 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.643 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.643 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2023-08-25T09:16:09.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2023-08-25T09:16:09.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.643 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.644 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.644 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2023-08-25T09:16:09.644 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.644 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.644 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.645 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.645 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.645 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.645 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.646 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2023-08-25T09:16:09.646 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2023-08-25T09:16:09.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.646 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.647 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.647 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.647 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.647 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.647 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.647 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.648 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.648 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.648 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.649 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.649 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.649 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.650 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.650 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.650 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.650 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.650 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.651 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.651 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.651 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.651 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.651 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.651 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.652 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.652 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.652 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.652 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.653 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.653 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.653 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.654 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.654 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.654 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.654 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.655 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.655 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.655 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.656 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.656 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.656 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.656 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.656 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.656 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.657 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.657 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.657 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.657 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.657 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.658 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.658 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.658 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.658 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.659 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.659 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.659 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.659 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.660 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.660 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.660 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.660 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.661 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.661 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.661 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.661 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.661 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.662 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2023-08-25T09:16:09.662 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2023-08-25T09:16:09.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.662 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.662 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.663 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.663 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.663 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.663 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.663 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2023-08-25T09:16:09.664 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.664 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.664 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2023-08-25T09:16:09.665 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2023-08-25T09:16:09.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.665 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.665 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.666 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.666 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.666 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.666 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.666 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.667 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.667 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:09.667 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:09.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.668 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.668 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2023-08-25T09:16:09.668 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.668 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.669 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.669 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.669 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.669 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.670 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:09.670 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:09.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.670 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.671 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2023-08-25T09:16:09.671 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.671 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.671 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.671 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.672 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.672 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.672 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2023-08-25T09:16:09.673 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2023-08-25T09:16:09.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.673 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.673 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.673 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.674 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.674 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.674 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.674 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.674 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.675 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.675 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2023-08-25T09:16:09.675 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2023-08-25T09:16:09.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.676 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.676 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2023-08-25T09:16:09.676 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.676 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.677 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.677 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.677 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.677 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.678 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2023-08-25T09:16:09.678 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2023-08-25T09:16:09.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.678 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.679 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2023-08-25T09:16:09.679 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.679 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.679 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.679 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.680 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2023-08-25T09:16:09.680 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.680 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.680 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.681 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.681 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2023-08-25T09:16:09.681 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2023-08-25T09:16:09.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.681 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.682 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.682 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2023-08-25T09:16:09.682 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2023-08-25T09:16:09.682 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2023-08-25T09:16:09.682 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.683 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.683 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.683 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.684 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2023-08-25T09:16:09.684 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2023-08-25T09:16:09.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.684 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.685 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.685 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.685 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.685 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.685 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.685 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.686 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.687 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2023-08-25T09:16:09.687 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.687 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.687 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.688 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2023-08-25T09:16:09.688 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2023-08-25T09:16:09.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.688 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.688 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.689 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.689 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.689 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.689 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2023-08-25T09:16:09.689 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.690 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.690 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.690 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.691 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.691 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.691 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.691 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.692 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.692 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.692 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.692 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.692 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.693 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.693 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.693 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.693 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.693 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.694 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.694 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.694 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.694 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.695 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.695 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.695 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.696 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.696 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.696 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.696 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.696 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.697 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.697 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.698 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.698 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.698 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.698 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.698 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.698 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.699 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.699 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.699 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.699 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.699 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.700 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.700 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.700 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.700 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.701 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.701 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.701 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.702 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.702 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.702 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.702 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.702 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.703 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.703 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.703 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.703 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.704 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2023-08-25T09:16:09.704 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2023-08-25T09:16:09.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.704 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.705 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2023-08-25T09:16:09.705 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.705 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.705 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.705 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-25T09:16:09.706 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.706 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.706 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2023-08-25T09:16:09.706 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2023-08-25T09:16:09.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.707 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.707 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.707 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2023-08-25T09:16:09.707 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.708 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.708 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.708 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-25T09:16:09.708 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.709 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.709 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2023-08-25T09:16:09.709 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2023-08-25T09:16:09.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.710 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.710 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2023-08-25T09:16:09.710 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.710 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.710 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.711 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2023-08-25T09:16:09.711 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.711 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.712 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2023-08-25T09:16:09.712 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2023-08-25T09:16:09.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.712 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.713 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2023-08-25T09:16:09.713 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.713 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.713 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.713 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-25T09:16:09.714 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.714 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.714 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2023-08-25T09:16:09.715 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2023-08-25T09:16:09.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.715 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.715 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.715 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.716 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.716 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.716 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-25T09:16:09.716 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.717 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.717 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:09.717 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:09.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.718 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.718 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:09.718 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.718 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.718 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.719 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2023-08-25T09:16:09.719 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.719 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.720 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2023-08-25T09:16:09.720 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2023-08-25T09:16:09.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.720 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.720 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.721 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2023-08-25T09:16:09.721 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.721 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.721 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.721 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-25T09:16:09.721 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2023-08-25T09:16:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2023-08-25T09:16:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2023-08-25T09:16:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2023-08-25T09:16:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2023-08-25T09:16:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2023-08-25T09:16:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2023-08-25T09:16:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2023-08-25T09:16:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2023-08-25T09:16:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2023-08-25T09:16:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2023-08-25T09:16:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2023-08-25T09:16:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2023-08-25T09:16:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-25T09:16:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2023-08-25T09:16:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2023-08-25T09:16:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2023-08-25T09:16:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-25T09:16:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2023-08-25T09:16:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2023-08-25T09:16:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2023-08-25T09:16:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2023-08-25T09:16:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2023-08-25T09:16:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-25T09:16:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2023-08-25T09:16:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2023-08-25T09:16:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2023-08-25T09:16:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2023-08-25T09:16:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2023-08-25T09:16:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2023-08-25T09:16:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2023-08-25T09:16:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2023-08-25T09:16:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2023-08-25T09:16:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2023-08-25T09:16:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T09:16:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2023-08-25T09:16:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2023-08-25T09:16:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2023-08-25T09:16:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-25T09:16:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2023-08-25T09:16:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2023-08-25T09:16:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2023-08-25T09:16:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2023-08-25T09:16:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2023-08-25T09:16:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2023-08-25T09:16:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T09:16:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2023-08-25T09:16:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2023-08-25T09:16:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2023-08-25T09:16:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2023-08-25T09:16:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2023-08-25T09:16:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2023-08-25T09:16:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-25T09:16:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2023-08-25T09:16:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2023-08-25T09:16:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2023-08-25T09:16:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-25T09:16:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2023-08-25T09:16:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2023-08-25T09:16:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2023-08-25T09:16:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2023-08-25T09:16:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2023-08-25T09:16:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-25T09:16:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2023-08-25T09:16:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2023-08-25T09:16:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2023-08-25T09:16:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-25T09:16:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2023-08-25T09:16:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2023-08-25T09:16:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2023-08-25T09:16:09.857 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.857 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.857 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.857 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2023-08-25T09:16:09.857 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.858 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.858 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2023-08-25T09:16:09.858 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2023-08-25T09:16:09.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:09.859 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.859 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2023-08-25T09:16:09.859 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.859 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.860 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.860 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2023-08-25T09:16:09.860 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.860 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.860 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.861 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2023-08-25T09:16:09.861 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2023-08-25T09:16:09.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.861 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.862 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.862 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.862 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.862 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.863 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2023-08-25T09:16:09.863 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.863 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.864 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2023-08-25T09:16:09.864 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2023-08-25T09:16:09.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.865 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.865 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2023-08-25T09:16:09.865 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.865 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.866 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.866 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-25T09:16:09.866 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-25T09:16:09.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.866 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.867 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2023-08-25T09:16:09.868 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.868 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.868 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.868 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.868 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.869 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.869 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.869 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.870 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.870 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.870 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.870 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.870 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.871 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.871 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.871 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.871 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.871 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.872 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.872 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.872 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.872 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.873 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.873 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.874 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.874 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.874 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.874 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.875 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.875 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.875 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.876 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.876 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.876 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.876 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.877 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.877 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.877 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.877 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.877 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.878 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.878 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.878 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.878 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.878 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.879 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.879 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.879 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.880 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.880 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.880 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.882 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.882 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2023-08-25T09:16:09.883 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.883 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.883 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.883 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.883 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.884 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.884 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.884 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.885 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.885 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.885 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.885 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.885 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.886 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.886 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.886 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.886 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.886 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.886 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.887 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.887 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.887 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.888 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.888 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.888 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.888 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.889 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.889 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.889 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.889 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.890 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.890 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.890 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.891 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.891 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.891 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.891 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.891 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.892 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.892 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.892 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.892 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.892 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.893 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.893 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.893 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.893 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.893 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.894 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.894 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.894 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.895 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.895 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.895 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.895 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.895 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.895 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.896 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.896 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.896 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.896 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.897 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2023-08-25T09:16:09.897 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.898 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.898 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.898 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.898 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.898 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.899 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.899 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.899 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.899 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.899 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.900 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.900 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.900 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.900 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.900 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.901 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.901 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.901 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.901 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.901 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.902 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.902 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.902 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.902 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.903 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.903 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.903 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.903 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.904 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.904 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.904 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.905 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.905 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.905 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.906 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.906 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.906 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.906 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.906 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.906 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.907 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.907 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.907 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.907 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.907 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.908 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.908 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.908 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.909 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.909 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.909 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.910 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.910 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.910 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.910 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.910 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.910 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.911 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.912 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2023-08-25T09:16:09.912 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.912 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.912 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.913 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2023-08-25T09:16:09.913 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2023-08-25T09:16:09.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.913 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.913 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.914 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.914 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.914 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.914 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.914 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-25T09:16:09.914 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.915 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.915 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.916 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.916 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.916 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.917 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.917 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.917 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.917 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.917 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.918 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.918 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.918 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.918 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.918 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.918 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.919 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.919 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.919 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.919 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.920 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.920 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.920 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.920 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.921 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.921 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.921 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.921 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.921 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.922 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.922 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.922 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.922 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.923 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.923 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.923 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.923 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.923 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.924 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.924 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.924 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.924 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.924 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.925 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.925 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.925 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.925 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.925 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.926 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.926 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.926 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.927 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.927 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.927 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.927 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.928 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.928 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.928 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.928 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.928 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.929 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2023-08-25T09:16:09.929 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2023-08-25T09:16:09.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.929 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.930 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.930 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.930 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.930 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.931 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2023-08-25T09:16:09.931 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2023-08-25T09:16:09.931 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2023-08-25T09:16:09.931 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.931 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2023-08-25T09:16:09.932 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.932 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.933 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2023-08-25T09:16:09.933 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.933 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.933 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.934 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.934 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.934 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.935 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.935 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.935 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.935 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.935 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.936 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.936 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.936 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.936 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.936 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.937 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.937 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.937 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.937 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.937 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.938 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.938 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.938 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.939 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.939 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.939 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.939 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.939 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.940 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.940 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.940 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.941 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.941 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.941 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.942 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.942 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.942 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.942 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.942 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.942 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.943 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.943 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.943 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.943 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.943 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.944 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.944 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.944 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.945 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.945 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.945 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.946 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.946 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.946 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.946 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.946 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.947 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.948 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2023-08-25T09:16:09.948 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.948 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.948 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.948 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.949 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.949 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.949 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.950 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.950 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.950 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.950 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.950 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.951 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.951 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.951 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.951 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.951 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.952 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.952 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.952 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.952 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.952 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.953 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.953 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.954 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.954 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.954 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.954 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.954 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.955 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.955 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.955 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.955 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.956 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.956 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.956 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.957 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.957 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.957 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.957 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.957 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.958 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.958 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.958 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.958 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.958 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.959 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.959 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.959 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.959 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.960 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.960 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.960 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.961 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.961 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.961 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.961 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.961 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.962 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.963 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2023-08-25T09:16:09.963 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.963 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.963 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.963 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.963 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.964 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.964 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.964 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.965 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.965 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.965 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.965 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.965 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.966 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.966 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.966 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.966 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.966 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.967 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.967 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.967 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.967 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.968 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.968 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.968 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.968 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.972 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.972 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.972 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.972 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.972 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.973 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.973 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.973 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.973 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.974 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.974 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:09.974 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:09.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.975 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.975 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.975 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.975 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.975 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.976 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.976 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.976 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.977 INFO:teuthology.orchestra.run.smithi099.stdout: "noautoscale": { 2023-08-25T09:16:09.977 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "noautoscale", 2023-08-25T09:16:09.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.977 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.977 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.978 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.978 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.978 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.978 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.978 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "global autoscale flag", 2023-08-25T09:16:09.979 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-25T09:16:09.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.979 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.979 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:09.979 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:09.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:09.980 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.980 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:09.980 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2023-08-25T09:16:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2023-08-25T09:16:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2023-08-25T09:16:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2023-08-25T09:16:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2023-08-25T09:16:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-25T09:16:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2023-08-25T09:16:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2023-08-25T09:16:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2023-08-25T09:16:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-25T09:16:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2023-08-25T09:16:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2023-08-25T09:16:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.992 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:09.992 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:09.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.992 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.993 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:09.993 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.993 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.993 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:09.993 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:09.993 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:09.994 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:09.994 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:09.994 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:09.994 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:09.994 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:09.995 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.995 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.995 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.996 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:09.996 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:09.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:09.996 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.997 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:09.997 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:09.997 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:09.997 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:09.997 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:09.998 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:09.998 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:09.998 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:09.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:09.998 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:09.998 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:09.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:09.999 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:09.999 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:09.999 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:09.999 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.000 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.000 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.000 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.000 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.000 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.001 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.001 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.001 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.001 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.001 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.002 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.002 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.002 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.003 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.003 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.003 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.003 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.004 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.004 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.004 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.004 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.004 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.005 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.005 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2023-08-25T09:16:10.005 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2023-08-25T09:16:10.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.006 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.006 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2023-08-25T09:16:10.006 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.006 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.007 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.007 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2023-08-25T09:16:10.007 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.007 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.007 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.008 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:10.008 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:10.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.008 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.009 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:10.009 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.009 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.009 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.009 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2023-08-25T09:16:10.010 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.010 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.010 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.011 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2023-08-25T09:16:10.011 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.011 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.011 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2023-08-25T09:16:10.012 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2023-08-25T09:16:10.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.012 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.013 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.013 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.013 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.013 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.013 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.014 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.014 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.014 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.014 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2023-08-25T09:16:10.015 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2023-08-25T09:16:10.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.015 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.015 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.015 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.016 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.016 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.016 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-25T09:16:10.016 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-25T09:16:10.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.017 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.017 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.017 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.018 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.018 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.018 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.018 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.018 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.019 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.019 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.019 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.019 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.019 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.019 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.020 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.020 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.020 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.020 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.021 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.021 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.021 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.022 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.022 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.022 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.022 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.023 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.023 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.023 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.023 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.023 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.024 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.024 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.024 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.025 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.025 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.025 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.025 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.026 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.026 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.026 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.026 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.026 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.027 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.027 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.027 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.027 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.028 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.028 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.028 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.029 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.029 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.029 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.029 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.031 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2023-08-25T09:16:10.031 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2023-08-25T09:16:10.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.031 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.031 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.033 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.033 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.033 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-25T09:16:10.034 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-25T09:16:10.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.034 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.034 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2023-08-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2023-08-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2023-08-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:10.037 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.037 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2023-08-25T09:16:10.037 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.037 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.039 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:10.039 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:10.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.039 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.040 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2023-08-25T09:16:10.040 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.040 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.040 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.040 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-25T09:16:10.041 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.041 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.041 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2023-08-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2023-08-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2023-08-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2023-08-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.046 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.046 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.046 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.046 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.046 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.047 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2023-08-25T09:16:10.047 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2023-08-25T09:16:10.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.047 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2023-08-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2023-08-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2023-08-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2023-08-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2023-08-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2023-08-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2023-08-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2023-08-25T09:16:10.052 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.052 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.052 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.052 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2023-08-25T09:16:10.054 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.054 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.054 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.054 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.054 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.055 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.055 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.055 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.055 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.056 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.056 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.056 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.056 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.056 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.057 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.057 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.057 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.057 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.057 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.058 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.058 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.058 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.059 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.059 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.059 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.059 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.060 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.060 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.060 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.060 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.060 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.061 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.061 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.062 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.062 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.062 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.062 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.063 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.063 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.063 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.063 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.063 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.063 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.064 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.064 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.064 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.064 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.065 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.065 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.065 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.066 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.066 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.066 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.066 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.067 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.067 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.067 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.067 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.067 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2023-08-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2023-08-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.069 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:10.069 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.069 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.069 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.069 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2023-08-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2023-08-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.072 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.072 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.072 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.072 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.072 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.072 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2023-08-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2023-08-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.076 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2023-08-25T09:16:10.076 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2023-08-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2023-08-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2023-08-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2023-08-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.082 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.082 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.082 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.083 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.083 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.084 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.084 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.084 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.084 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.084 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.085 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.085 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.085 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.085 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.086 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.086 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.086 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.086 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.086 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.087 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.087 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.087 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.088 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.088 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.088 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.089 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.089 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.089 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.089 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.090 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.090 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.090 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.090 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.091 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.091 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.092 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.092 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.092 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.092 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.092 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.093 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.093 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.093 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.093 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.094 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.094 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.094 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.094 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.095 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.095 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.096 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.096 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.096 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.096 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.096 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.097 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.097 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.097 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.097 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.098 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:10.098 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:10.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.098 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.099 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.099 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.099 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.099 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.099 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.100 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.100 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.100 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.100 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:10.101 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:10.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.101 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.102 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.102 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.102 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.102 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.102 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.103 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.103 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.103 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.103 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.104 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2023-08-25T09:16:10.104 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.105 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.105 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.105 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.105 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.106 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.106 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.106 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.106 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.106 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.107 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.107 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.107 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.107 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.107 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.108 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.108 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.108 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.108 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.108 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.109 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.109 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.109 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.110 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.110 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.110 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.110 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.110 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.111 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.111 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.111 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.112 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.112 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.112 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.112 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.113 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.113 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.113 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.113 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.113 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.113 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.114 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.114 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.114 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.114 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.114 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.115 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.115 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.115 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.115 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.116 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.116 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.116 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.117 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.117 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.117 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.117 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.117 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.118 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.118 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.119 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2023-08-25T09:16:10.119 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.119 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.119 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.120 INFO:teuthology.orchestra.run.smithi099.stdout: "drive_group_interval": { 2023-08-25T09:16:10.120 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "drive_group_interval", 2023-08-25T09:16:10.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:10.120 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.120 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.121 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300.0", 2023-08-25T09:16:10.121 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.121 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.121 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.121 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-25T09:16:10.122 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.122 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.122 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.122 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.123 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.123 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.123 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.124 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.124 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.124 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.124 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.124 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.125 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.125 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.125 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.125 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.125 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.125 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.126 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.126 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.126 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.126 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.127 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.127 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.127 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.128 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.128 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.128 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.128 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.128 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.129 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.129 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.129 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.129 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.130 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.130 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.130 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.130 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.130 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.131 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.131 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.131 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.131 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.131 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.132 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.132 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.132 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.132 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.132 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.133 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.133 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.133 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.134 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.134 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.134 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.134 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.135 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.135 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.135 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.135 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.135 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.135 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.136 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2023-08-25T09:16:10.136 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2023-08-25T09:16:10.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.136 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.137 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.137 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2023-08-25T09:16:10.137 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.137 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.137 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.138 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-25T09:16:10.138 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.138 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.138 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.139 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2023-08-25T09:16:10.140 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.140 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.140 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.140 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.140 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.141 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.141 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.141 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.141 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.142 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.142 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.142 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.142 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.142 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.143 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.143 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.143 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.143 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.143 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.144 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.144 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.144 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.145 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.145 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.145 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.145 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.146 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.146 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.146 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.146 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.147 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.147 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.147 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.148 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.148 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.148 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.148 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.149 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.149 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.149 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.149 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.149 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.150 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.150 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.150 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.150 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.150 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.151 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.151 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.151 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.152 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.152 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.152 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.152 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.152 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.153 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.153 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.153 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.154 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2023-08-25T09:16:10.154 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2023-08-25T09:16:10.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.154 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.155 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.155 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.155 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.155 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.155 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.155 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.156 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.156 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2023-08-25T09:16:10.156 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2023-08-25T09:16:10.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.157 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.157 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2023-08-25T09:16:10.157 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.158 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.158 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.158 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.158 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.159 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.159 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2023-08-25T09:16:10.159 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2023-08-25T09:16:10.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.159 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.160 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.160 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.160 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.160 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.161 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.161 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.161 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.161 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.162 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2023-08-25T09:16:10.162 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2023-08-25T09:16:10.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.162 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.162 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.163 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.163 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.163 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.163 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.163 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.164 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.164 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2023-08-25T09:16:10.164 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2023-08-25T09:16:10.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:10.165 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.165 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.165 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.165 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.166 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.166 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.166 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.166 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.167 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2023-08-25T09:16:10.167 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2023-08-25T09:16:10.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.167 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.168 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.168 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.168 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.168 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.168 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.169 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.169 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.169 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.169 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2023-08-25T09:16:10.170 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2023-08-25T09:16:10.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.170 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.170 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.170 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.171 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.171 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.171 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.171 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.171 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.172 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.172 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.172 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2023-08-25T09:16:10.172 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2023-08-25T09:16:10.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.173 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.173 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.173 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.173 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.174 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.174 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.174 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.174 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.175 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2023-08-25T09:16:10.175 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2023-08-25T09:16:10.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.175 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.176 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.176 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:10.176 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2023-08-25T09:16:10.176 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.177 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.177 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.177 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.177 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.178 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2023-08-25T09:16:10.178 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2023-08-25T09:16:10.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.178 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.179 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.179 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.179 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.179 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.179 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.179 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.180 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.180 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.180 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2023-08-25T09:16:10.180 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2023-08-25T09:16:10.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.181 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.181 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.181 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.182 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.182 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.182 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.182 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.182 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.183 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.183 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2023-08-25T09:16:10.183 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2023-08-25T09:16:10.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.183 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.184 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.184 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.184 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.184 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.185 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.185 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.185 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.185 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.185 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.186 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2023-08-25T09:16:10.186 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.186 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.187 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.187 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2023-08-25T09:16:10.187 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2023-08-25T09:16:10.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.187 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.188 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.188 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.188 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.188 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.189 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2023-08-25T09:16:10.189 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.189 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.190 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2023-08-25T09:16:10.190 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2023-08-25T09:16:10.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.190 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.191 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.191 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.191 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.191 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.191 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2023-08-25T09:16:10.191 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.192 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.192 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.192 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.194 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.194 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.194 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.194 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.194 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.194 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.195 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.195 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.195 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.195 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.195 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.196 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.196 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.196 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.197 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.197 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.197 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.198 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.198 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.198 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.198 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.198 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.199 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.199 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.200 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.200 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.200 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.200 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.201 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.201 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.201 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.201 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.201 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.202 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.202 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.202 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.202 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.202 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.203 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.203 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.203 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.204 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.204 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.204 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.204 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.205 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.205 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.205 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.206 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2023-08-25T09:16:10.206 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.206 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.207 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.207 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.207 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.207 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.208 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.208 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.208 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.208 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.209 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.209 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.209 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.209 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.209 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.209 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.210 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.210 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.210 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.210 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.211 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.211 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.211 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.212 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.212 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.212 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.212 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.213 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.213 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.214 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.214 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.214 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.215 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.215 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.215 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.215 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.215 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.216 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.216 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.216 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.216 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.216 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.217 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.217 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.217 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.217 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.217 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.218 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.218 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.218 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.219 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.219 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.219 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.219 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.219 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.220 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.220 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.221 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2023-08-25T09:16:10.221 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.221 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.221 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.222 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.222 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.223 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.223 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.223 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.223 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.223 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.224 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.224 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.224 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.224 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.224 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.226 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.226 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.226 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.227 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.227 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.227 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.227 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.227 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.228 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.228 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.228 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.230 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.230 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.230 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.230 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.230 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.231 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.231 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.231 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.231 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.231 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.232 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.232 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.232 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.233 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.233 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.234 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.234 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.234 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.235 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.235 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.235 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.236 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2023-08-25T09:16:10.236 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.236 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.237 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.237 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2023-08-25T09:16:10.237 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2023-08-25T09:16:10.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.237 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.238 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-25T09:16:10.238 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.238 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.238 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.239 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.239 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.239 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:10.240 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:10.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.240 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.240 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2023-08-25T09:16:10.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.241 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.241 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.241 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.241 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.242 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.242 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.242 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.243 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.243 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.243 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.244 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.244 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.244 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.244 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.244 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2023-08-25T09:16:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2023-08-25T09:16:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2023-08-25T09:16:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-25T09:16:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2023-08-25T09:16:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2023-08-25T09:16:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-25T09:16:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2023-08-25T09:16:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2023-08-25T09:16:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-25T09:16:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2023-08-25T09:16:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2023-08-25T09:16:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-25T09:16:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2023-08-25T09:16:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2023-08-25T09:16:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-25T09:16:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2023-08-25T09:16:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2023-08-25T09:16:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2023-08-25T09:16:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2023-08-25T09:16:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2023-08-25T09:16:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2023-08-25T09:16:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-25T09:16:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2023-08-25T09:16:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2023-08-25T09:16:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2023-08-25T09:16:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2023-08-25T09:16:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2023-08-25T09:16:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2023-08-25T09:16:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2023-08-25T09:16:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2023-08-25T09:16:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2023-08-25T09:16:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2023-08-25T09:16:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2023-08-25T09:16:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2023-08-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2023-08-25T09:16:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2023-08-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2023-08-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2023-08-25T09:16:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2023-08-25T09:16:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-25T09:16:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2023-08-25T09:16:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2023-08-25T09:16:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.338 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.338 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.338 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.338 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.338 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.339 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.339 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.339 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.339 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.340 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.340 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.340 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.341 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.341 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.341 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.341 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.341 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.342 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.342 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.342 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.343 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2023-08-25T09:16:10.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2023-08-25T09:16:10.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.343 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.344 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2023-08-25T09:16:10.344 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.344 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.344 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-25T09:16:10.345 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.345 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.345 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2023-08-25T09:16:10.345 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2023-08-25T09:16:10.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.346 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.346 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:10.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.347 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.347 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-25T09:16:10.347 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.348 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.349 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2023-08-25T09:16:10.349 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.349 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.349 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.349 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2023-08-25T09:16:10.349 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2023-08-25T09:16:10.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:10.350 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.350 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2023-08-25T09:16:10.350 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.351 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.351 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.351 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.351 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2023-08-25T09:16:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2023-08-25T09:16:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.353 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.353 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.353 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.353 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.353 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.354 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.354 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.354 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.354 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.355 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:10.355 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:10.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.355 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.356 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:10.356 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.356 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.356 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.356 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.357 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.357 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.357 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.357 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.358 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.358 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.358 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.359 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.359 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.359 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.359 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.359 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.360 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.360 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.360 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.360 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.360 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.361 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.361 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.362 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.362 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.362 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.363 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.363 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.363 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.364 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.364 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.364 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.365 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.365 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.365 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.365 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.366 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.366 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.366 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.366 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.366 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.367 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.367 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.367 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.367 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.367 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.368 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.368 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.368 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.368 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.369 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.369 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.369 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.369 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.370 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.370 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.370 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.370 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.371 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2023-08-25T09:16:10.371 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2023-08-25T09:16:10.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.371 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.372 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.372 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.372 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.372 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.373 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.373 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.373 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.373 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2023-08-25T09:16:10.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2023-08-25T09:16:10.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.374 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2023-08-25T09:16:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2023-08-25T09:16:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2023-08-25T09:16:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-25T09:16:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "gid": 25199, 2023-08-25T09:16:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "z", 2023-08-25T09:16:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr_features": 4540138322906710015, 2023-08-25T09:16:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2023-08-25T09:16:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2023-08-25T09:16:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2023-08-25T09:16:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2023-08-25T09:16:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2023-08-25T09:16:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2023-08-25T09:16:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2023-08-25T09:16:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2023-08-25T09:16:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2023-08-25T09:16:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2023-08-25T09:16:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2023-08-25T09:16:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2023-08-25T09:16:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2023-08-25T09:16:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2023-08-25T09:16:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2023-08-25T09:16:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2023-08-25T09:16:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2023-08-25T09:16:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2023-08-25T09:16:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2023-08-25T09:16:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2023-08-25T09:16:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2023-08-25T09:16:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2023-08-25T09:16:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2023-08-25T09:16:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2023-08-25T09:16:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2023-08-25T09:16:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-25T09:16:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2023-08-25T09:16:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2023-08-25T09:16:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2023-08-25T09:16:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2023-08-25T09:16:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2023-08-25T09:16:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2023-08-25T09:16:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2023-08-25T09:16:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2023-08-25T09:16:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2023-08-25T09:16:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2023-08-25T09:16:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2023-08-25T09:16:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T09:16:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2023-08-25T09:16:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2023-08-25T09:16:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2023-08-25T09:16:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2023-08-25T09:16:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-25T09:16:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T09:16:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2023-08-25T09:16:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2023-08-25T09:16:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2023-08-25T09:16:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2023-08-25T09:16:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-25T09:16:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2023-08-25T09:16:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2023-08-25T09:16:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2023-08-25T09:16:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-25T09:16:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-25T09:16:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2023-08-25T09:16:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2023-08-25T09:16:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2023-08-25T09:16:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2023-08-25T09:16:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2023-08-25T09:16:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2023-08-25T09:16:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-25T09:16:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2023-08-25T09:16:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2023-08-25T09:16:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2023-08-25T09:16:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2023-08-25T09:16:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T09:16:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2023-08-25T09:16:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2023-08-25T09:16:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2023-08-25T09:16:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2023-08-25T09:16:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-25T09:16:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T09:16:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2023-08-25T09:16:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2023-08-25T09:16:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-25T09:16:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2023-08-25T09:16:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2023-08-25T09:16:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2023-08-25T09:16:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2023-08-25T09:16:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2023-08-25T09:16:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap" 2023-08-25T09:16:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2023-08-25T09:16:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2023-08-25T09:16:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2023-08-25T09:16:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-25T09:16:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-25T09:16:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2023-08-25T09:16:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2023-08-25T09:16:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-25T09:16:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-25T09:16:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2023-08-25T09:16:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2023-08-25T09:16:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-25T09:16:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.472 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.473 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2023-08-25T09:16:10.473 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.473 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.473 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.474 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2023-08-25T09:16:10.474 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2023-08-25T09:16:10.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:10.474 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.475 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2023-08-25T09:16:10.475 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.475 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.475 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.475 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-25T09:16:10.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.476 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.476 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2023-08-25T09:16:10.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2023-08-25T09:16:10.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.477 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2023-08-25T09:16:10.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.478 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.478 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.478 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-25T09:16:10.478 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.479 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.479 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2023-08-25T09:16:10.479 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2023-08-25T09:16:10.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.480 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2023-08-25T09:16:10.480 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.480 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.481 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-25T09:16:10.481 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.481 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.482 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2023-08-25T09:16:10.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2023-08-25T09:16:10.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.482 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.483 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.483 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.483 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.483 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.483 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-25T09:16:10.484 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-25T09:16:10.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.484 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.484 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2023-08-25T09:16:10.485 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2023-08-25T09:16:10.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.485 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.485 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:10.486 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.486 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.486 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.486 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-25T09:16:10.486 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.487 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.487 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2023-08-25T09:16:10.487 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2023-08-25T09:16:10.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:10.488 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.488 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2023-08-25T09:16:10.488 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.488 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.489 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.489 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-25T09:16:10.489 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.489 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2023-08-25T09:16:10.490 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2023-08-25T09:16:10.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.490 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.491 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.491 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.491 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.492 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-25T09:16:10.492 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.492 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2023-08-25T09:16:10.493 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2023-08-25T09:16:10.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.493 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.493 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.494 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.494 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.494 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-25T09:16:10.494 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.495 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.495 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2023-08-25T09:16:10.495 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2023-08-25T09:16:10.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.496 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.496 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.496 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.496 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.497 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-25T09:16:10.497 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.497 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.498 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2023-08-25T09:16:10.498 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2023-08-25T09:16:10.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.498 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.499 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-25T09:16:10.499 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.499 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.499 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.500 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:10.500 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.500 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.501 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2023-08-25T09:16:10.501 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2023-08-25T09:16:10.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.501 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.502 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-25T09:16:10.502 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.502 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.502 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.503 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2023-08-25T09:16:10.503 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.503 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.504 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2023-08-25T09:16:10.504 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2023-08-25T09:16:10.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.504 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.505 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-25T09:16:10.505 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.505 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.505 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.505 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2023-08-25T09:16:10.506 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.506 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.506 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2023-08-25T09:16:10.507 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2023-08-25T09:16:10.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.507 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.507 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-25T09:16:10.508 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.508 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.508 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.508 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:10.508 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.509 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.509 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.509 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2023-08-25T09:16:10.509 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2023-08-25T09:16:10.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.510 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.510 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-25T09:16:10.510 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.510 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.511 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.511 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2023-08-25T09:16:10.511 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.511 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.512 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2023-08-25T09:16:10.512 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2023-08-25T09:16:10.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.512 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.513 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-25T09:16:10.513 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.513 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.513 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.514 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2023-08-25T09:16:10.514 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.514 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.515 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2023-08-25T09:16:10.515 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2023-08-25T09:16:10.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.515 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.515 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-25T09:16:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2023-08-25T09:16:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2023-08-25T09:16:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2023-08-25T09:16:10.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.518 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.518 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-25T09:16:10.518 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.519 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.519 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.519 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2023-08-25T09:16:10.519 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.520 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.520 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2023-08-25T09:16:10.520 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2023-08-25T09:16:10.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.521 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.521 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-25T09:16:10.521 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.521 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.522 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.522 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2023-08-25T09:16:10.522 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.522 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.523 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2023-08-25T09:16:10.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2023-08-25T09:16:10.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.523 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.524 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-25T09:16:10.524 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.524 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.524 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.524 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2023-08-25T09:16:10.525 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.525 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.525 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2023-08-25T09:16:10.526 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2023-08-25T09:16:10.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.526 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.526 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-25T09:16:10.527 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.527 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.527 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:10.527 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.528 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.528 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.528 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2023-08-25T09:16:10.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2023-08-25T09:16:10.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.529 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.529 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-08-25T09:16:10.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.530 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.530 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.530 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2023-08-25T09:16:10.530 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2023-08-25T09:16:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2023-08-25T09:16:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-25T09:16:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2023-08-25T09:16:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2023-08-25T09:16:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-25T09:16:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.537 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.537 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2023-08-25T09:16:10.537 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.537 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2023-08-25T09:16:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2023-08-25T09:16:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-25T09:16:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2023-08-25T09:16:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2023-08-25T09:16:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2023-08-25T09:16:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2023-08-25T09:16:10.543 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.543 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.544 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2023-08-25T09:16:10.544 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2023-08-25T09:16:10.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.544 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.545 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:10.545 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.545 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.546 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.546 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-25T09:16:10.546 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.547 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.547 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2023-08-25T09:16:10.548 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2023-08-25T09:16:10.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.549 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.549 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2023-08-25T09:16:10.550 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.550 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.551 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.551 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-25T09:16:10.551 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.552 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2023-08-25T09:16:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2023-08-25T09:16:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.554 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2023-08-25T09:16:10.554 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.555 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.555 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.555 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-25T09:16:10.556 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.556 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.557 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.557 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2023-08-25T09:16:10.557 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2023-08-25T09:16:10.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.558 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.558 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2023-08-25T09:16:10.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.559 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.559 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.560 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2023-08-25T09:16:10.560 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.560 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.561 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2023-08-25T09:16:10.561 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2023-08-25T09:16:10.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.562 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.562 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.562 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.563 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.563 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.563 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-25T09:16:10.563 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.564 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.564 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2023-08-25T09:16:10.564 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2023-08-25T09:16:10.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.565 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.565 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:10.565 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.566 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.566 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.566 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2023-08-25T09:16:10.566 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.567 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.567 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2023-08-25T09:16:10.568 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2023-08-25T09:16:10.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.568 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.570 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.570 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.570 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2023-08-25T09:16:10.570 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.571 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.571 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.571 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.572 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.572 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.572 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.572 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.573 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.573 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.573 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.573 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.573 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.574 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.574 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.574 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.574 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.574 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.575 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.575 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2023-08-25T09:16:10.575 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2023-08-25T09:16:10.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.576 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.576 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.576 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.577 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.577 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.577 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-25T09:16:10.577 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.578 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.578 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.578 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.579 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.579 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.579 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.580 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.580 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.580 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-25T09:16:10.580 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.581 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.581 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.581 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.582 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.582 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.583 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.583 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.583 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.583 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.584 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.584 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.584 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.584 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.584 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.585 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.585 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.585 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.586 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.586 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.586 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.587 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.587 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.587 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.588 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.588 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.589 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.589 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.589 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.590 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.591 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-25T09:16:10.591 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-25T09:16:10.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.592 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.592 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.593 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.593 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.594 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.594 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.594 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-25T09:16:10.595 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.595 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.596 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-25T09:16:10.596 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-25T09:16:10.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.597 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.597 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2023-08-25T09:16:10.597 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.597 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.598 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.598 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-25T09:16:10.598 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.598 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.599 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.599 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2023-08-25T09:16:10.599 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2023-08-25T09:16:10.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.600 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.600 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:10.600 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.600 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.601 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.601 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2023-08-25T09:16:10.601 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.601 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.601 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.602 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2023-08-25T09:16:10.602 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2023-08-25T09:16:10.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.602 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.603 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.603 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:10.603 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.603 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.603 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.604 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-25T09:16:10.604 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.604 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.604 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.605 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2023-08-25T09:16:10.605 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2023-08-25T09:16:10.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.605 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.606 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.606 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.606 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.606 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.606 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2023-08-25T09:16:10.607 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.607 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.607 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2023-08-25T09:16:10.608 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2023-08-25T09:16:10.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.608 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.608 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2023-08-25T09:16:10.609 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.609 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.609 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.609 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2023-08-25T09:16:10.609 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2023-08-25T09:16:10.610 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.610 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2023-08-25T09:16:10.610 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.610 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.611 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2023-08-25T09:16:10.611 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2023-08-25T09:16:10.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.612 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.612 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-25T09:16:10.612 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.612 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.613 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.613 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-25T09:16:10.613 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.613 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.614 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.614 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2023-08-25T09:16:10.614 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2023-08-25T09:16:10.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.615 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.615 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.615 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.616 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.616 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.616 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-25T09:16:10.616 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.617 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.617 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2023-08-25T09:16:10.617 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2023-08-25T09:16:10.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.618 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.618 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.618 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.618 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.619 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.619 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-25T09:16:10.619 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.620 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.620 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2023-08-25T09:16:10.620 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2023-08-25T09:16:10.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.621 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.621 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.621 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.621 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.622 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.622 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-25T09:16:10.622 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.622 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.623 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2023-08-25T09:16:10.623 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2023-08-25T09:16:10.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.623 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.624 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.624 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.624 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.624 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.624 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-25T09:16:10.625 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.625 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.626 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2023-08-25T09:16:10.626 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2023-08-25T09:16:10.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.627 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.628 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.628 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.629 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.629 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.630 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-25T09:16:10.630 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.630 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.631 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.632 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2023-08-25T09:16:10.632 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2023-08-25T09:16:10.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.633 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.634 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2023-08-25T09:16:10.634 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.634 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.635 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.635 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2023-08-25T09:16:10.636 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.636 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2023-08-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2023-08-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.640 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.641 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2023-08-25T09:16:10.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2023-08-25T09:16:10.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.641 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.642 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.642 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.642 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.643 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.643 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-25T09:16:10.643 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.643 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.644 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2023-08-25T09:16:10.645 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2023-08-25T09:16:10.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.645 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.645 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.645 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.646 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.646 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.646 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.646 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-25T09:16:10.647 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.647 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.647 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2023-08-25T09:16:10.648 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2023-08-25T09:16:10.648 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.648 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.648 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.648 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.649 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.649 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.649 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.649 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2023-08-25T09:16:10.649 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.650 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.650 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2023-08-25T09:16:10.650 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2023-08-25T09:16:10.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.651 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.652 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.652 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.652 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.653 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.653 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-25T09:16:10.653 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.654 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.654 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2023-08-25T09:16:10.654 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2023-08-25T09:16:10.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.655 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.655 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.655 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.655 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.656 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.656 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.656 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-25T09:16:10.656 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.656 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.657 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.658 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2023-08-25T09:16:10.658 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.658 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.658 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.658 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.658 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.659 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.659 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.659 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.659 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.660 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.660 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.660 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.660 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.660 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.661 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.661 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.661 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.661 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.661 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.662 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.662 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.662 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.663 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.663 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.663 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.663 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.664 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.664 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.664 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.664 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.664 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.665 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.665 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.665 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.666 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.666 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.666 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.667 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.667 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.667 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.667 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.667 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.668 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.668 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.668 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.668 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.668 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.669 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.669 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.669 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.670 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.670 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.670 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.671 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.671 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.671 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.672 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.672 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.673 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.673 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.674 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2023-08-25T09:16:10.674 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2023-08-25T09:16:10.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.675 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.675 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.675 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2023-08-25T09:16:10.676 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.676 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.677 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.677 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-25T09:16:10.677 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.678 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.679 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2023-08-25T09:16:10.679 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2023-08-25T09:16:10.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.680 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.680 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2023-08-25T09:16:10.680 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.681 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.681 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.681 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-25T09:16:10.681 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.682 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.683 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2023-08-25T09:16:10.683 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.683 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.683 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.684 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-25T09:16:10.684 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-25T09:16:10.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.684 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.685 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:10.685 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.685 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.685 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.685 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.686 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.686 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.686 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2023-08-25T09:16:10.687 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-25T09:16:10.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.687 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.688 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.688 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.689 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.689 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.690 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.690 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.690 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.691 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-25T09:16:10.691 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-25T09:16:10.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.691 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.692 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.692 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.692 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.692 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.692 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.693 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.693 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.693 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.693 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.694 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2023-08-25T09:16:10.694 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2023-08-25T09:16:10.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.694 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.695 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.695 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.695 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.695 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.695 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.696 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.696 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.697 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-25T09:16:10.697 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-25T09:16:10.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.697 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.697 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.697 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.698 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.698 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.698 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.698 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.698 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.699 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.699 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2023-08-25T09:16:10.699 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-25T09:16:10.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.700 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.700 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.700 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.701 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.701 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.701 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.702 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.702 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.702 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.703 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-25T09:16:10.703 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-25T09:16:10.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.703 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.704 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.704 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.704 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.704 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.704 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.705 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.705 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.705 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.706 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-25T09:16:10.706 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-25T09:16:10.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.706 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.707 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.707 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.707 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.707 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.708 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.708 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.708 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.709 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.709 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-25T09:16:10.709 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-25T09:16:10.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.710 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.711 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.711 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.711 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.712 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.712 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.712 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.712 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.713 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.713 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-25T09:16:10.713 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-25T09:16:10.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.714 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.714 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.714 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.715 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.715 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.716 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.716 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.716 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.717 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-25T09:16:10.717 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-25T09:16:10.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.717 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.718 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.718 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.718 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.718 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.719 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.719 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.720 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.720 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-25T09:16:10.720 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-25T09:16:10.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-25T09:16:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-25T09:16:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-25T09:16:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-25T09:16:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2023-08-25T09:16:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-25T09:16:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-25T09:16:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-25T09:16:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2023-08-25T09:16:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2023-08-25T09:16:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2023-08-25T09:16:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-25T09:16:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-25T09:16:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-25T09:16:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-25T09:16:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-25T09:16:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-25T09:16:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-25T09:16:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-25T09:16:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-25T09:16:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2023-08-25T09:16:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-25T09:16:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-25T09:16:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-25T09:16:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-25T09:16:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-25T09:16:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-25T09:16:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-25T09:16:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-25T09:16:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-25T09:16:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-25T09:16:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-25T09:16:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-25T09:16:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-25T09:16:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-25T09:16:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-25T09:16:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-25T09:16:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-25T09:16:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2023-08-25T09:16:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-25T09:16:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-25T09:16:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-25T09:16:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-25T09:16:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-25T09:16:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-25T09:16:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-25T09:16:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-25T09:16:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2023-08-25T09:16:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-25T09:16:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-25T09:16:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2023-08-25T09:16:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2023-08-25T09:16:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-25T09:16:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-25T09:16:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-25T09:16:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2023-08-25T09:16:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-25T09:16:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2023-08-25T09:16:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-25T09:16:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-25T09:16:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-25T09:16:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-25T09:16:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-25T09:16:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-25T09:16:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-25T09:16:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2023-08-25T09:16:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2023-08-25T09:16:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2023-08-25T09:16:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2023-08-25T09:16:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2023-08-25T09:16:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2023-08-25T09:16:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2023-08-25T09:16:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2023-08-25T09:16:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2023-08-25T09:16:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2023-08-25T09:16:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2023-08-25T09:16:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2023-08-25T09:16:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2023-08-25T09:16:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2023-08-25T09:16:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2023-08-25T09:16:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2023-08-25T09:16:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2023-08-25T09:16:10.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.855 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.855 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.855 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2023-08-25T09:16:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2023-08-25T09:16:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2023-08-25T09:16:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2023-08-25T09:16:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2023-08-25T09:16:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2023-08-25T09:16:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2023-08-25T09:16:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2023-08-25T09:16:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2023-08-25T09:16:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2023-08-25T09:16:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.871 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.871 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2023-08-25T09:16:10.871 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.871 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.872 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.872 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.873 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2023-08-25T09:16:10.873 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2023-08-25T09:16:10.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.873 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.874 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2023-08-25T09:16:10.874 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2023-08-25T09:16:10.874 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2023-08-25T09:16:10.874 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.874 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.875 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.875 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.875 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2023-08-25T09:16:10.876 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2023-08-25T09:16:10.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.876 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.876 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.877 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.877 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.877 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.877 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.878 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.878 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.879 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2023-08-25T09:16:10.879 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.879 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.880 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.880 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2023-08-25T09:16:10.880 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2023-08-25T09:16:10.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.880 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.881 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.881 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.881 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.881 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.881 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.882 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2023-08-25T09:16:10.882 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.882 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.883 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.883 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.883 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.884 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.885 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.885 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.886 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.886 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.886 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.887 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.887 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.887 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.888 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.888 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.889 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.889 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.890 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.891 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.891 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.892 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.892 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.893 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.893 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.893 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.894 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.894 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.894 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.895 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.895 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.896 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.896 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.897 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.898 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.898 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.899 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.899 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.899 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.900 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.900 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.900 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.900 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.900 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.901 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.901 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.901 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.901 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.901 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.902 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.902 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.902 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.903 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.903 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.903 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.903 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.903 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.904 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.904 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.904 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2023-08-25T09:16:10.905 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2023-08-25T09:16:10.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.905 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.905 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2023-08-25T09:16:10.906 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.906 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.906 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.906 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-25T09:16:10.906 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.907 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.907 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.907 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2023-08-25T09:16:10.907 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2023-08-25T09:16:10.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.908 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.908 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2023-08-25T09:16:10.908 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.908 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.909 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.909 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-25T09:16:10.909 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.909 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.910 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2023-08-25T09:16:10.910 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2023-08-25T09:16:10.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.910 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.911 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2023-08-25T09:16:10.911 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.911 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.911 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.912 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2023-08-25T09:16:10.912 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.912 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.912 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.913 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2023-08-25T09:16:10.913 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2023-08-25T09:16:10.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.913 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.913 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.913 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2023-08-25T09:16:10.914 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.914 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.914 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.914 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-25T09:16:10.914 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.915 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.915 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2023-08-25T09:16:10.915 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2023-08-25T09:16:10.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.916 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.916 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:10.916 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.917 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.917 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.917 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-25T09:16:10.917 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.917 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.918 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:10.918 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:10.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.919 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.919 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.919 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:10.919 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.919 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.920 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.920 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2023-08-25T09:16:10.920 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.920 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.920 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.921 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2023-08-25T09:16:10.921 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2023-08-25T09:16:10.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.921 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.922 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2023-08-25T09:16:10.922 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.922 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.922 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.923 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-25T09:16:10.923 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.923 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.924 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2023-08-25T09:16:10.924 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.924 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.925 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.925 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.925 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.925 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.926 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.926 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.926 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.926 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.927 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.927 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.927 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.927 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.927 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.927 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.928 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.928 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.928 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.928 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.929 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.929 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.929 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.930 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.930 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.930 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.930 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.931 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.931 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.931 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.931 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.932 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.932 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.932 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.933 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.933 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.933 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.933 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.933 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.933 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.934 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.934 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.934 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.934 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.934 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.935 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.935 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.935 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.935 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.936 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.936 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.936 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.937 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.937 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.937 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.937 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.937 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.938 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.938 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.938 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2023-08-25T09:16:10.938 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2023-08-25T09:16:10.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.939 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.939 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2023-08-25T09:16:10.939 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.940 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.940 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.940 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.940 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.941 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.941 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2023-08-25T09:16:10.941 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2023-08-25T09:16:10.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.942 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.942 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2023-08-25T09:16:10.942 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.942 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.942 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.943 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.943 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.943 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.944 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:10.944 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:10.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.944 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.944 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.945 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:10.945 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.945 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.945 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.945 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.946 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.946 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.947 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2023-08-25T09:16:10.947 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2023-08-25T09:16:10.947 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2023-08-25T09:16:10.947 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.948 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2023-08-25T09:16:10.948 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2023-08-25T09:16:10.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.948 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.949 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2023-08-25T09:16:10.949 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.949 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.949 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.949 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-25T09:16:10.950 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.950 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.950 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.950 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2023-08-25T09:16:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2023-08-25T09:16:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2023-08-25T09:16:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.952 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.952 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.952 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-25T09:16:10.952 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.952 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.953 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.953 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2023-08-25T09:16:10.953 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2023-08-25T09:16:10.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.954 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.954 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.954 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.954 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.955 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.955 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2023-08-25T09:16:10.955 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.955 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.956 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:10.956 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:10.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:10.956 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.957 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2023-08-25T09:16:10.957 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2023-08-25T09:16:10.957 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.957 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.958 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-25T09:16:10.958 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.958 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.958 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.959 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.959 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.959 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.960 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.960 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.960 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.960 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.960 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.960 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.961 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.961 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.961 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.961 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.961 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.962 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.962 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.962 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.962 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.963 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.963 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.963 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.963 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.963 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.964 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.964 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.964 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.964 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.964 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.964 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.965 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.965 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.965 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.966 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.966 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.966 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.967 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.967 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.967 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.967 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.967 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.967 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.968 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.968 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.968 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.968 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.968 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.969 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.969 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.969 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:10.969 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:10.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.970 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.970 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.970 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.971 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.971 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.971 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.971 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.972 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.972 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2023-08-25T09:16:10.972 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2023-08-25T09:16:10.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.973 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.973 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.973 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.973 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.973 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.974 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2023-08-25T09:16:10.974 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.974 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.974 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.975 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2023-08-25T09:16:10.975 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2023-08-25T09:16:10.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.975 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.976 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2023-08-25T09:16:10.976 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.976 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.976 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.976 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2023-08-25T09:16:10.976 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.977 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.977 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.977 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2023-08-25T09:16:10.977 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2023-08-25T09:16:10.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.978 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.978 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2023-08-25T09:16:10.978 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.979 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.979 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.979 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T09:16:10.979 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.980 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.980 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2023-08-25T09:16:10.980 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2023-08-25T09:16:10.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:10.981 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.981 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:10.981 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:10.981 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2023-08-25T09:16:10.981 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.982 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-25T09:16:10.982 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.982 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.983 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2023-08-25T09:16:10.983 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2023-08-25T09:16:10.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.983 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.984 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.984 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.984 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.984 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.984 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2023-08-25T09:16:10.985 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.985 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.985 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.985 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2023-08-25T09:16:10.985 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2023-08-25T09:16:10.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.986 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:10.986 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2023-08-25T09:16:10.986 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.987 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.987 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.987 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T09:16:10.987 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.988 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:10.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:10.989 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2023-08-25T09:16:10.989 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:10.989 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:10.989 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:10.989 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:10.990 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:10.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.990 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.990 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:10.991 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.991 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.991 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.991 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.991 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.992 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.992 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.992 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.992 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.992 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.993 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.993 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.993 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.993 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.993 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:10.994 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:10.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:10.994 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.994 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.994 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:10.995 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.995 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.995 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:10.995 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.995 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:10.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:10.996 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:10.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:10.996 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:10.996 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:10.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:10.997 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:10.997 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:10.997 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:10.997 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:10.998 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:10.998 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:10.998 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:10.998 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:10.998 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:10.998 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:10.999 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:10.999 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:10.999 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:10.999 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:10.999 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.000 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.000 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.000 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.001 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.001 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.001 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.002 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.002 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.002 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.002 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.002 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.003 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2023-08-25T09:16:11.004 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.004 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.004 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.004 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.004 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.005 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.005 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.005 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.006 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.006 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.006 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.006 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.006 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.006 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.007 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.007 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.007 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.007 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.007 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.008 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.008 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.009 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.009 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.009 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.010 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.010 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.010 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.010 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.010 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.011 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.011 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.011 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.012 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.012 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.012 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.013 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.013 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.013 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.013 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.013 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.013 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.014 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.014 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.014 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.014 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.014 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.015 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.015 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.015 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.016 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.016 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.016 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.017 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.017 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.017 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.017 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.017 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.017 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.018 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2023-08-25T09:16:11.019 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.019 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.019 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.019 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2023-08-25T09:16:11.019 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2023-08-25T09:16:11.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.020 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.020 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2023-08-25T09:16:11.020 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.021 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.021 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.021 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-25T09:16:11.021 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.021 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.022 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2023-08-25T09:16:11.022 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2023-08-25T09:16:11.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.022 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.023 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:11.023 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2023-08-25T09:16:11.023 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.023 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.024 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-25T09:16:11.024 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.024 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.025 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.025 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.025 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.025 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.026 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.026 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.026 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.026 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.026 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.027 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.027 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.027 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.027 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.027 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.028 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.028 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.028 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.028 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.029 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.029 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.029 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.030 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.030 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.030 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.030 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.030 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.031 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.031 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.031 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.032 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.032 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.032 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.033 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.033 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.033 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.033 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.033 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.033 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.034 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.034 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.034 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.034 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.034 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.035 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.035 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.035 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.036 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.036 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.036 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.036 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.037 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.037 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.037 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.037 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.038 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.039 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2023-08-25T09:16:11.039 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.039 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.039 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.039 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2023-08-25T09:16:11.040 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2023-08-25T09:16:11.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.040 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.040 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2023-08-25T09:16:11.040 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.041 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.041 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.041 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2023-08-25T09:16:11.041 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-25T09:16:11.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.042 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.042 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.042 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.043 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.043 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.043 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.043 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.044 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.044 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.044 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.044 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.044 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.044 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.045 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.045 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.045 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.045 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.046 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.046 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.046 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.049 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.049 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.049 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.050 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.050 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.050 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.050 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.050 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.051 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.051 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.051 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.051 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.051 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.052 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.052 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.052 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.052 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.053 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.053 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.053 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.054 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.054 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.054 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.054 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.054 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.055 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.055 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.055 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2023-08-25T09:16:11.056 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2023-08-25T09:16:11.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.056 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.056 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.057 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.057 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.057 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.057 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2023-08-25T09:16:11.057 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-25T09:16:11.057 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.058 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.058 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2023-08-25T09:16:11.058 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2023-08-25T09:16:11.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.059 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.059 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2023-08-25T09:16:11.059 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.059 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.060 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.060 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2023-08-25T09:16:11.060 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.060 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.060 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.061 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2023-08-25T09:16:11.061 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2023-08-25T09:16:11.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.061 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.062 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2023-08-25T09:16:11.062 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.062 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.062 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.062 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2023-08-25T09:16:11.063 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.063 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.063 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2023-08-25T09:16:11.064 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2023-08-25T09:16:11.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.064 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.064 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2023-08-25T09:16:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.066 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.066 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2023-08-25T09:16:11.066 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2023-08-25T09:16:11.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.067 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.067 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2023-08-25T09:16:11.067 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.068 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.068 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.068 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-25T09:16:11.068 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-25T09:16:11.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.068 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.069 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2023-08-25T09:16:11.070 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.070 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.070 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.070 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.070 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.071 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.071 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.071 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.072 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.072 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.072 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.072 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.072 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.073 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.073 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.073 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.073 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.073 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.073 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.074 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.074 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.074 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.074 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.075 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.075 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.075 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.076 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.076 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.076 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.076 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.077 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.077 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.077 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.078 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.078 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.078 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.078 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.078 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.079 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.079 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.079 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.079 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.079 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.080 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.080 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.080 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.080 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.080 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.081 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.081 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.081 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.082 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.082 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.082 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.082 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.083 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.083 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.083 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.083 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.084 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2023-08-25T09:16:11.084 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.085 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.085 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.085 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.085 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.086 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.086 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.086 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.086 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.087 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.087 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.087 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.087 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.087 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.087 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.088 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.088 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.088 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.088 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.089 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.089 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.089 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.090 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.090 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.090 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.090 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.091 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.091 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.091 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.091 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.092 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.092 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.092 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.093 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.093 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.093 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.093 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.093 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.094 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.094 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.094 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.094 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.094 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.095 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.095 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.095 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.095 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.095 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.096 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.096 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.096 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.096 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.097 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.097 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.097 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.097 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.097 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.098 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.098 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.098 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.099 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2023-08-25T09:16:11.099 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.099 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.099 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.100 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.100 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.100 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.100 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.101 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.101 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.101 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.101 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.102 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.102 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.102 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.102 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.103 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.103 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.103 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.103 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.103 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.104 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.104 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.104 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.104 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.105 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.105 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.105 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.105 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.106 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.106 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.106 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.106 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.107 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.107 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.107 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.108 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.108 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.108 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.108 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.108 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.109 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.109 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.109 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.109 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.109 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.109 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.110 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.110 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.110 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.110 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.110 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.111 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.111 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.111 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.112 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.112 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.112 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.112 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.112 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.113 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.113 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.113 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.114 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2023-08-25T09:16:11.114 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.114 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.115 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.115 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2023-08-25T09:16:11.115 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2023-08-25T09:16:11.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.115 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.116 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.116 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.116 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.116 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.117 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-25T09:16:11.117 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.117 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.117 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.118 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.118 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.118 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.118 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.119 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.119 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.119 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.119 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.119 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.120 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.120 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.120 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.120 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.120 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.121 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.121 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.121 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.122 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.122 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.122 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.122 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.123 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.123 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.123 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.123 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.123 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.124 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.124 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.124 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.124 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.125 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.125 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.125 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.125 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.126 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.126 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.126 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.126 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.126 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.127 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.127 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.127 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.127 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.127 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.127 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.128 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.128 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.128 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.128 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.129 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.129 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.129 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.129 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.130 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.130 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.130 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.130 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.130 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.131 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.131 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2023-08-25T09:16:11.131 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2023-08-25T09:16:11.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.132 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.132 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.132 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.132 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.132 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.133 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.133 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2023-08-25T09:16:11.133 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2023-08-25T09:16:11.133 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2023-08-25T09:16:11.133 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.134 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2023-08-25T09:16:11.134 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.134 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.134 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.135 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2023-08-25T09:16:11.135 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.135 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.136 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.136 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.136 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.136 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.137 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.137 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.137 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.137 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.137 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.138 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.138 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.138 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.138 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.138 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.139 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.139 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.139 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.139 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.140 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.140 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.140 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.141 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.141 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.141 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.141 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.141 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.142 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.142 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.142 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.142 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.143 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.143 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.143 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.144 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.144 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.144 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.144 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.145 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.145 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.145 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.145 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.145 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.146 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.146 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.146 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.146 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.147 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.147 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.147 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.147 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.148 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.148 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.148 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.148 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.149 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.149 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.149 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.150 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2023-08-25T09:16:11.150 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.151 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.151 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.151 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.151 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.151 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.152 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.152 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.152 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.152 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.152 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.153 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.153 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.153 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.153 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.153 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.154 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.154 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.154 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.154 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.154 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.155 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.155 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.155 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.155 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.155 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.156 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.156 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.156 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.156 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.156 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.157 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.157 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.157 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.157 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.158 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.158 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.158 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.158 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.159 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.159 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.159 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.159 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.159 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.159 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.160 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.160 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.160 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.160 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.160 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.161 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.161 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.161 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.161 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.162 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.162 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.162 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.163 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.163 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.163 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.163 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.163 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.164 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.164 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.165 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2023-08-25T09:16:11.165 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.165 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.165 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.166 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.166 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.166 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.167 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.167 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.167 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.167 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.167 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.168 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.168 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.168 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.168 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.168 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.169 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.169 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.169 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.169 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.170 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.170 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.170 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.171 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.171 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.171 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.171 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.171 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.172 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.172 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.172 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.172 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.173 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.173 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.173 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.174 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.174 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.174 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.174 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.174 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.175 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.175 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.175 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.175 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.175 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.175 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.176 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.176 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.176 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.177 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.177 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.177 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.178 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.178 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.178 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.178 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.178 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.179 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.179 INFO:teuthology.orchestra.run.smithi099.stdout: "noautoscale": { 2023-08-25T09:16:11.179 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "noautoscale", 2023-08-25T09:16:11.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.180 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.180 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.180 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.181 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.181 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.181 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "global autoscale flag", 2023-08-25T09:16:11.181 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-25T09:16:11.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.182 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.182 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:11.182 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:11.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.183 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.183 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:11.183 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.183 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.183 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.184 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.184 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.184 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.185 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2023-08-25T09:16:11.185 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2023-08-25T09:16:11.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:11.185 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.186 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2023-08-25T09:16:11.186 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2023-08-25T09:16:11.186 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.186 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.186 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2023-08-25T09:16:11.187 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-25T09:16:11.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.187 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.188 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2023-08-25T09:16:11.188 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.188 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.188 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.189 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2023-08-25T09:16:11.189 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2023-08-25T09:16:11.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.189 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.189 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.190 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.190 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.190 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.190 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.190 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-25T09:16:11.190 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.191 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.191 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2023-08-25T09:16:11.191 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2023-08-25T09:16:11.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.192 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.192 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.192 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.193 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.193 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.193 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.193 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.194 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.194 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.194 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.195 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.195 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.195 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.195 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.195 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.196 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.196 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.196 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.196 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.196 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.197 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.197 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.197 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.197 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.197 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.198 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.198 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.198 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.199 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.199 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.199 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.199 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.200 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.200 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.200 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.200 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.201 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.201 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.201 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.202 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.202 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.202 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.202 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.203 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.203 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.203 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.203 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.203 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.203 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.204 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.204 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.204 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.204 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.205 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.205 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.206 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.206 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.206 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.207 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.207 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.207 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.207 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2023-08-25T09:16:11.208 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2023-08-25T09:16:11.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.208 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.208 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2023-08-25T09:16:11.209 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.209 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.209 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.209 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2023-08-25T09:16:11.209 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.210 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.210 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:11.210 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:11.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.211 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.211 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:11.211 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.211 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.212 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.212 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2023-08-25T09:16:11.212 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.212 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.213 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2023-08-25T09:16:11.214 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.214 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.214 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.214 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2023-08-25T09:16:11.214 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2023-08-25T09:16:11.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.215 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.215 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.215 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.216 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.216 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.216 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.216 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.217 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.217 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2023-08-25T09:16:11.217 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2023-08-25T09:16:11.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.218 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.218 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.218 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.218 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.219 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.219 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-25T09:16:11.219 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-25T09:16:11.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.219 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.220 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.220 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.221 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.221 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.221 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.221 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.222 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.222 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.222 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.222 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.222 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.223 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.223 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.223 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.223 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.223 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.224 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.224 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.225 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.225 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.225 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.225 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.225 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.226 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.226 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.226 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.227 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.227 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.227 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.228 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.228 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.228 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.228 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.228 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.228 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.229 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.229 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.229 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.229 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.229 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.230 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.230 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.230 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.230 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.231 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.231 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.231 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.232 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.232 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.232 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.233 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2023-08-25T09:16:11.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2023-08-25T09:16:11.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.234 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.234 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.235 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.235 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.235 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.235 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.236 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.236 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-25T09:16:11.236 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-25T09:16:11.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.236 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.237 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2023-08-25T09:16:11.237 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.237 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.237 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.238 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.238 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.238 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.239 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2023-08-25T09:16:11.239 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2023-08-25T09:16:11.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:11.239 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.239 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.240 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2023-08-25T09:16:11.240 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.240 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.240 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.240 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.241 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.241 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.241 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:11.241 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:11.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.242 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.242 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2023-08-25T09:16:11.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.243 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.243 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.243 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-25T09:16:11.243 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.244 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.244 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:11.244 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:11.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.245 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2023-08-25T09:16:11.245 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.245 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.245 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.246 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-25T09:16:11.246 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.246 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.247 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2023-08-25T09:16:11.247 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2023-08-25T09:16:11.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.247 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.248 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2023-08-25T09:16:11.248 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.248 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.248 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.248 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.249 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.249 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.249 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2023-08-25T09:16:11.250 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2023-08-25T09:16:11.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.253 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.253 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2023-08-25T09:16:11.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.254 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2023-08-25T09:16:11.254 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2023-08-25T09:16:11.254 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.255 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.256 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2023-08-25T09:16:11.256 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2023-08-25T09:16:11.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.256 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.257 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2023-08-25T09:16:11.257 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2023-08-25T09:16:11.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2023-08-25T09:16:11.257 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.257 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.257 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2023-08-25T09:16:11.259 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.259 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.259 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.260 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.260 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.260 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.261 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.261 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.261 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.261 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.261 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.262 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.262 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.262 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.262 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.262 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.262 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.263 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.263 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.263 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.264 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.264 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.264 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.265 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.265 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.265 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.265 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.266 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.266 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.266 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.267 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.267 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.267 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.268 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.268 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.268 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.268 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.268 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.269 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.269 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.269 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.269 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.270 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.270 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.270 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.270 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.271 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.271 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.271 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.271 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.272 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.272 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.272 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.273 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.273 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2023-08-25T09:16:11.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2023-08-25T09:16:11.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.274 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:11.274 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.275 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.275 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.275 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.276 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2023-08-25T09:16:11.276 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2023-08-25T09:16:11.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.276 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.277 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.277 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.278 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.278 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.278 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2023-08-25T09:16:11.279 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2023-08-25T09:16:11.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.279 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.279 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.279 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.280 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.280 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.280 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.280 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.281 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.282 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2023-08-25T09:16:11.282 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.282 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.282 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.282 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2023-08-25T09:16:11.283 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2023-08-25T09:16:11.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.283 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.284 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.284 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.284 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.284 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2023-08-25T09:16:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2023-08-25T09:16:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2023-08-25T09:16:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2023-08-25T09:16:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "drive_group_interval": { 2023-08-25T09:16:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "drive_group_interval", 2023-08-25T09:16:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300.0", 2023-08-25T09:16:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-25T09:16:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2023-08-25T09:16:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2023-08-25T09:16:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2023-08-25T09:16:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-25T09:16:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2023-08-25T09:16:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2023-08-25T09:16:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2023-08-25T09:16:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2023-08-25T09:16:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2023-08-25T09:16:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2023-08-25T09:16:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2023-08-25T09:16:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2023-08-25T09:16:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2023-08-25T09:16:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2023-08-25T09:16:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2023-08-25T09:16:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2023-08-25T09:16:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2023-08-25T09:16:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2023-08-25T09:16:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2023-08-25T09:16:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2023-08-25T09:16:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2023-08-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2023-08-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2023-08-25T09:16:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2023-08-25T09:16:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2023-08-25T09:16:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2023-08-25T09:16:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2023-08-25T09:16:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2023-08-25T09:16:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2023-08-25T09:16:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2023-08-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2023-08-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2023-08-25T09:16:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2023-08-25T09:16:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2023-08-25T09:16:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2023-08-25T09:16:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2023-08-25T09:16:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2023-08-25T09:16:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.394 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.394 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.394 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.394 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.395 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.395 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2023-08-25T09:16:11.395 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.395 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.396 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.396 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.396 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.397 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.397 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.397 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.397 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.397 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.398 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.398 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.398 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.398 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.398 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.399 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.399 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.399 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.400 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.400 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.400 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.401 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.401 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.401 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.401 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.401 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.402 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.402 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.403 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.403 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.403 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.404 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.404 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.404 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.405 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.405 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.405 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.405 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.405 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.405 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.406 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.406 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.406 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.407 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.407 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.407 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.407 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.408 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.408 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.408 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.408 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.408 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.409 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.410 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2023-08-25T09:16:11.410 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.410 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.410 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.411 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.411 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.412 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.413 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.413 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.413 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.413 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.413 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.414 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.414 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.414 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.414 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.415 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.415 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.415 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.415 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.416 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.416 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.416 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.416 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.416 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.417 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.417 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.418 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.418 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.418 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.419 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.419 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.419 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.420 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.420 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.420 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.420 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.420 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.421 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.421 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.421 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.422 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.422 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.422 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.423 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.423 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.423 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.423 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.423 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.423 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.424 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2023-08-25T09:16:11.425 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.425 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.425 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.426 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.426 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.426 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.427 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.427 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.427 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.427 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.427 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.428 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.428 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.428 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.428 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.428 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.429 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.429 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.429 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.430 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.430 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.430 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.430 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.431 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.431 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.431 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.431 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.431 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.432 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.432 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.432 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.433 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.433 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.437 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.437 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.437 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.438 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.438 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.438 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.439 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2023-08-25T09:16:11.440 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.440 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.440 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.440 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2023-08-25T09:16:11.440 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2023-08-25T09:16:11.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.441 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.441 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-25T09:16:11.441 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.443 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.443 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:11.443 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:11.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.444 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.444 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2023-08-25T09:16:11.444 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.444 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.445 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.445 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.445 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.445 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.446 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.446 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.446 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.447 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.447 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.447 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.447 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.447 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.448 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.448 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.448 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.448 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.448 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.449 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.449 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.449 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.449 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.450 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.450 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.450 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.451 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.451 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.451 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.451 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.451 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.452 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.452 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.453 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.453 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.453 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.454 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.454 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.454 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.454 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.454 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.455 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.455 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.455 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.455 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.455 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.456 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.456 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.456 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.457 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.457 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.457 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.458 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.458 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.458 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.458 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.459 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.460 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2023-08-25T09:16:11.460 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.460 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.460 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.460 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2023-08-25T09:16:11.461 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2023-08-25T09:16:11.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.461 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.461 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.462 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.462 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.462 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.462 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-25T09:16:11.462 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2023-08-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2023-08-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.464 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.464 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.464 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.464 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.465 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.465 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-25T09:16:11.465 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.465 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.466 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2023-08-25T09:16:11.466 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2023-08-25T09:16:11.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.466 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.467 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.467 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.467 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.467 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.467 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-25T09:16:11.468 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.468 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.468 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2023-08-25T09:16:11.469 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2023-08-25T09:16:11.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.469 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.469 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.470 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.470 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.470 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-25T09:16:11.470 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.471 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.471 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2023-08-25T09:16:11.471 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2023-08-25T09:16:11.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.472 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.472 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.472 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.473 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.473 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.473 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-25T09:16:11.473 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.473 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.474 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2023-08-25T09:16:11.474 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2023-08-25T09:16:11.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.474 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.475 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.475 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.475 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.475 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.476 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.476 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.477 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2023-08-25T09:16:11.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2023-08-25T09:16:11.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.478 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.478 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.478 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.478 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.478 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.479 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.479 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2023-08-25T09:16:11.479 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2023-08-25T09:16:11.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.480 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-25T09:16:11.480 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.481 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.481 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.482 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.482 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2023-08-25T09:16:11.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2023-08-25T09:16:11.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.483 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.483 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.483 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.483 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.483 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.484 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.484 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.484 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.485 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:11.485 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:11.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.485 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.486 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2023-08-25T09:16:11.486 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2023-08-25T09:16:11.486 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.486 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.486 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.487 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.487 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.488 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2023-08-25T09:16:11.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2023-08-25T09:16:11.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.488 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2023-08-25T09:16:11.489 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.489 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.489 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.489 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.490 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.490 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.490 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2023-08-25T09:16:11.490 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2023-08-25T09:16:11.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.491 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.491 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.492 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.492 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.492 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.492 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.493 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.493 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2023-08-25T09:16:11.493 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2023-08-25T09:16:11.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.494 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.494 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.494 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.495 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.495 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.495 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.495 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.496 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.496 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.497 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.497 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.497 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.497 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.498 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.498 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.498 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.498 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.498 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.499 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.499 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.499 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.499 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.500 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.500 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.501 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.501 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.501 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.501 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.501 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.502 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.502 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.502 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.503 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.503 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.503 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.504 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.504 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.504 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.504 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.504 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.504 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.505 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.505 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.505 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.505 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.506 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.506 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.506 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.507 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.507 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.507 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.508 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.508 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.508 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.509 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.509 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.509 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.509 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.509 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.510 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2023-08-25T09:16:11.510 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2023-08-25T09:16:11.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.510 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.511 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.511 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.511 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.512 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.512 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.512 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.512 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.513 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2023-08-25T09:16:11.513 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2023-08-25T09:16:11.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.513 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.514 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.514 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.514 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.514 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.515 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.515 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.515 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.516 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2023-08-25T09:16:11.516 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2023-08-25T09:16:11.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.516 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.517 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-25T09:16:11.517 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.517 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.517 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.517 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.517 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.518 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.519 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2023-08-25T09:16:11.519 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.519 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.519 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.520 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.520 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.520 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.521 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.521 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.521 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.521 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.521 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.522 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.522 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.522 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.522 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.522 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.523 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.523 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.523 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.523 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.524 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.524 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.524 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.525 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.525 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.525 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.525 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.525 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.526 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.526 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.526 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.526 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.527 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.527 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.527 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.527 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.528 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.528 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.528 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.528 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.529 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.529 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.529 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.529 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.529 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.529 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.530 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.530 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.530 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.531 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.531 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.531 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.532 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.532 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.532 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.532 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.532 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.533 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.533 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.534 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2023-08-25T09:16:11.534 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.534 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.534 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.534 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.535 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.535 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.535 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.536 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.536 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.536 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.536 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.537 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.537 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.537 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.537 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.537 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.538 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.538 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.538 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.538 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.539 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.539 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.539 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.540 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.540 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.540 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.540 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.540 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.541 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.541 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.541 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.542 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.542 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.542 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.543 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.543 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.543 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.543 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.543 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.543 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.544 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.544 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.544 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.544 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.544 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.545 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.545 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.545 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.546 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.546 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.546 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.547 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.547 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.547 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.547 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.548 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.548 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2023-08-25T09:16:11.548 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2023-08-25T09:16:11.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.548 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.549 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2023-08-25T09:16:11.549 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.549 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.549 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.550 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-25T09:16:11.550 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.550 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.551 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2023-08-25T09:16:11.551 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2023-08-25T09:16:11.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.551 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.551 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.552 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:11.552 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.552 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.552 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.552 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-25T09:16:11.553 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.553 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.554 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2023-08-25T09:16:11.554 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.555 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.555 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.555 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2023-08-25T09:16:11.555 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2023-08-25T09:16:11.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:11.556 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.556 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.556 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2023-08-25T09:16:11.556 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.557 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.557 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.557 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.557 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.558 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.558 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2023-08-25T09:16:11.558 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2023-08-25T09:16:11.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.559 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.559 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.559 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.560 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.560 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.560 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.560 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.561 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:11.561 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:11.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.561 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.562 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:11.562 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.562 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.562 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.562 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.563 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.563 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.563 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.564 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.564 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.564 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.565 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.565 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.565 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.565 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.565 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.566 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.566 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.566 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.566 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.566 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.567 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.567 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.567 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.568 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.568 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.568 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.569 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.569 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.569 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.569 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.569 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.570 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.570 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.570 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.571 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.571 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.571 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.571 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.572 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.572 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.572 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.572 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.572 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.573 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.573 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.573 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.573 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.573 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.574 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.574 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.574 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.575 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.575 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.575 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.575 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.576 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.576 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.576 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.576 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.576 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.577 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.577 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2023-08-25T09:16:11.577 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2023-08-25T09:16:11.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.578 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.578 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.578 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.578 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.579 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.579 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.579 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.579 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.579 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.580 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2023-08-25T09:16:11.580 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2023-08-25T09:16:11.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.580 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.581 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2023-08-25T09:16:11.581 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.581 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.581 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.581 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.582 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.582 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.582 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2023-08-25T09:16:11.583 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2023-08-25T09:16:11.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.583 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.583 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.583 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-25T09:16:11.584 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.584 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.584 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.584 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.584 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.585 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.586 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:11.586 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.586 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.586 INFO:teuthology.orchestra.run.smithi099.stdout: "modules": [ 2023-08-25T09:16:11.586 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2023-08-25T09:16:11.587 INFO:teuthology.orchestra.run.smithi099.stdout: "dashboard", 2023-08-25T09:16:11.587 INFO:teuthology.orchestra.run.smithi099.stdout: "iostat", 2023-08-25T09:16:11.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nfs", 2023-08-25T09:16:11.587 INFO:teuthology.orchestra.run.smithi099.stdout: "restful", 2023-08-25T09:16:11.587 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_schedule" 2023-08-25T09:16:11.588 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.588 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2023-08-25T09:16:11.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.588 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2023-08-25T09:16:11.588 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.589 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.589 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.589 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:11.589 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:11.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.590 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.590 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:11.590 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.591 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.591 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.591 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2023-08-25T09:16:11.591 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.592 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.592 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.592 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.593 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.593 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.593 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.593 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.594 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.594 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.594 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.594 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.594 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.595 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.595 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.595 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.595 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.595 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.596 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.596 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.596 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.596 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.597 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.597 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.597 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.598 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.598 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.598 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.598 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.598 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.599 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.599 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.599 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.600 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.600 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.600 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.600 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.601 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.601 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.601 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.601 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.601 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.602 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.602 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.602 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.602 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.602 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.603 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.603 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.603 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.604 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.604 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.604 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.604 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.605 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.605 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.605 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.605 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.606 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2023-08-25T09:16:11.606 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2023-08-25T09:16:11.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.606 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.607 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.607 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.607 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.607 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.607 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.607 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2023-08-25T09:16:11.608 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.608 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.608 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.609 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2023-08-25T09:16:11.609 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2023-08-25T09:16:11.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.610 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.610 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2023-08-25T09:16:11.610 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.610 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.611 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.611 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2023-08-25T09:16:11.611 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.611 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.612 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2023-08-25T09:16:11.612 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2023-08-25T09:16:11.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.612 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.613 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.613 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.613 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.613 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.614 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2023-08-25T09:16:11.614 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.614 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.615 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2023-08-25T09:16:11.615 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2023-08-25T09:16:11.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.615 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.616 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.616 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.616 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.616 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.616 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2023-08-25T09:16:11.617 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.617 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.617 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2023-08-25T09:16:11.618 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2023-08-25T09:16:11.618 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.618 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.618 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2023-08-25T09:16:11.619 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.619 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.619 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.619 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2023-08-25T09:16:11.619 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.620 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.620 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2023-08-25T09:16:11.620 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2023-08-25T09:16:11.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.621 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.621 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.621 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.621 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.622 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.622 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2023-08-25T09:16:11.622 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.622 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.623 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2023-08-25T09:16:11.623 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2023-08-25T09:16:11.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.623 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.624 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.624 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.624 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.624 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.625 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-25T09:16:11.625 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.625 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.626 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2023-08-25T09:16:11.626 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2023-08-25T09:16:11.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.626 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.626 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.627 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.627 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.627 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.627 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.627 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2023-08-25T09:16:11.627 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.628 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.628 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.629 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2023-08-25T09:16:11.629 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.629 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.629 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.630 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2023-08-25T09:16:11.630 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2023-08-25T09:16:11.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.630 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.631 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.631 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.631 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.631 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.631 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2023-08-25T09:16:11.632 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.632 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.632 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2023-08-25T09:16:11.633 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2023-08-25T09:16:11.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.633 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.633 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2023-08-25T09:16:11.633 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.634 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.634 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.634 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2023-08-25T09:16:11.634 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T09:16:11.634 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.635 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.635 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2023-08-25T09:16:11.635 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2023-08-25T09:16:11.635 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:11.636 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.636 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:11.636 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2023-08-25T09:16:11.636 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2023-08-25T09:16:11.637 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.637 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-25T09:16:11.637 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T09:16:11.637 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.637 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.638 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2023-08-25T09:16:11.638 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2023-08-25T09:16:11.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:11.638 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.639 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.639 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2023-08-25T09:16:11.639 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:11.639 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2023-08-25T09:16:11.639 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.640 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-25T09:16:11.640 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.640 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.641 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2023-08-25T09:16:11.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2023-08-25T09:16:11.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.641 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.641 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2023-08-25T09:16:11.642 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.642 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.642 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.642 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-25T09:16:11.642 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-25T09:16:11.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.643 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.643 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2023-08-25T09:16:11.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2023-08-25T09:16:11.644 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:11.644 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.644 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.644 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2023-08-25T09:16:11.644 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2023-08-25T09:16:11.645 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2023-08-25T09:16:11.645 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.645 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2023-08-25T09:16:11.645 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-25T09:16:11.645 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.646 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.646 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2023-08-25T09:16:11.646 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2023-08-25T09:16:11.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.647 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.647 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2023-08-25T09:16:11.647 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.647 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.648 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.648 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2023-08-25T09:16:11.648 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T09:16:11.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.648 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.649 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2023-08-25T09:16:11.649 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2023-08-25T09:16:11.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:11.650 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.650 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.651 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:11.651 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2023-08-25T09:16:11.651 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2023-08-25T09:16:11.652 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.652 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-25T09:16:11.652 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T09:16:11.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.653 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.653 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.653 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.654 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.654 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.654 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.655 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.655 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.655 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.655 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.655 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.656 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.656 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.656 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.656 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.656 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.657 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.657 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.657 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.657 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.658 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.658 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.658 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.659 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.659 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.659 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.659 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.659 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.660 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.660 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.660 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.660 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.661 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.661 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.661 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.661 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.661 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.662 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.662 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.662 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.662 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.662 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.663 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.663 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.663 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.663 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.663 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.664 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.664 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.664 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.665 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.665 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.665 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.665 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.666 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.666 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.666 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.666 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.667 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2023-08-25T09:16:11.667 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2023-08-25T09:16:11.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:11.667 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.668 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:11.668 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.668 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.668 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.668 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-25T09:16:11.669 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.669 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.669 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2023-08-25T09:16:11.670 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2023-08-25T09:16:11.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.670 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.670 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2023-08-25T09:16:11.671 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.671 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.671 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.671 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2023-08-25T09:16:11.671 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2023-08-25T09:16:11.671 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap" 2023-08-25T09:16:11.672 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.672 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2023-08-25T09:16:11.672 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.672 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.673 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2023-08-25T09:16:11.673 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2023-08-25T09:16:11.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.673 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.674 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.674 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.674 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.674 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.675 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-25T09:16:11.675 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.675 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.675 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:11.676 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:11.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.676 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.676 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:11.677 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.677 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.677 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.677 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-25T09:16:11.677 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.678 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.678 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2023-08-25T09:16:11.678 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2023-08-25T09:16:11.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.679 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.679 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:11.679 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:11.679 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.680 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.680 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-25T09:16:11.680 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-25T09:16:11.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.681 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.681 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2023-08-25T09:16:11.681 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2023-08-25T09:16:11.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:11.681 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.682 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.682 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:11.682 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.682 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.682 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.683 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-25T09:16:11.683 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.683 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.684 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2023-08-25T09:16:11.684 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.685 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.685 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.685 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2023-08-25T09:16:11.685 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2023-08-25T09:16:11.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:11.686 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.686 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.686 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2023-08-25T09:16:11.686 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.686 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.686 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.687 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-25T09:16:11.687 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.687 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.688 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2023-08-25T09:16:11.688 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2023-08-25T09:16:11.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.688 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.689 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2023-08-25T09:16:11.689 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.689 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.689 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.689 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-25T09:16:11.690 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.690 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.690 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2023-08-25T09:16:11.690 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2023-08-25T09:16:11.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.691 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.691 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2023-08-25T09:16:11.691 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.692 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.692 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.692 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-25T09:16:11.692 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.693 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.693 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2023-08-25T09:16:11.693 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2023-08-25T09:16:11.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.694 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.694 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.694 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.694 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.694 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.695 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-25T09:16:11.695 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-25T09:16:11.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.695 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.696 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2023-08-25T09:16:11.696 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2023-08-25T09:16:11.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.696 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.697 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:11.697 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.697 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.697 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.697 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-25T09:16:11.698 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.698 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.698 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.698 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2023-08-25T09:16:11.699 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2023-08-25T09:16:11.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:11.699 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.699 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2023-08-25T09:16:11.699 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.700 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.700 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.700 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-25T09:16:11.700 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.701 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.701 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2023-08-25T09:16:11.701 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2023-08-25T09:16:11.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.702 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.702 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.702 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.702 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.702 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.702 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.703 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-25T09:16:11.703 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.703 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.704 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2023-08-25T09:16:11.704 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2023-08-25T09:16:11.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.704 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.705 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.705 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.705 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.705 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.705 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-25T09:16:11.706 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.706 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.706 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2023-08-25T09:16:11.706 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2023-08-25T09:16:11.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.707 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.707 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.707 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.707 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.708 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.708 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.708 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-25T09:16:11.708 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.709 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.709 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2023-08-25T09:16:11.709 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2023-08-25T09:16:11.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.709 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.710 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-25T09:16:11.710 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.710 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.710 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.711 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:11.711 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.711 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.712 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2023-08-25T09:16:11.712 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2023-08-25T09:16:11.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.712 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.713 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-25T09:16:11.713 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.713 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.713 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.713 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2023-08-25T09:16:11.714 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.714 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.714 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2023-08-25T09:16:11.714 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2023-08-25T09:16:11.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.715 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.715 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-25T09:16:11.715 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.716 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.716 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.716 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2023-08-25T09:16:11.716 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.717 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.717 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2023-08-25T09:16:11.717 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2023-08-25T09:16:11.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.718 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.718 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-25T09:16:11.718 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.718 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.718 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.719 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:11.719 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.719 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.720 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2023-08-25T09:16:11.720 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2023-08-25T09:16:11.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.720 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.720 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.721 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-25T09:16:11.721 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.721 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.721 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.721 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2023-08-25T09:16:11.722 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.722 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.722 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2023-08-25T09:16:11.723 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2023-08-25T09:16:11.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.723 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.723 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-25T09:16:11.724 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.724 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.724 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.724 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2023-08-25T09:16:11.724 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.725 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.725 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.725 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2023-08-25T09:16:11.725 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2023-08-25T09:16:11.725 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.726 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.726 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-25T09:16:11.726 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.726 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.727 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.727 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2023-08-25T09:16:11.727 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.727 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.728 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2023-08-25T09:16:11.728 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2023-08-25T09:16:11.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.728 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.729 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.729 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-25T09:16:11.729 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.729 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.729 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.730 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2023-08-25T09:16:11.730 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.730 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.730 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.731 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2023-08-25T09:16:11.731 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2023-08-25T09:16:11.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.731 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.732 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-25T09:16:11.732 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.732 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.732 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.732 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2023-08-25T09:16:11.732 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.733 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.733 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2023-08-25T09:16:11.733 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2023-08-25T09:16:11.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.734 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.734 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.734 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-25T09:16:11.734 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.735 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.735 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.735 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2023-08-25T09:16:11.735 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.735 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.736 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.736 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2023-08-25T09:16:11.736 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2023-08-25T09:16:11.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.737 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.737 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-25T09:16:11.737 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.737 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.737 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.738 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:11.738 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.738 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.739 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2023-08-25T09:16:11.739 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2023-08-25T09:16:11.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.739 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.739 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-08-25T09:16:11.740 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.740 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.740 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.740 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2023-08-25T09:16:11.740 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.741 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.741 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2023-08-25T09:16:11.741 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2023-08-25T09:16:11.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.742 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.742 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-25T09:16:11.742 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.743 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.743 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.743 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2023-08-25T09:16:11.743 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.744 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.744 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2023-08-25T09:16:11.744 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2023-08-25T09:16:11.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.745 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.745 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-25T09:16:11.745 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.745 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.745 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.746 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2023-08-25T09:16:11.746 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.746 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.747 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2023-08-25T09:16:11.747 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2023-08-25T09:16:11.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.747 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.748 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-25T09:16:11.748 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.748 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.749 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.749 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2023-08-25T09:16:11.749 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.749 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.750 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2023-08-25T09:16:11.750 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2023-08-25T09:16:11.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.750 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.751 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.751 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.751 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.751 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.752 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2023-08-25T09:16:11.752 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.752 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.752 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.753 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2023-08-25T09:16:11.753 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2023-08-25T09:16:11.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.753 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-25T09:16:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2023-08-25T09:16:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2023-08-25T09:16:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2023-08-25T09:16:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-25T09:16:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2023-08-25T09:16:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2023-08-25T09:16:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2023-08-25T09:16:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-25T09:16:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2023-08-25T09:16:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2023-08-25T09:16:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2023-08-25T09:16:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2023-08-25T09:16:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2023-08-25T09:16:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2023-08-25T09:16:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-25T09:16:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2023-08-25T09:16:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2023-08-25T09:16:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2023-08-25T09:16:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2023-08-25T09:16:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2023-08-25T09:16:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2023-08-25T09:16:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2023-08-25T09:16:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2023-08-25T09:16:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-25T09:16:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-25T09:16:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-25T09:16:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-25T09:16:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-25T09:16:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-25T09:16:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-25T09:16:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2023-08-25T09:16:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-25T09:16:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2023-08-25T09:16:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2023-08-25T09:16:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2023-08-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2023-08-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2023-08-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2023-08-25T09:16:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2023-08-25T09:16:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2023-08-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2023-08-25T09:16:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2023-08-25T09:16:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2023-08-25T09:16:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2023-08-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2023-08-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2023-08-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2023-08-25T09:16:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2023-08-25T09:16:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2023-08-25T09:16:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2023-08-25T09:16:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-25T09:16:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2023-08-25T09:16:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2023-08-25T09:16:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-25T09:16:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2023-08-25T09:16:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2023-08-25T09:16:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-25T09:16:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2023-08-25T09:16:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2023-08-25T09:16:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-25T09:16:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2023-08-25T09:16:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2023-08-25T09:16:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-25T09:16:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2023-08-25T09:16:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2023-08-25T09:16:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2023-08-25T09:16:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2023-08-25T09:16:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2023-08-25T09:16:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2023-08-25T09:16:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-25T09:16:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2023-08-25T09:16:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2023-08-25T09:16:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-25T09:16:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2023-08-25T09:16:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2023-08-25T09:16:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-25T09:16:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2023-08-25T09:16:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2023-08-25T09:16:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2023-08-25T09:16:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2023-08-25T09:16:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2023-08-25T09:16:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-25T09:16:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2023-08-25T09:16:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2023-08-25T09:16:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-25T09:16:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2023-08-25T09:16:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.849 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.849 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.849 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.849 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.849 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.849 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.850 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.850 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.850 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.851 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.851 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.851 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.851 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.852 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.852 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.852 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.852 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.852 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.853 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.853 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2023-08-25T09:16:11.853 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2023-08-25T09:16:11.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.854 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.854 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2023-08-25T09:16:11.854 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.854 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.855 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.855 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-25T09:16:11.855 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.855 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.856 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2023-08-25T09:16:11.856 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2023-08-25T09:16:11.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:11.856 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.857 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2023-08-25T09:16:11.857 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.857 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.857 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.857 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-25T09:16:11.858 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.858 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:11.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:11.859 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2023-08-25T09:16:11.859 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:11.859 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:11.859 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:11.860 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-25T09:16:11.860 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-25T09:16:11.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.860 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.861 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:11.861 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.861 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.861 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.861 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.862 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.862 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.862 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2023-08-25T09:16:11.863 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-25T09:16:11.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.863 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.863 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.863 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.864 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.864 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.864 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.864 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.864 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.865 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.865 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-25T09:16:11.865 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-25T09:16:11.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.866 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.866 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.866 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.866 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.866 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.867 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.867 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.867 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.868 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2023-08-25T09:16:11.868 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2023-08-25T09:16:11.868 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.868 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.868 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.869 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.869 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.869 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.869 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.869 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.869 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.870 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.870 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-25T09:16:11.870 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-25T09:16:11.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.871 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.871 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.871 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.872 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.872 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.872 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.872 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.873 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.873 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2023-08-25T09:16:11.873 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-25T09:16:11.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.873 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.874 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.874 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.874 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.874 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.875 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.875 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.875 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.875 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-25T09:16:11.876 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-25T09:16:11.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.876 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.877 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.877 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.877 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.877 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.877 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.877 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.878 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.878 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-25T09:16:11.878 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-25T09:16:11.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.879 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.879 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.879 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.880 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.880 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.880 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.880 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.880 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.881 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-25T09:16:11.881 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-25T09:16:11.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.881 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.882 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.882 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.882 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.882 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.883 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.883 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.883 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.883 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.884 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-25T09:16:11.884 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-25T09:16:11.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.884 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.885 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.885 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.885 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.885 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.885 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.886 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.886 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.886 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-25T09:16:11.886 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-25T09:16:11.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.887 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.887 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.887 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.888 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.888 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.888 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.888 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.888 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.889 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.889 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-25T09:16:11.889 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-25T09:16:11.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.890 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.890 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.890 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.890 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.890 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.891 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.891 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.891 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.891 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.892 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-25T09:16:11.892 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-25T09:16:11.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.892 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.892 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.893 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.893 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.893 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.893 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.893 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.894 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.894 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.894 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-25T09:16:11.894 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-25T09:16:11.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.895 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.895 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.895 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.895 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.896 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.896 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.896 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.896 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.896 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.897 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.897 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2023-08-25T09:16:11.897 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-25T09:16:11.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.897 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.898 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.898 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:11.898 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.898 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.898 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.899 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.899 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.899 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.899 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.900 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-25T09:16:11.900 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-25T09:16:11.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.900 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.901 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.901 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.901 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.901 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.901 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.902 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.902 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.902 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2023-08-25T09:16:11.902 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2023-08-25T09:16:11.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.903 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.903 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.903 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.904 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.904 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.904 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.904 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.905 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.905 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2023-08-25T09:16:11.905 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-25T09:16:11.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.905 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.906 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:11.906 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.906 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.906 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.907 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.907 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.907 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.907 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.908 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-25T09:16:11.908 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-25T09:16:11.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.908 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.909 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.909 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.909 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.909 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.909 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.909 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.910 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.910 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.910 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-25T09:16:11.910 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-25T09:16:11.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.911 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.911 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.911 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.912 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.912 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.912 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.912 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.912 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.913 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.913 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-25T09:16:11.913 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-25T09:16:11.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.913 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.914 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.914 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.914 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.914 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.915 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.915 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.915 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.916 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-25T09:16:11.916 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-25T09:16:11.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.916 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.916 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.917 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.917 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.917 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.917 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.917 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.918 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.918 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.918 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2023-08-25T09:16:11.918 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-25T09:16:11.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.919 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.919 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.919 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.919 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.919 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.920 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.920 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.920 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.920 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.920 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.921 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-25T09:16:11.921 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-25T09:16:11.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.921 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.922 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.922 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.922 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.922 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.923 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.923 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.923 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.923 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-25T09:16:11.924 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-25T09:16:11.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.924 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.924 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.924 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.925 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.925 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.925 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.925 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.925 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.926 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.926 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-25T09:16:11.926 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-25T09:16:11.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.927 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.927 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.927 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.927 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.927 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.928 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.928 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.928 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.928 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.929 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-25T09:16:11.929 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-25T09:16:11.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.929 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.930 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.930 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.930 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.930 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.931 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.931 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.931 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.931 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.931 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-25T09:16:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-25T09:16:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-25T09:16:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-25T09:16:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-25T09:16:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-25T09:16:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-25T09:16:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-25T09:16:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2023-08-25T09:16:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-25T09:16:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-25T09:16:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-25T09:16:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-25T09:16:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-25T09:16:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-25T09:16:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-25T09:16:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-25T09:16:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2023-08-25T09:16:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-25T09:16:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-25T09:16:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2023-08-25T09:16:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2023-08-25T09:16:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-25T09:16:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-25T09:16:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-25T09:16:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2023-08-25T09:16:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2023-08-25T09:16:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-25T09:16:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2023-08-25T09:16:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-25T09:16:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-25T09:16:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-25T09:16:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-25T09:16:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-25T09:16:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-25T09:16:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-25T09:16:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2023-08-25T09:16:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2023-08-25T09:16:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2023-08-25T09:16:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2023-08-25T09:16:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2023-08-25T09:16:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2023-08-25T09:16:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2023-08-25T09:16:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2023-08-25T09:16:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2023-08-25T09:16:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2023-08-25T09:16:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2023-08-25T09:16:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2023-08-25T09:16:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2023-08-25T09:16:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2023-08-25T09:16:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2023-08-25T09:16:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2023-08-25T09:16:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2023-08-25T09:16:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2023-08-25T09:16:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2023-08-25T09:16:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2023-08-25T09:16:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2023-08-25T09:16:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2023-08-25T09:16:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2023-08-25T09:16:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2023-08-25T09:16:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2023-08-25T09:16:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2023-08-25T09:16:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2023-08-25T09:16:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2023-08-25T09:16:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2023-08-25T09:16:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2023-08-25T09:16:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2023-08-25T09:16:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2023-08-25T09:16:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2023-08-25T09:16:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2023-08-25T09:16:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2023-08-25T09:16:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2023-08-25T09:16:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2023-08-25T09:16:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2023-08-25T09:16:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2023-08-25T09:16:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2023-08-25T09:16:12.043 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2023-08-25T09:16:12.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.043 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.043 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2023-08-25T09:16:12.044 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.044 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.044 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.044 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-25T09:16:12.044 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.045 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.045 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2023-08-25T09:16:12.045 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2023-08-25T09:16:12.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.046 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.046 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2023-08-25T09:16:12.046 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.046 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.047 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.047 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-25T09:16:12.047 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.047 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.048 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2023-08-25T09:16:12.048 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2023-08-25T09:16:12.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.048 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.049 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2023-08-25T09:16:12.049 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.049 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.049 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.050 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2023-08-25T09:16:12.050 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.050 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.050 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2023-08-25T09:16:12.051 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2023-08-25T09:16:12.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.051 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.051 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2023-08-25T09:16:12.052 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.052 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.052 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.052 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-25T09:16:12.052 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.053 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.053 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2023-08-25T09:16:12.053 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2023-08-25T09:16:12.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.054 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.054 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.054 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.054 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.057 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:12.057 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.057 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.057 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.058 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2023-08-25T09:16:12.058 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.058 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.059 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2023-08-25T09:16:12.059 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2023-08-25T09:16:12.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.059 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.060 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2023-08-25T09:16:12.060 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.060 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.060 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.060 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-25T09:16:12.060 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.061 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.061 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.062 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2023-08-25T09:16:12.062 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.062 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.062 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.063 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.063 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.063 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.064 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.064 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.064 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.064 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.064 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.065 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.065 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.065 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.065 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.065 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.065 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.066 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.066 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.066 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.067 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.067 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.067 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.068 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.068 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.068 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.068 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.068 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.069 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.069 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.069 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.069 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.070 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.070 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.070 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.071 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.071 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.071 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.071 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.071 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.072 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.072 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.072 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.072 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.072 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.073 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.073 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.073 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.073 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.074 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.074 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.074 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.075 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.075 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.075 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.075 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.076 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.076 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2023-08-25T09:16:12.076 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2023-08-25T09:16:12.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.077 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.077 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2023-08-25T09:16:12.077 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.077 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.077 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.078 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.078 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.078 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.079 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2023-08-25T09:16:12.079 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2023-08-25T09:16:12.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.079 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.080 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2023-08-25T09:16:12.080 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.080 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.080 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.080 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.080 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.081 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.081 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:12.081 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:12.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.082 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.082 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2023-08-25T09:16:12.082 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.083 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.083 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.083 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.083 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.083 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.084 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2023-08-25T09:16:12.085 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2023-08-25T09:16:12.085 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2023-08-25T09:16:12.085 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.085 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2023-08-25T09:16:12.085 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2023-08-25T09:16:12.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.086 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.086 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2023-08-25T09:16:12.086 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.087 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.087 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.087 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-25T09:16:12.087 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.087 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.088 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2023-08-25T09:16:12.088 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2023-08-25T09:16:12.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.088 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.089 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.089 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2023-08-25T09:16:12.089 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.089 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.089 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.090 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-25T09:16:12.090 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.090 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.090 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.091 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2023-08-25T09:16:12.091 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2023-08-25T09:16:12.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.091 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.091 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.092 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.092 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.092 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.092 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2023-08-25T09:16:12.092 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.093 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.093 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.093 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:12.093 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:12.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.094 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.094 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.094 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2023-08-25T09:16:12.094 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2023-08-25T09:16:12.095 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.095 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.095 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-25T09:16:12.095 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.095 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.095 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.096 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.096 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.096 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.097 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.097 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.097 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.097 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.098 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.098 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.098 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.098 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.098 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.099 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.099 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.099 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.099 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.100 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.100 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.100 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.101 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.101 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.101 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.101 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.102 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.102 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.102 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.102 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.102 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.103 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.103 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.103 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.104 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.104 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.104 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.104 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.104 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.105 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.105 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.105 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.105 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.105 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.106 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.106 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.106 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.106 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.107 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.107 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.107 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.107 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.108 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.108 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.108 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.108 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.108 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.109 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.109 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.109 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2023-08-25T09:16:12.110 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2023-08-25T09:16:12.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.110 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.110 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.111 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.111 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.111 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.111 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2023-08-25T09:16:12.111 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.112 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.112 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2023-08-25T09:16:12.112 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2023-08-25T09:16:12.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.113 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.113 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2023-08-25T09:16:12.113 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.113 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.114 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.114 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2023-08-25T09:16:12.114 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.114 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.115 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2023-08-25T09:16:12.115 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2023-08-25T09:16:12.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.115 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.115 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.116 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2023-08-25T09:16:12.116 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.116 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.116 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.116 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T09:16:12.117 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.117 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.117 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.117 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2023-08-25T09:16:12.118 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2023-08-25T09:16:12.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.118 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.118 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:12.119 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:12.119 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2023-08-25T09:16:12.119 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.119 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-25T09:16:12.119 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.119 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.120 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.120 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2023-08-25T09:16:12.120 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2023-08-25T09:16:12.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.121 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.121 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.121 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.121 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.122 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.122 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2023-08-25T09:16:12.122 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.122 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.122 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.123 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2023-08-25T09:16:12.123 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2023-08-25T09:16:12.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.123 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.124 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2023-08-25T09:16:12.124 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.124 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.124 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.124 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T09:16:12.125 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.125 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.125 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.126 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2023-08-25T09:16:12.126 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.126 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.126 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.127 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.127 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.127 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.128 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.128 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.128 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.128 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.128 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.129 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.129 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.129 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.129 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.129 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.129 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.130 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.130 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.130 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.130 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.131 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.131 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.131 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.131 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.132 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.132 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.132 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.132 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.132 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.133 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.133 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.133 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.133 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.134 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.134 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.134 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.134 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.135 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.135 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.135 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.135 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.135 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.135 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.136 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.136 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.136 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.136 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.136 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.137 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.137 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.137 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.138 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.138 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.138 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.138 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.139 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.139 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.139 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.139 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.139 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.140 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2023-08-25T09:16:12.141 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.141 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.141 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.141 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.141 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.142 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.142 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.142 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.142 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.143 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.143 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.143 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.143 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.144 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.144 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.144 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.144 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.144 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.145 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.145 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.145 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.146 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.146 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.146 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.147 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.147 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.147 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.147 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.147 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.147 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.148 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.148 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.148 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.149 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.149 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.149 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.149 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.150 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.150 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.150 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.150 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.150 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.150 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.151 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.151 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.151 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.151 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.151 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.152 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.152 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.152 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.152 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.153 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.153 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.153 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.154 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.154 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.154 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.154 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.154 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.154 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.155 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2023-08-25T09:16:12.156 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.156 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.156 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.156 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2023-08-25T09:16:12.156 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2023-08-25T09:16:12.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.157 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.157 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2023-08-25T09:16:12.157 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.157 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.158 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.158 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-25T09:16:12.158 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.158 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2023-08-25T09:16:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2023-08-25T09:16:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2023-08-25T09:16:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-25T09:16:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2023-08-25T09:16:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2023-08-25T09:16:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2023-08-25T09:16:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2023-08-25T09:16:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2023-08-25T09:16:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-25T09:16:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2023-08-25T09:16:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2023-08-25T09:16:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2023-08-25T09:16:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-25T09:16:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2023-08-25T09:16:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2023-08-25T09:16:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2023-08-25T09:16:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2023-08-25T09:16:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2023-08-25T09:16:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2023-08-25T09:16:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2023-08-25T09:16:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2023-08-25T09:16:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2023-08-25T09:16:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2023-08-25T09:16:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2023-08-25T09:16:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2023-08-25T09:16:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2023-08-25T09:16:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2023-08-25T09:16:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-25T09:16:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-25T09:16:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2023-08-25T09:16:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2023-08-25T09:16:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2023-08-25T09:16:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2023-08-25T09:16:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2023-08-25T09:16:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2023-08-25T09:16:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.254 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.254 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-25T09:16:12.254 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.254 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.255 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.255 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.255 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.256 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.256 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.256 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.256 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.257 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.257 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.257 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.257 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.257 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.258 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.258 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.258 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.258 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.259 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.259 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.260 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.260 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.260 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.260 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.261 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.261 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.261 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.262 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.262 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.262 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.262 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.264 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.264 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.264 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.264 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.264 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.265 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.265 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.265 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.265 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.266 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.266 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.266 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.267 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.267 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.267 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.267 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.267 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.268 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.268 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2023-08-25T09:16:12.268 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2023-08-25T09:16:12.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.269 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.269 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.269 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.269 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.270 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.270 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2023-08-25T09:16:12.270 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2023-08-25T09:16:12.270 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2023-08-25T09:16:12.270 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.271 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2023-08-25T09:16:12.271 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.271 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.272 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2023-08-25T09:16:12.272 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.273 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.273 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.273 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.273 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.274 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.274 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.275 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.275 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.275 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.275 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.275 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.276 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.276 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.276 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.276 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.277 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.277 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.277 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.277 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.278 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.278 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.278 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.278 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.279 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.279 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.279 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.280 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.280 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.280 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.282 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.282 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.282 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.282 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.282 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.282 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.286 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.286 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2023-08-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.288 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.288 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.290 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.290 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.290 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.290 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.290 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.293 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.293 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.294 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.294 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.294 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.296 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.296 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.296 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.296 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.297 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.297 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.297 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.297 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.297 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.298 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.298 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.298 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.299 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.299 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.299 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.300 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.300 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.300 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.301 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.302 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2023-08-25T09:16:12.302 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.302 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.302 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.302 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.303 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.303 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.304 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.304 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.304 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.304 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.305 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.305 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.305 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.305 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.305 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.306 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.306 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.306 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.306 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.307 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.307 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.308 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.308 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.308 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.308 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.308 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.309 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.309 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.309 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.310 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.310 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.310 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.310 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.311 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.311 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.311 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.311 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.311 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.312 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.312 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.312 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.312 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.312 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.313 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.313 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.313 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.314 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.314 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.314 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.314 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.315 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.315 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.315 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.315 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.316 INFO:teuthology.orchestra.run.smithi099.stdout: "noautoscale": { 2023-08-25T09:16:12.316 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "noautoscale", 2023-08-25T09:16:12.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.316 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.317 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.317 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.317 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.317 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.317 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "global autoscale flag", 2023-08-25T09:16:12.318 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-25T09:16:12.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.318 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.318 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:12.319 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:12.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.319 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.319 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:12.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.320 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.320 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.321 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.321 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2023-08-25T09:16:12.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2023-08-25T09:16:12.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:12.322 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.322 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2023-08-25T09:16:12.322 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2023-08-25T09:16:12.322 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.323 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.323 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2023-08-25T09:16:12.323 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-25T09:16:12.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.323 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.324 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2023-08-25T09:16:12.325 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.325 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.325 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.325 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2023-08-25T09:16:12.325 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2023-08-25T09:16:12.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.326 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.326 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.326 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.326 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.327 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-25T09:16:12.327 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.327 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.328 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2023-08-25T09:16:12.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2023-08-25T09:16:12.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.328 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.329 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.329 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.329 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.329 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.330 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.330 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.330 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.330 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.331 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.331 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.331 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.332 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.332 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.332 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.332 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.332 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.333 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.333 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.333 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.333 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.333 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.333 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.334 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.334 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.334 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.335 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.335 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.335 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.336 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.336 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.336 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.336 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.336 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.337 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.337 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.337 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.338 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.338 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.338 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.339 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.339 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.339 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.339 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.339 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.340 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.340 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.340 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.340 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.340 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.341 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.341 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.341 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.342 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.342 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.342 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.343 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.343 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.343 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2023-08-25T09:16:12.344 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2023-08-25T09:16:12.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.344 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.345 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2023-08-25T09:16:12.345 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.345 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.345 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.345 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2023-08-25T09:16:12.346 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.346 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.346 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2023-08-25T09:16:12.347 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2023-08-25T09:16:12.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.347 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.347 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2023-08-25T09:16:12.348 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.348 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.348 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.348 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2023-08-25T09:16:12.348 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.349 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.350 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2023-08-25T09:16:12.350 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.350 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.350 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.350 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2023-08-25T09:16:12.351 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2023-08-25T09:16:12.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.351 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.351 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.352 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.352 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.352 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.352 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.352 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.353 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.353 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2023-08-25T09:16:12.353 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2023-08-25T09:16:12.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.354 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.354 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.354 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.354 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.354 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.355 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.355 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-25T09:16:12.355 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-25T09:16:12.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.355 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.356 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.356 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.356 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.357 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.357 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.357 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.357 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.358 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.358 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.358 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.358 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.358 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.359 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.359 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.359 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.359 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.360 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.360 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.360 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.361 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.361 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.361 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.361 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.361 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.362 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.362 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.362 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.363 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.363 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.363 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.364 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.364 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.364 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.364 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.364 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.365 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.365 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.365 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.365 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.365 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.365 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.366 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.366 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.366 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.367 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.367 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.367 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.367 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.368 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.368 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.368 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.368 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.369 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.369 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2023-08-25T09:16:12.369 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2023-08-25T09:16:12.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.370 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.370 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.370 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.370 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.371 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.371 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.371 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.371 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.372 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-25T09:16:12.372 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-25T09:16:12.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.372 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.373 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2023-08-25T09:16:12.373 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.373 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.373 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.374 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.374 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.374 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.374 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2023-08-25T09:16:12.375 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2023-08-25T09:16:12.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:12.375 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.375 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.375 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2023-08-25T09:16:12.376 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.376 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.376 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.376 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.376 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.377 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.377 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:12.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:12.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.378 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.378 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2023-08-25T09:16:12.378 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.379 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.379 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.379 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-25T09:16:12.379 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.379 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.380 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:12.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:12.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.380 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.381 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2023-08-25T09:16:12.381 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.381 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.381 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.382 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-25T09:16:12.382 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.382 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.383 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2023-08-25T09:16:12.383 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2023-08-25T09:16:12.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.383 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.383 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2023-08-25T09:16:12.384 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.384 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.384 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.384 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.384 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.385 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.385 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2023-08-25T09:16:12.385 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2023-08-25T09:16:12.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.386 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.386 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2023-08-25T09:16:12.386 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.387 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.387 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.387 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2023-08-25T09:16:12.387 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2023-08-25T09:16:12.387 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.387 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.388 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.388 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.388 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2023-08-25T09:16:12.389 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2023-08-25T09:16:12.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.389 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.389 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2023-08-25T09:16:12.390 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2023-08-25T09:16:12.390 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2023-08-25T09:16:12.390 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.390 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.390 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.391 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.392 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2023-08-25T09:16:12.392 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.392 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.392 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.392 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.393 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.393 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.393 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.394 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.394 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.394 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.394 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.394 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.395 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.395 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.395 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.395 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.395 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.395 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.396 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.396 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.396 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.397 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.397 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.397 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.398 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.398 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.398 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.398 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.398 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.399 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.399 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.400 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.400 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.400 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.400 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.401 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.401 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.401 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.401 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.401 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.403 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.403 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.403 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.404 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.404 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.404 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.405 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.405 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.405 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.405 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.406 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2023-08-25T09:16:12.406 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2023-08-25T09:16:12.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.406 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.407 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2023-08-25T09:16:12.407 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.407 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.407 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.407 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.408 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.408 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.408 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2023-08-25T09:16:12.409 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2023-08-25T09:16:12.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.409 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.409 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.409 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.410 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.410 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.410 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.410 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.410 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.411 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.411 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2023-08-25T09:16:12.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2023-08-25T09:16:12.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.412 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.412 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.413 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.413 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.414 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2023-08-25T09:16:12.414 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.415 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.415 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.415 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2023-08-25T09:16:12.415 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2023-08-25T09:16:12.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.416 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.416 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.416 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.416 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.416 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.417 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.417 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.417 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.418 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2023-08-25T09:16:12.418 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2023-08-25T09:16:12.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.418 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.419 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.419 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.420 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.420 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.420 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.421 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.421 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.421 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.422 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.422 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.422 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.422 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.423 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.423 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.423 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.423 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.423 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.423 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.424 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.424 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.424 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.425 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.425 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.426 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.426 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.426 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.426 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.427 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.427 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.427 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.428 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.428 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.428 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.429 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.429 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.429 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.430 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.430 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.430 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.430 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.430 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.431 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.431 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.431 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.432 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.432 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.432 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.433 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.433 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.433 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.433 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.434 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.434 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.434 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2023-08-25T09:16:12.435 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2023-08-25T09:16:12.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.435 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.435 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.436 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.436 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.436 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.436 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.437 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.437 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2023-08-25T09:16:12.437 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2023-08-25T09:16:12.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.438 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.438 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.438 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.439 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.439 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.439 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.439 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.440 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2023-08-25T09:16:12.441 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.441 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.441 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.441 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.441 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.442 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.442 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.443 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.443 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.443 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.443 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.443 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.443 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.444 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.444 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.444 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.444 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.444 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.445 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.445 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.445 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.446 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.446 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.446 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.447 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.447 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.447 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.447 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.447 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.447 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.448 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.448 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.448 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.449 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.449 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.449 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.449 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.450 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.450 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.450 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.450 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.450 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.451 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.451 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.451 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.451 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.452 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.452 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.453 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.453 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.453 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.453 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.454 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.454 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.454 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.455 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2023-08-25T09:16:12.455 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.456 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.456 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.456 INFO:teuthology.orchestra.run.smithi099.stdout: "drive_group_interval": { 2023-08-25T09:16:12.456 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "drive_group_interval", 2023-08-25T09:16:12.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:12.457 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.457 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300.0", 2023-08-25T09:16:12.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.457 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.457 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.458 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-25T09:16:12.458 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.458 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.459 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.459 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.459 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.460 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.460 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.460 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.460 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.460 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.461 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.461 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.461 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.461 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.461 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.464 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.464 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.464 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.464 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.465 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.465 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.465 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.465 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.466 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.466 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.466 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.467 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.467 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.467 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.467 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.467 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.468 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.468 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.468 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.468 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.468 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.469 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.469 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.469 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.469 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.470 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.471 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.471 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.471 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.472 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.472 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2023-08-25T09:16:12.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2023-08-25T09:16:12.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.473 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.473 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2023-08-25T09:16:12.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.474 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.474 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.474 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-25T09:16:12.474 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.474 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.474 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2023-08-25T09:16:12.476 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.476 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.476 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.476 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.476 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.477 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.477 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.478 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.478 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.478 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.478 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.478 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.478 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.479 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.479 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.479 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.479 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.479 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.480 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.480 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.480 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.481 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.482 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.482 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.482 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.482 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.482 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.483 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.483 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.483 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.484 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.484 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.484 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.485 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.485 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.485 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.485 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.485 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.486 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.486 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.486 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.486 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.487 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.487 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.487 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.487 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.488 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.488 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.488 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.488 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.489 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.489 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.489 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.490 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2023-08-25T09:16:12.490 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2023-08-25T09:16:12.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.490 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.490 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.491 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.491 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.491 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.491 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.492 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2023-08-25T09:16:12.492 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2023-08-25T09:16:12.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.493 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.493 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2023-08-25T09:16:12.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.494 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.494 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.494 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.494 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.494 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.495 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2023-08-25T09:16:12.495 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2023-08-25T09:16:12.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.495 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.496 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.496 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.496 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.496 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.497 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.497 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.497 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.498 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2023-08-25T09:16:12.498 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2023-08-25T09:16:12.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.498 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.498 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.499 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.499 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.499 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.499 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.499 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.500 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.500 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2023-08-25T09:16:12.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2023-08-25T09:16:12.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2023-08-25T09:16:12.501 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.501 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.501 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.502 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.502 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.502 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.502 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.502 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.503 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2023-08-25T09:16:12.503 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2023-08-25T09:16:12.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.503 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.504 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.504 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.504 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.504 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.505 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.505 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.505 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.506 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2023-08-25T09:16:12.506 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2023-08-25T09:16:12.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.506 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.507 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.507 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.507 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.507 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.507 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.507 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.508 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.508 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2023-08-25T09:16:12.508 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2023-08-25T09:16:12.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.509 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.509 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.509 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.510 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.510 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.510 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.510 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.511 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.511 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2023-08-25T09:16:12.511 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2023-08-25T09:16:12.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.512 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.512 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.512 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2023-08-25T09:16:12.512 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2023-08-25T09:16:12.512 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.513 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.513 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.513 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.514 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2023-08-25T09:16:12.514 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2023-08-25T09:16:12.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.514 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.515 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.515 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.515 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.515 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.515 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.515 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.516 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.516 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2023-08-25T09:16:12.516 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2023-08-25T09:16:12.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.517 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.517 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.517 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.518 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.518 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.518 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.518 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.519 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.519 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2023-08-25T09:16:12.519 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2023-08-25T09:16:12.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.519 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.520 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.520 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.520 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.520 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.521 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.521 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.521 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.522 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2023-08-25T09:16:12.522 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.523 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.523 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.523 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2023-08-25T09:16:12.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2023-08-25T09:16:12.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.523 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.524 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.524 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.524 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.524 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.525 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2023-08-25T09:16:12.525 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.525 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.526 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2023-08-25T09:16:12.526 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2023-08-25T09:16:12.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.526 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.527 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.527 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.527 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.527 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2023-08-25T09:16:12.527 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.528 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.528 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.528 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.529 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.529 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.530 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.530 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.530 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.530 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.530 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.531 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.531 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.531 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.531 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.531 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.531 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.532 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.532 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.532 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.533 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.533 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.533 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.534 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.534 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.534 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.534 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.535 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.535 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.535 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.535 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.536 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.536 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.536 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.537 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.537 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.537 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.537 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.537 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.538 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.538 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.538 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.538 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.539 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.539 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.539 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.540 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.540 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.540 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.540 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.540 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.541 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.541 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.541 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.542 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2023-08-25T09:16:12.542 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.543 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.543 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.543 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.543 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.545 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.545 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.545 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.545 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.545 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.545 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.546 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.546 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.546 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.546 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.547 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.547 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.547 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.548 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.548 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.548 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.548 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.549 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.549 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.549 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.549 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.550 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.550 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.550 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.551 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.551 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.551 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.551 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.551 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.551 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.552 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.552 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.552 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.552 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.552 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.553 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.553 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.553 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.553 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.554 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.554 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.554 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.555 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.555 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.555 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.555 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.555 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.556 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.556 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.556 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.557 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2023-08-25T09:16:12.557 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.557 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.558 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.558 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.558 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.558 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.559 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.559 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.559 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.559 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.560 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.560 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.560 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.560 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.560 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.561 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.561 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.561 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.561 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.563 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.563 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.563 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.563 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.563 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.564 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.564 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.564 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.565 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.565 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.565 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.566 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.566 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.566 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.566 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.567 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.567 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.567 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.567 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.567 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.567 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.568 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.568 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.568 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.569 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.569 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.569 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.570 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.570 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.570 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.570 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.570 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.571 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.571 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.572 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2023-08-25T09:16:12.572 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.572 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.573 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.573 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2023-08-25T09:16:12.573 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2023-08-25T09:16:12.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.573 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.574 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-25T09:16:12.574 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.574 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.574 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.575 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.575 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.575 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.576 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:12.576 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:12.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.576 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.577 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2023-08-25T09:16:12.577 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.577 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.577 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.577 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.578 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.578 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.578 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.579 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.579 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.579 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.579 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.580 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.580 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.580 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.580 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.580 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.581 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.581 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.581 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.581 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.581 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.582 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.582 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.582 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.583 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.583 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.583 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.583 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.584 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.584 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.584 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.584 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.584 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.585 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.585 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.585 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.586 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.586 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.586 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.586 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.587 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.587 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.587 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.587 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.587 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.588 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.588 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.588 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.588 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.588 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.589 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.589 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.589 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.589 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.590 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.590 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.590 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.591 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.591 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.591 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.591 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.591 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.592 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2023-08-25T09:16:12.593 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.593 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.593 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.593 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2023-08-25T09:16:12.593 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2023-08-25T09:16:12.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.594 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.594 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.594 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.594 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.595 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.595 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.595 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-25T09:16:12.595 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.596 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.596 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2023-08-25T09:16:12.596 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2023-08-25T09:16:12.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.597 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.597 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.597 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.597 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.598 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.598 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-25T09:16:12.598 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.598 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.598 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.599 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2023-08-25T09:16:12.599 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2023-08-25T09:16:12.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.599 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.600 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2023-08-25T09:16:12.600 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.600 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.600 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.601 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-25T09:16:12.601 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.601 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.601 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.602 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2023-08-25T09:16:12.602 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2023-08-25T09:16:12.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.602 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.602 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.603 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.603 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.603 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.603 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-25T09:16:12.603 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.604 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.604 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.604 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2023-08-25T09:16:12.605 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2023-08-25T09:16:12.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.605 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.605 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.605 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.606 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.606 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.606 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.606 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-25T09:16:12.606 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.607 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.607 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2023-08-25T09:16:12.607 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2023-08-25T09:16:12.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.608 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.608 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.608 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.608 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.609 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.609 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.609 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.609 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.610 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2023-08-25T09:16:12.610 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2023-08-25T09:16:12.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.610 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.611 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.611 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.611 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.611 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.612 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.612 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.612 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.612 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.613 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2023-08-25T09:16:12.613 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2023-08-25T09:16:12.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.613 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.614 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-25T09:16:12.614 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.614 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.614 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.614 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.615 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.615 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.615 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2023-08-25T09:16:12.615 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2023-08-25T09:16:12.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.616 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.616 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.616 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.617 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.617 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.617 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.617 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.617 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.618 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.618 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:12.618 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:12.618 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.619 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.619 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2023-08-25T09:16:12.619 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2023-08-25T09:16:12.619 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.620 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.620 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.620 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.620 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.621 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2023-08-25T09:16:12.621 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2023-08-25T09:16:12.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.621 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.622 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.622 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2023-08-25T09:16:12.622 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.622 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.622 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.623 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.623 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.623 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.623 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.623 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2023-08-25T09:16:12.624 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2023-08-25T09:16:12.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.625 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.625 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.625 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.625 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.625 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.626 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.626 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.626 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.626 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.627 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2023-08-25T09:16:12.627 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2023-08-25T09:16:12.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.627 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.628 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.628 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.628 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.628 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.629 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.629 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.629 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.629 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.630 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.630 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.630 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.631 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.631 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.631 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.631 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.631 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.632 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.632 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.632 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.632 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.632 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.633 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.633 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.633 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.634 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.634 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.634 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.634 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.634 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.635 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.635 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.635 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.635 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.635 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.636 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.636 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.636 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.637 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.637 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.637 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.638 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.638 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.638 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.638 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.638 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.639 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.639 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.639 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.639 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.639 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.640 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.640 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.640 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.641 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.641 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.641 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.642 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.642 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.642 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.642 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.643 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.643 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2023-08-25T09:16:12.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2023-08-25T09:16:12.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.644 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.644 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.644 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.644 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.644 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.645 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.645 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.645 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.645 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.645 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.646 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2023-08-25T09:16:12.646 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2023-08-25T09:16:12.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.646 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.647 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.647 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.647 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.647 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.647 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2023-08-25T09:16:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2023-08-25T09:16:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-25T09:16:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2023-08-25T09:16:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2023-08-25T09:16:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2023-08-25T09:16:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2023-08-25T09:16:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2023-08-25T09:16:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-25T09:16:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2023-08-25T09:16:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2023-08-25T09:16:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2023-08-25T09:16:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-25T09:16:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2023-08-25T09:16:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2023-08-25T09:16:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2023-08-25T09:16:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2023-08-25T09:16:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2023-08-25T09:16:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2023-08-25T09:16:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2023-08-25T09:16:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2023-08-25T09:16:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2023-08-25T09:16:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2023-08-25T09:16:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2023-08-25T09:16:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2023-08-25T09:16:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2023-08-25T09:16:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2023-08-25T09:16:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2023-08-25T09:16:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2023-08-25T09:16:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2023-08-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2023-08-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2023-08-25T09:16:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2023-08-25T09:16:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2023-08-25T09:16:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2023-08-25T09:16:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2023-08-25T09:16:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2023-08-25T09:16:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2023-08-25T09:16:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2023-08-25T09:16:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2023-08-25T09:16:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2023-08-25T09:16:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2023-08-25T09:16:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2023-08-25T09:16:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2023-08-25T09:16:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-08-25T09:16:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2023-08-25T09:16:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2023-08-25T09:16:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2023-08-25T09:16:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2023-08-25T09:16:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2023-08-25T09:16:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2023-08-25T09:16:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2023-08-25T09:16:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2023-08-25T09:16:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2023-08-25T09:16:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2023-08-25T09:16:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2023-08-25T09:16:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2023-08-25T09:16:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2023-08-25T09:16:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-25T09:16:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2023-08-25T09:16:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2023-08-25T09:16:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2023-08-25T09:16:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2023-08-25T09:16:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2023-08-25T09:16:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2023-08-25T09:16:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2023-08-25T09:16:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "services": { 2023-08-25T09:16:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "dashboard": "https://172.21.15.99:8443/" 2023-08-25T09:16:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "always_on_modules": { 2023-08-25T09:16:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "octopus": [ 2023-08-25T09:16:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2023-08-25T09:16:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2023-08-25T09:16:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2023-08-25T09:16:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2023-08-25T09:16:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2023-08-25T09:16:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2023-08-25T09:16:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2023-08-25T09:16:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2023-08-25T09:16:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2023-08-25T09:16:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2023-08-25T09:16:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "pacific": [ 2023-08-25T09:16:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2023-08-25T09:16:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2023-08-25T09:16:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2023-08-25T09:16:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2023-08-25T09:16:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2023-08-25T09:16:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2023-08-25T09:16:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2023-08-25T09:16:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2023-08-25T09:16:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2023-08-25T09:16:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2023-08-25T09:16:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "quincy": [ 2023-08-25T09:16:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2023-08-25T09:16:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2023-08-25T09:16:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2023-08-25T09:16:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2023-08-25T09:16:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2023-08-25T09:16:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2023-08-25T09:16:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2023-08-25T09:16:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2023-08-25T09:16:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2023-08-25T09:16:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2023-08-25T09:16:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-08-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "reef": [ 2023-08-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2023-08-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2023-08-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2023-08-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2023-08-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2023-08-25T09:16:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2023-08-25T09:16:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2023-08-25T09:16:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2023-08-25T09:16:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2023-08-25T09:16:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2023-08-25T09:16:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "last_failure_osd_epoch": 88, 2023-08-25T09:16:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "active_clients": [ 2023-08-25T09:16:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2023-08-25T09:16:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-08-25T09:16:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-08-25T09:16:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2023-08-25T09:16:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3807395113 2023-08-25T09:16:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "libcephsqlite", 2023-08-25T09:16:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-08-25T09:16:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-08-25T09:16:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2023-08-25T09:16:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3195749723 2023-08-25T09:16:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2023-08-25T09:16:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-08-25T09:16:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-08-25T09:16:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2023-08-25T09:16:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2200243568 2023-08-25T09:16:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2023-08-25T09:16:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-08-25T09:16:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-08-25T09:16:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2023-08-25T09:16:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 190066340 2023-08-25T09:16:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2023-08-25T09:16:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-08-25T09:16:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-08-25T09:16:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2023-08-25T09:16:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1480012710 2023-08-25T09:16:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2023-08-25T09:16:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-08-25T09:16:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-08-25T09:16:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-08-25T09:16:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2023-08-25T09:16:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2072678867 2023-08-25T09:16:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-08-25T09:16:12.739 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T09:16:12.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:09 smithi181 ceph-mon[131296]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 658 op/s 2023-08-25T09:16:12.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:09 smithi181 ceph-mon[131296]: osdmap e111: 12 total, 12 up, 12 in 2023-08-25T09:16:12.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:09 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2257901246' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-25T09:16:12.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:11 smithi181 ceph-mon[131296]: pgmap v536: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 790 op/s 2023-08-25T09:16:12.741 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:09 smithi157 ceph-mon[130879]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 658 op/s 2023-08-25T09:16:12.742 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:09 smithi157 ceph-mon[130879]: osdmap e111: 12 total, 12 up, 12 in 2023-08-25T09:16:12.742 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:09 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2257901246' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-25T09:16:12.742 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:11 smithi157 ceph-mon[130879]: pgmap v536: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 790 op/s 2023-08-25T09:16:12.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.226+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f60100a02a0 msgr2=0x7f60100a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:12.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.226+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f60100a02a0 0x7f60100a2760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6014009730 tx=0x7f6014006cb0 comp rx=0 tx=0).stop 2023-08-25T09:16:12.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.226+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6024101260 msgr2=0x7f602410d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:12.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.226+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6024101260 0x7f602410d6e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9a0 tx=0x7f602000ed60 comp rx=0 tx=0).stop 2023-08-25T09:16:12.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.227+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 shutdown_connections 2023-08-25T09:16:12.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.227+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f60100a02a0 0x7f60100a2760 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6014009730 tx=0x7f6014006cb0 comp rx=0 tx=0).stop 2023-08-25T09:16:12.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.227+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60241068b0 0x7f602410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:12.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.227+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241008a0 0x7f602410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:12.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.227+0000 7f602b7ff700 1 --2- 172.21.15.99:0/2257901246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6024101260 0x7f602410d6e0 secure :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9a0 tx=0x7f602000ed60 comp rx=0 tx=0).stop 2023-08-25T09:16:12.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.227+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 >> 172.21.15.99:0/2257901246 conn(0x7f60240fc3b0 msgr2=0x7f60240fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:12.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.232+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 shutdown_connections 2023-08-25T09:16:12.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:09.234+0000 7f602b7ff700 1 -- 172.21.15.99:0/2257901246 wait complete. 2023-08-25T09:16:12.795 DEBUG:tasks.check_counter:Getting stats from a 2023-08-25T09:16:12.795 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.a perf dump 2023-08-25T09:16:12.796 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:09 smithi099 ceph-mon[97641]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 658 op/s 2023-08-25T09:16:12.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:09 smithi099 ceph-mon[97641]: osdmap e111: 12 total, 12 up, 12 in 2023-08-25T09:16:12.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:09 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2257901246' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-25T09:16:12.797 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:11 smithi099 ceph-mon[97641]: pgmap v536: 97 pgs: 97 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 790 op/s 2023-08-25T09:16:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T09:16:13.791 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:13 smithi099 ceph-mon[97641]: pgmap v537: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 418 op/s 2023-08-25T09:16:13.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:13 smithi157 ceph-mon[130879]: pgmap v537: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 418 op/s 2023-08-25T09:16:13.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:13 smithi181 ceph-mon[131296]: pgmap v537: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 418 op/s 2023-08-25T09:16:15.131 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T09:16:15.131 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:15.132 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 4, 2023-08-25T09:16:15.132 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 4, 2023-08-25T09:16:15.132 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 2944, 2023-08-25T09:16:15.132 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 448, 2023-08-25T09:16:15.132 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2023-08-25T09:16:15.133 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2023-08-25T09:16:15.133 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.004118047, 2023-08-25T09:16:15.133 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000062473, 2023-08-25T09:16:15.134 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.001326485, 2023-08-25T09:16:15.135 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:15.135 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:15.135 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2023-08-25T09:16:15.136 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001290899, 2023-08-25T09:16:15.136 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000430299 2023-08-25T09:16:15.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.136 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:15.137 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:15.137 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:15.137 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:15.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.138 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-25T09:16:15.138 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2023-08-25T09:16:15.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.138 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:15.138 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3, 2023-08-25T09:16:15.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 234, 2023-08-25T09:16:15.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 480, 2023-08-25T09:16:15.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 269904, 2023-08-25T09:16:15.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:15.140 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2023-08-25T09:16:15.140 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.026177853, 2023-08-25T09:16:15.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.017993867, 2023-08-25T09:16:15.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.003531613, 2023-08-25T09:16:15.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:15.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:15.141 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 231, 2023-08-25T09:16:15.142 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.021717423, 2023-08-25T09:16:15.142 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000094014 2023-08-25T09:16:15.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.142 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:15.143 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:15.143 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:15.143 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:15.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.143 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T09:16:15.144 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 269904 2023-08-25T09:16:15.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.144 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:15.144 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 317, 2023-08-25T09:16:15.144 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 306, 2023-08-25T09:16:15.145 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1573184, 2023-08-25T09:16:15.145 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 184112, 2023-08-25T09:16:15.145 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2023-08-25T09:16:15.145 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2023-08-25T09:16:15.145 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.062470965, 2023-08-25T09:16:15.146 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.023968532, 2023-08-25T09:16:15.146 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.038595440, 2023-08-25T09:16:15.146 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.006598248, 2023-08-25T09:16:15.146 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:15.147 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 305, 2023-08-25T09:16:15.147 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.018765537, 2023-08-25T09:16:15.147 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000061526 2023-08-25T09:16:15.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.148 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:15.148 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:15.148 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:15.148 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:15.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.149 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1573184, 2023-08-25T09:16:15.149 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 184112 2023-08-25T09:16:15.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.149 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-08-25T09:16:15.149 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2023-08-25T09:16:15.150 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-08-25T09:16:15.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.150 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-08-25T09:16:15.151 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:15.151 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:15.151 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:15.152 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:15.152 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:15.153 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:15.153 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:15.154 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:15.154 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:15.155 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:15.155 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:15.156 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:15.156 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:15.157 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:15.157 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:15.157 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:15.158 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:15.158 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:15.159 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:15.159 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:15.159 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:15.160 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:15.160 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:15.161 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:15.161 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:15.162 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:15.162 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:15.162 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:15.162 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:15.163 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:15.163 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:15.163 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-08-25T09:16:15.163 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:15.164 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:15.164 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:15.164 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:15.164 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 149745, 2023-08-25T09:16:15.164 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 31, 2023-08-25T09:16:15.165 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:15.165 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:15.165 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-08-25T09:16:15.165 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-08-25T09:16:15.165 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:15.166 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:15.166 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:15.166 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:15.166 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2023-08-25T09:16:15.166 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2023-08-25T09:16:15.167 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:15.167 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:15.167 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:15.167 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-08-25T09:16:15.167 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2023-08-25T09:16:15.168 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2023-08-25T09:16:15.168 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:15.168 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-08-25T09:16:15.168 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:15.169 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-08-25T09:16:15.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:15.169 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:15.169 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-08-25T09:16:15.169 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-08-25T09:16:15.170 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-08-25T09:16:15.170 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 320, 2023-08-25T09:16:15.170 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1528467, 2023-08-25T09:16:15.170 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:15.171 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 320, 2023-08-25T09:16:15.171 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-08-25T09:16:15.171 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-08-25T09:16:15.171 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 320, 2023-08-25T09:16:15.171 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1528467, 2023-08-25T09:16:15.172 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-08-25T09:16:15.172 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:15.172 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:15.172 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:15.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:15.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:15.173 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T09:16:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:15 smithi099 ceph-mon[97641]: osdmap e112: 12 total, 12 up, 12 in 2023-08-25T09:16:15.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:15 smithi099 ceph-mon[97641]: pgmap v539: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 369 KiB/s wr, 38 op/s 2023-08-25T09:16:15.627 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-08-25T09:16:15.628 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-08-25T09:16:15.628 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-08-25T09:16:15.628 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-08-25T09:16:15.628 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-08-25T09:16:15.629 DEBUG:tasks.check_counter:Getting stats from d 2023-08-25T09:16:15.629 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.d perf dump 2023-08-25T09:16:15.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:15 smithi157 ceph-mon[130879]: osdmap e112: 12 total, 12 up, 12 in 2023-08-25T09:16:15.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:15 smithi157 ceph-mon[130879]: pgmap v539: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 369 KiB/s wr, 38 op/s 2023-08-25T09:16:15.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:15 smithi181 ceph-mon[131296]: osdmap e112: 12 total, 12 up, 12 in 2023-08-25T09:16:15.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:15 smithi181 ceph-mon[131296]: pgmap v539: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 369 KiB/s wr, 38 op/s 2023-08-25T09:16:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T09:16:17.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:17 smithi099 ceph-mon[97641]: pgmap v540: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 439 KiB/s wr, 47 op/s 2023-08-25T09:16:17.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:17 smithi157 ceph-mon[130879]: pgmap v540: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 439 KiB/s wr, 47 op/s 2023-08-25T09:16:17.907 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T09:16:17.907 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:17.908 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 326, 2023-08-25T09:16:17.908 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 313, 2023-08-25T09:16:17.908 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1577488, 2023-08-25T09:16:17.908 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 187680, 2023-08-25T09:16:17.909 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2023-08-25T09:16:17.909 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2023-08-25T09:16:17.909 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.062670271, 2023-08-25T09:16:17.909 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.018994818, 2023-08-25T09:16:17.910 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.041396721, 2023-08-25T09:16:17.910 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.007252885, 2023-08-25T09:16:17.910 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:17.910 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 313, 2023-08-25T09:16:17.910 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.021406180, 2023-08-25T09:16:17.911 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000068390 2023-08-25T09:16:17.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.911 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:17.911 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:17.911 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:17.912 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:17.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.912 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1577488, 2023-08-25T09:16:17.912 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 187680 2023-08-25T09:16:17.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.913 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:17.913 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3, 2023-08-25T09:16:17.913 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 238, 2023-08-25T09:16:17.914 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 480, 2023-08-25T09:16:17.914 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 270608, 2023-08-25T09:16:17.914 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:17.915 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2023-08-25T09:16:17.915 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.024239265, 2023-08-25T09:16:17.915 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.016624921, 2023-08-25T09:16:17.915 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.003360960, 2023-08-25T09:16:17.915 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:17.916 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:17.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 235, 2023-08-25T09:16:17.916 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.022796264, 2023-08-25T09:16:17.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000097005 2023-08-25T09:16:17.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:17.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:17.918 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:17.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:17.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T09:16:17.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 270608 2023-08-25T09:16:17.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.919 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:17.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2023-08-25T09:16:17.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2023-08-25T09:16:17.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2023-08-25T09:16:17.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2023-08-25T09:16:17.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2023-08-25T09:16:17.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2023-08-25T09:16:17.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.000721670, 2023-08-25T09:16:17.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T09:16:17.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000552002, 2023-08-25T09:16:17.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:17.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:17.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:17.922 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:17.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:17.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.922 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:17.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:17.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:17.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:17.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.923 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T09:16:17.924 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T09:16:17.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.924 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-08-25T09:16:17.924 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2023-08-25T09:16:17.925 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-08-25T09:16:17.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.925 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-08-25T09:16:17.925 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:17.926 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:17.926 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:17.926 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:17.926 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:17.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:17.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:17.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:17.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:17.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:17.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:17.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:17.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:17.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:17.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:17.929 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:17.929 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:17.929 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:17.929 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:17.929 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:17.930 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:17.930 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:17.930 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:17.930 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:17.930 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:17.931 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:17.931 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:17.931 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:17.931 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:17.932 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:17.932 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:17.932 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-08-25T09:16:17.932 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:17.932 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:17.933 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:17.933 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:17.933 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 149745, 2023-08-25T09:16:17.933 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 31, 2023-08-25T09:16:17.933 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:17.934 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:17.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-08-25T09:16:17.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-08-25T09:16:17.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:17.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:17.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:17.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:17.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2023-08-25T09:16:17.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2023-08-25T09:16:17.936 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:17.936 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:17.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:17.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-08-25T09:16:17.937 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2023-08-25T09:16:17.937 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2023-08-25T09:16:17.937 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:17.938 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-08-25T09:16:17.938 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:17.938 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-08-25T09:16:17.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:17.939 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:17.939 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-08-25T09:16:17.939 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-08-25T09:16:17.939 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-08-25T09:16:17.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 325, 2023-08-25T09:16:17.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1529117, 2023-08-25T09:16:17.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:17.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 325, 2023-08-25T09:16:17.940 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-08-25T09:16:17.941 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-08-25T09:16:17.941 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 325, 2023-08-25T09:16:17.941 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1529117, 2023-08-25T09:16:17.941 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-08-25T09:16:17.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:17.942 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:17.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:17.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:17.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:17.942 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T09:16:17.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:17 smithi181 ceph-mon[131296]: pgmap v540: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 439 KiB/s wr, 47 op/s 2023-08-25T09:16:18.344 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-08-25T09:16:18.345 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-08-25T09:16:18.345 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-08-25T09:16:18.345 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-08-25T09:16:18.346 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-08-25T09:16:18.346 DEBUG:tasks.check_counter:Getting stats from g 2023-08-25T09:16:18.346 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.g perf dump 2023-08-25T09:16:19.068 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T09:16:19.601 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:19 smithi099 ceph-mon[97641]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 353 KiB/s wr, 38 op/s 2023-08-25T09:16:19.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:19 smithi157 ceph-mon[130879]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 353 KiB/s wr, 38 op/s 2023-08-25T09:16:19.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:19 smithi181 ceph-mon[131296]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 353 KiB/s wr, 38 op/s 2023-08-25T09:16:20.586 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T09:16:20.586 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:20.587 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2023-08-25T09:16:20.587 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2023-08-25T09:16:20.587 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2023-08-25T09:16:20.587 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2023-08-25T09:16:20.588 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2023-08-25T09:16:20.588 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2023-08-25T09:16:20.588 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.003482390, 2023-08-25T09:16:20.589 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T09:16:20.589 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000728950, 2023-08-25T09:16:20.589 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:20.589 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:20.589 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:20.590 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:20.590 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:20.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.590 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:20.590 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:20.590 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:20.591 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:20.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.591 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T09:16:20.591 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T09:16:20.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.592 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:20.592 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 323, 2023-08-25T09:16:20.592 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 310, 2023-08-25T09:16:20.592 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1576672, 2023-08-25T09:16:20.592 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 185808, 2023-08-25T09:16:20.593 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2023-08-25T09:16:20.593 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2023-08-25T09:16:20.593 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.063880961, 2023-08-25T09:16:20.593 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.024887896, 2023-08-25T09:16:20.593 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.039737458, 2023-08-25T09:16:20.594 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.006633434, 2023-08-25T09:16:20.594 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:20.594 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 310, 2023-08-25T09:16:20.595 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.021899633, 2023-08-25T09:16:20.595 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000070643 2023-08-25T09:16:20.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.595 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:20.595 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:20.595 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:20.596 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:20.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.596 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1576672, 2023-08-25T09:16:20.596 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 185808 2023-08-25T09:16:20.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.597 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:20.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3, 2023-08-25T09:16:20.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 235, 2023-08-25T09:16:20.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 480, 2023-08-25T09:16:20.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 270080, 2023-08-25T09:16:20.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:20.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2023-08-25T09:16:20.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.024504760, 2023-08-25T09:16:20.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.016669218, 2023-08-25T09:16:20.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.003458728, 2023-08-25T09:16:20.599 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:20.599 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:20.599 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 232, 2023-08-25T09:16:20.599 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.019213673, 2023-08-25T09:16:20.599 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000082817 2023-08-25T09:16:20.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.600 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:20.600 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:20.600 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:20.600 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:20.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.601 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T09:16:20.601 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 270080 2023-08-25T09:16:20.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.601 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-08-25T09:16:20.602 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2023-08-25T09:16:20.602 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-08-25T09:16:20.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.602 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-08-25T09:16:20.602 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:20.602 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:20.603 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:20.603 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:20.603 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:20.603 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:20.604 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:20.604 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:20.604 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:20.604 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:20.605 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:20.605 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:20.605 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:20.605 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:20.606 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:20.606 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:20.606 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:20.606 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:20.606 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:20.607 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:20.607 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:20.607 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:20.607 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:20.607 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:20.607 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:20.608 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:20.608 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:20.608 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:20.608 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:20.609 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:20.609 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:20.609 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-08-25T09:16:20.609 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:20.610 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:20.610 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:20.610 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:20.610 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148731, 2023-08-25T09:16:20.610 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 31, 2023-08-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-08-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-08-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:20.612 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:20.612 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:20.612 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2023-08-25T09:16:20.612 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2023-08-25T09:16:20.613 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:20.613 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:20.613 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:20.613 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-08-25T09:16:20.613 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2023-08-25T09:16:20.613 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2023-08-25T09:16:20.614 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:20.614 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-08-25T09:16:20.614 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:20.614 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-08-25T09:16:20.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:20.615 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:20.615 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-08-25T09:16:20.615 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-08-25T09:16:20.615 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-08-25T09:16:20.615 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 322, 2023-08-25T09:16:20.616 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1528727, 2023-08-25T09:16:20.616 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:20.616 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 322, 2023-08-25T09:16:20.616 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-08-25T09:16:20.616 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-08-25T09:16:20.617 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 322, 2023-08-25T09:16:20.617 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1528727, 2023-08-25T09:16:20.617 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-08-25T09:16:20.617 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:20.617 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:20.618 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:20.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:20.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:20.618 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T09:16:21.051 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-08-25T09:16:21.052 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-08-25T09:16:21.052 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-08-25T09:16:21.052 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-08-25T09:16:21.052 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-08-25T09:16:21.053 DEBUG:tasks.check_counter:Getting stats from j 2023-08-25T09:16:21.053 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.j perf dump 2023-08-25T09:16:21.777 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T09:16:21.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:21 smithi099 ceph-mon[97641]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 351 KiB/s wr, 37 op/s 2023-08-25T09:16:21.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:21 smithi157 ceph-mon[130879]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 351 KiB/s wr, 37 op/s 2023-08-25T09:16:21.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:21 smithi181 ceph-mon[131296]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 351 KiB/s wr, 37 op/s 2023-08-25T09:16:23.327 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-08-25T09:16:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2023-08-25T09:16:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2023-08-25T09:16:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2023-08-25T09:16:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2023-08-25T09:16:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2023-08-25T09:16:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2023-08-25T09:16:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.003180428, 2023-08-25T09:16:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T09:16:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000692938, 2023-08-25T09:16:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:23.331 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:23.331 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:23.331 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:23.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.331 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T09:16:23.331 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T09:16:23.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.332 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:23.332 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 7, 2023-08-25T09:16:23.332 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 242, 2023-08-25T09:16:23.332 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 3424, 2023-08-25T09:16:23.333 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 271408, 2023-08-25T09:16:23.333 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:23.333 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2023-08-25T09:16:23.333 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.026701836, 2023-08-25T09:16:23.333 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.018398693, 2023-08-25T09:16:23.334 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.003815138, 2023-08-25T09:16:23.334 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:23.334 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:23.334 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 240, 2023-08-25T09:16:23.334 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.024682732, 2023-08-25T09:16:23.335 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000102844 2023-08-25T09:16:23.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.335 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:23.335 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:23.335 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:23.336 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:23.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.336 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-08-25T09:16:23.336 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 271408 2023-08-25T09:16:23.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.337 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:23.337 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 324, 2023-08-25T09:16:23.337 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 313, 2023-08-25T09:16:23.337 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1575088, 2023-08-25T09:16:23.337 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 188480, 2023-08-25T09:16:23.337 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2023-08-25T09:16:23.338 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2023-08-25T09:16:23.338 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.065336587, 2023-08-25T09:16:23.338 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.024930223, 2023-08-25T09:16:23.338 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.042364740, 2023-08-25T09:16:23.338 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.006764854, 2023-08-25T09:16:23.339 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:23.339 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 312, 2023-08-25T09:16:23.339 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.021219744, 2023-08-25T09:16:23.339 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000068012 2023-08-25T09:16:23.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.340 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:23.340 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:23.340 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:23.340 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:23.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.341 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1575088, 2023-08-25T09:16:23.341 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 188480 2023-08-25T09:16:23.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.341 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-08-25T09:16:23.341 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2023-08-25T09:16:23.341 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-08-25T09:16:23.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.342 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-08-25T09:16:23.342 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:23.342 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:23.342 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:23.344 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:23.344 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:23.344 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:23.344 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:23.344 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:23.345 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:23.345 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:23.345 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:23.345 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:23.345 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:23.346 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:23.346 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:23.346 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:23.348 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:23.348 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:23.348 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:23.349 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:23.349 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:23.349 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:23.349 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:23.349 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:23.350 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:23.350 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:23.350 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-08-25T09:16:23.350 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:23.350 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:23.350 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:23.351 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:23.351 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148731, 2023-08-25T09:16:23.351 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 31, 2023-08-25T09:16:23.351 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:23.351 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:23.352 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-08-25T09:16:23.352 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-08-25T09:16:23.352 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:23.352 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:23.352 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:23.353 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:23.353 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2023-08-25T09:16:23.353 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2023-08-25T09:16:23.353 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:23.353 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:23.354 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:23.354 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-08-25T09:16:23.354 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2023-08-25T09:16:23.354 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2023-08-25T09:16:23.354 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:23.355 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-08-25T09:16:23.355 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:23.355 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-08-25T09:16:23.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-08-25T09:16:23.355 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:23.355 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-08-25T09:16:23.356 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-08-25T09:16:23.356 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-08-25T09:16:23.356 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 327, 2023-08-25T09:16:23.356 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1529377, 2023-08-25T09:16:23.356 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:23.357 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 327, 2023-08-25T09:16:23.357 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-08-25T09:16:23.357 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-08-25T09:16:23.357 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 327, 2023-08-25T09:16:23.357 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1529377, 2023-08-25T09:16:23.358 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-08-25T09:16:23.358 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-08-25T09:16:23.358 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-08-25T09:16:23.358 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-08-25T09:16:23.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:23.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-08-25T09:16:23.359 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-08-25T09:16:23.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:23 smithi099 ceph-mon[97641]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 15 op/s 2023-08-25T09:16:23.766 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-08-25T09:16:23.766 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-08-25T09:16:23.766 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-08-25T09:16:23.767 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-08-25T09:16:23.767 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-08-25T09:16:23.767 DEBUG:tasks.check_counter:Getting stats from b 2023-08-25T09:16:23.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.b perf dump 2023-08-25T09:16:23.865 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:23 smithi157 ceph-mon[130879]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 15 op/s 2023-08-25T09:16:23.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:23 smithi181 ceph-mon[131296]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 15 op/s 2023-08-25T09:16:24.441 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T09:16:25.554 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:25 smithi157 ceph-mon[130879]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 91 KiB/s wr, 15 op/s 2023-08-25T09:16:25.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:25 smithi181 ceph-mon[131296]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 91 KiB/s wr, 15 op/s 2023-08-25T09:16:25.766 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-08-25T09:16:25.767 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:25.767 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 126389, 2023-08-25T09:16:25.767 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 147790, 2023-08-25T09:16:25.768 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 37894028, 2023-08-25T09:16:25.768 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 339502218, 2023-08-25T09:16:25.769 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 10, 2023-08-25T09:16:25.769 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-08-25T09:16:25.769 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 7.653549947, 2023-08-25T09:16:25.769 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 3.015148243, 2023-08-25T09:16:25.769 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 3.603941668, 2023-08-25T09:16:25.770 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 1.277982969, 2023-08-25T09:16:25.770 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:25.770 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 147790, 2023-08-25T09:16:25.770 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 29.048195685, 2023-08-25T09:16:25.770 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000196550 2023-08-25T09:16:25.771 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.771 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:25.771 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 184906, 2023-08-25T09:16:25.771 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.030123957, 2023-08-25T09:16:25.772 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000162 2023-08-25T09:16:25.772 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.772 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 160, 2023-08-25T09:16:25.772 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 453120 2023-08-25T09:16:25.772 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.773 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:25.773 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 154996, 2023-08-25T09:16:25.773 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 149465, 2023-08-25T09:16:25.773 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 48217486, 2023-08-25T09:16:25.773 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 208888359, 2023-08-25T09:16:25.774 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 14, 2023-08-25T09:16:25.774 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2023-08-25T09:16:25.774 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 7.876429802, 2023-08-25T09:16:25.774 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 2.917857538, 2023-08-25T09:16:25.774 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 4.696139989, 2023-08-25T09:16:25.775 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 1.000907574, 2023-08-25T09:16:25.775 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:25.775 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 149466, 2023-08-25T09:16:25.775 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 32.349364098, 2023-08-25T09:16:25.776 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000216432 2023-08-25T09:16:25.777 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.777 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:25.777 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 236871, 2023-08-25T09:16:25.777 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.033542981, 2023-08-25T09:16:25.777 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000141 2023-08-25T09:16:25.778 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.778 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2293120, 2023-08-25T09:16:25.778 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 368112 2023-08-25T09:16:25.778 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.778 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:25.779 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 292298, 2023-08-25T09:16:25.779 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 422864, 2023-08-25T09:16:25.779 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 95345932, 2023-08-25T09:16:25.779 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 391960216, 2023-08-25T09:16:25.779 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 161, 2023-08-25T09:16:25.780 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 9, 2023-08-25T09:16:25.780 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 15.272498244, 2023-08-25T09:16:25.780 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 6.623150678, 2023-08-25T09:16:25.780 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 10.227953820, 2023-08-25T09:16:25.780 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.681841252, 2023-08-25T09:16:25.781 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:25.781 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 422858, 2023-08-25T09:16:25.781 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 36.543438663, 2023-08-25T09:16:25.781 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000086420 2023-08-25T09:16:25.781 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.781 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:25.782 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 562495, 2023-08-25T09:16:25.782 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.097095947, 2023-08-25T09:16:25.782 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000172 2023-08-25T09:16:25.782 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.782 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 102816, 2023-08-25T09:16:25.783 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 152544 2023-08-25T09:16:25.783 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.783 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2023-08-25T09:16:25.783 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2023-08-25T09:16:25.784 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2023-08-25T09:16:25.784 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.784 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2023-08-25T09:16:25.784 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-08-25T09:16:25.784 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-08-25T09:16:25.785 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 96609, 2023-08-25T09:16:25.785 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 91.866817578, 2023-08-25T09:16:25.785 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000950913 2023-08-25T09:16:25.785 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.785 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.786 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2023-08-25T09:16:25.786 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-08-25T09:16:25.786 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-08-25T09:16:25.786 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 40283, 2023-08-25T09:16:25.787 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 8.122979969, 2023-08-25T09:16:25.787 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000201647 2023-08-25T09:16:25.787 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.787 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.787 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2023-08-25T09:16:25.788 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 184565, 2023-08-25T09:16:25.788 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 169247, 2023-08-25T09:16:25.788 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2023-08-25T09:16:25.788 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 169247, 2023-08-25T09:16:25.788 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1185.364446044, 2023-08-25T09:16:25.789 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.007003754 2023-08-25T09:16:25.789 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.789 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2023-08-25T09:16:25.789 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 15318, 2023-08-25T09:16:25.789 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 31, 2023-08-25T09:16:25.789 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2023-08-25T09:16:25.790 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 2734, 2023-08-25T09:16:25.790 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 16, 2023-08-25T09:16:25.790 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 11, 2023-08-25T09:16:25.790 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 2126, 2023-08-25T09:16:25.790 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 875, 2023-08-25T09:16:25.791 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 1211, 2023-08-25T09:16:25.791 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 40, 2023-08-25T09:16:25.791 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 876, 2023-08-25T09:16:25.792 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 2791, 2023-08-25T09:16:25.792 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 315, 2023-08-25T09:16:25.792 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 315, 2023-08-25T09:16:25.792 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 23, 2023-08-25T09:16:25.793 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 285906, 2023-08-25T09:16:25.793 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 184134, 2023-08-25T09:16:25.793 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 15308, 2023-08-25T09:16:25.793 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 177, 2023-08-25T09:16:25.793 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 9, 2023-08-25T09:16:25.794 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2023-08-25T09:16:25.794 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 3557, 2023-08-25T09:16:25.794 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 80409, 2023-08-25T09:16:25.794 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2023-08-25T09:16:25.794 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 16, 2023-08-25T09:16:25.795 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 1869, 2023-08-25T09:16:25.795 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 18, 2023-08-25T09:16:25.795 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 705, 2023-08-25T09:16:25.795 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2023-08-25T09:16:25.795 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2023-08-25T09:16:25.796 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2023-08-25T09:16:25.796 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 744, 2023-08-25T09:16:25.796 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 215161046, 2023-08-25T09:16:25.796 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2023-08-25T09:16:25.796 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T09:16:25.797 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2023-08-25T09:16:25.797 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T09:16:25.797 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2023-08-25T09:16:25.797 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2023-08-25T09:16:25.797 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T09:16:25.798 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T09:16:25.798 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2023-08-25T09:16:25.798 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 15, 2023-08-25T09:16:25.798 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 35888, 2023-08-25T09:16:25.798 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 94494, 2023-08-25T09:16:25.799 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T09:16:25.799 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 2353, 2023-08-25T09:16:25.799 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T09:16:25.799 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 68440, 2023-08-25T09:16:25.799 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 24451, 2023-08-25T09:16:25.799 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 8148, 2023-08-25T09:16:25.800 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 112030 2023-08-25T09:16:25.800 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.800 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2023-08-25T09:16:25.800 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 15, 2023-08-25T09:16:25.800 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 21, 2023-08-25T09:16:25.801 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 14, 2023-08-25T09:16:25.801 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 184, 2023-08-25T09:16:25.801 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 154, 2023-08-25T09:16:25.801 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 91, 2023-08-25T09:16:25.801 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2023-08-25T09:16:25.802 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2023-08-25T09:16:25.802 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 24779, 2023-08-25T09:16:25.802 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 24688, 2023-08-25T09:16:25.802 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2023-08-25T09:16:25.802 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2023-08-25T09:16:25.803 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2023-08-25T09:16:25.803 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2023-08-25T09:16:25.803 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2023-08-25T09:16:25.803 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2023-08-25T09:16:25.803 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2023-08-25T09:16:25.804 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T09:16:25.804 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 16, 2023-08-25T09:16:25.804 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2023-08-25T09:16:25.804 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 27, 2023-08-25T09:16:25.804 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2023-08-25T09:16:25.805 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2023-08-25T09:16:25.805 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.805 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2023-08-25T09:16:25.805 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2023-08-25T09:16:25.805 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 131590, 2023-08-25T09:16:25.806 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 130641, 2023-08-25T09:16:25.806 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 130641, 2023-08-25T09:16:25.806 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 949, 2023-08-25T09:16:25.806 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2023-08-25T09:16:25.806 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 3530, 2023-08-25T09:16:25.807 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 248, 2023-08-25T09:16:25.807 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 246, 2023-08-25T09:16:25.807 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 246, 2023-08-25T09:16:25.807 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2023-08-25T09:16:25.807 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 1, 2023-08-25T09:16:25.807 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2023-08-25T09:16:25.808 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 5, 2023-08-25T09:16:25.808 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 519793639, 2023-08-25T09:16:25.808 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 521854198, 2023-08-25T09:16:25.808 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2023-08-25T09:16:25.808 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2023-08-25T09:16:25.809 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 51892, 2023-08-25T09:16:25.809 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 274.891961705, 2023-08-25T09:16:25.809 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005297386 2023-08-25T09:16:25.809 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.809 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2023-08-25T09:16:25.810 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.810 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2023-08-25T09:16:25.810 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 1241, 2023-08-25T09:16:25.810 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 27876, 2023-08-25T09:16:25.810 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 26635, 2023-08-25T09:16:25.811 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 172, 2023-08-25T09:16:25.811 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 395, 2023-08-25T09:16:25.811 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 223, 2023-08-25T09:16:25.811 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 2127, 2023-08-25T09:16:25.811 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 51820, 2023-08-25T09:16:25.811 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 49693, 2023-08-25T09:16:25.812 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 315, 2023-08-25T09:16:25.812 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 26759, 2023-08-25T09:16:25.812 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 26444, 2023-08-25T09:16:25.812 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 88108, 2023-08-25T09:16:25.812 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207152 2023-08-25T09:16:25.813 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.813 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2023-08-25T09:16:25.813 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 267421, 2023-08-25T09:16:25.813 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 14195, 2023-08-25T09:16:25.813 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 184565, 2023-08-25T09:16:25.814 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 3867, 2023-08-25T09:16:25.814 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 30184, 2023-08-25T09:16:25.814 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2023-08-25T09:16:25.814 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 25458, 2023-08-25T09:16:25.814 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 131.706615476, 2023-08-25T09:16:25.815 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005173486 2023-08-25T09:16:25.815 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.815 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2023-08-25T09:16:25.815 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 24594, 2023-08-25T09:16:25.815 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 141.795892895, 2023-08-25T09:16:25.816 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005765466 2023-08-25T09:16:25.816 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.816 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2023-08-25T09:16:25.816 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.816 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.817 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.817 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.817 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2023-08-25T09:16:25.817 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.817 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.818 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.818 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.818 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2023-08-25T09:16:25.818 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 38517, 2023-08-25T09:16:25.818 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 353.611707579, 2023-08-25T09:16:25.818 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.009180665 2023-08-25T09:16:25.819 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.819 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2023-08-25T09:16:25.819 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.819 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.819 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.820 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.820 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2023-08-25T09:16:25.820 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-08-25T09:16:25.820 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000195325, 2023-08-25T09:16:25.820 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000195325 2023-08-25T09:16:25.821 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.821 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2023-08-25T09:16:25.821 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.821 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.821 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.822 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.822 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2023-08-25T09:16:25.822 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.822 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.822 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.823 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.823 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2023-08-25T09:16:25.823 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.823 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.823 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.824 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.824 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2023-08-25T09:16:25.824 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 17, 2023-08-25T09:16:25.824 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.009686099, 2023-08-25T09:16:25.824 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000569770 2023-08-25T09:16:25.825 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.825 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2023-08-25T09:16:25.825 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 119, 2023-08-25T09:16:25.825 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.690046388, 2023-08-25T09:16:25.825 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005798709 2023-08-25T09:16:25.825 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.826 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2023-08-25T09:16:25.826 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.826 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.826 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.826 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.827 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2023-08-25T09:16:25.827 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 17, 2023-08-25T09:16:25.827 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 7.951061108, 2023-08-25T09:16:25.827 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.467709476 2023-08-25T09:16:25.827 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.828 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2023-08-25T09:16:25.828 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 9, 2023-08-25T09:16:25.828 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.011243122, 2023-08-25T09:16:25.828 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001249235 2023-08-25T09:16:25.828 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.829 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2023-08-25T09:16:25.829 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 48515, 2023-08-25T09:16:25.829 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 273.251784623, 2023-08-25T09:16:25.829 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005632315 2023-08-25T09:16:25.829 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.830 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2023-08-25T09:16:25.830 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6065, 2023-08-25T09:16:25.830 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 135.706801857, 2023-08-25T09:16:25.830 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.022375400 2023-08-25T09:16:25.830 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.831 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2023-08-25T09:16:25.831 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.831 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.831 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.831 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.831 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2023-08-25T09:16:25.832 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.832 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.832 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.832 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.832 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2023-08-25T09:16:25.833 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-08-25T09:16:25.833 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000912995, 2023-08-25T09:16:25.833 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000912995 2023-08-25T09:16:25.833 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.833 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2023-08-25T09:16:25.834 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11, 2023-08-25T09:16:25.834 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 6.660802241, 2023-08-25T09:16:25.834 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.605527476 2023-08-25T09:16:25.834 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.834 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2023-08-25T09:16:25.835 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.835 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.835 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.835 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.835 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2023-08-25T09:16:25.836 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 34, 2023-08-25T09:16:25.836 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.673413698, 2023-08-25T09:16:25.836 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.019806285 2023-08-25T09:16:25.836 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.836 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2023-08-25T09:16:25.837 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.837 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.837 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.837 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.837 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2023-08-25T09:16:25.838 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1104, 2023-08-25T09:16:25.838 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.912525543, 2023-08-25T09:16:25.838 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001732360 2023-08-25T09:16:25.838 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.838 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2023-08-25T09:16:25.839 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.839 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.839 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.839 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.839 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2023-08-25T09:16:25.840 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6, 2023-08-25T09:16:25.840 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.019716907, 2023-08-25T09:16:25.840 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003286151 2023-08-25T09:16:25.840 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.841 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2023-08-25T09:16:25.841 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.841 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.841 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.841 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.842 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2023-08-25T09:16:25.842 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 24779, 2023-08-25T09:16:25.842 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 131.362040188, 2023-08-25T09:16:25.842 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005301345 2023-08-25T09:16:25.842 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.843 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2023-08-25T09:16:25.843 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2023-08-25T09:16:25.843 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2023-08-25T09:16:25.843 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.843 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.844 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.844 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.844 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.844 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2023-08-25T09:16:25.844 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2023-08-25T09:16:25.845 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2023-08-25T09:16:25.845 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2023-08-25T09:16:25.845 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2023-08-25T09:16:25.845 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2023-08-25T09:16:25.845 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 33045, 2023-08-25T09:16:25.846 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 16522, 2023-08-25T09:16:25.846 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 1012 2023-08-25T09:16:25.846 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.846 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2023-08-25T09:16:25.846 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 360, 2023-08-25T09:16:25.847 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 360, 2023-08-25T09:16:25.847 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:25.847 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:25.847 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:25.847 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:25.848 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:25.848 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:25.848 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:25.848 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:25.848 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:25.849 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:25.849 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:25.849 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:25.849 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:25.849 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:25.850 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:25.850 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:25.850 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:25.850 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:25.850 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:25.851 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:25.851 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:25.851 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:25.851 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:25.851 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:25.852 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:25.852 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:25.852 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:25.852 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:25.852 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:25.853 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2023-08-25T09:16:25.853 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:25.853 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:25.853 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:25.853 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:25.854 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 342596, 2023-08-25T09:16:25.854 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 1082, 2023-08-25T09:16:25.854 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 88, 2023-08-25T09:16:25.854 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 1, 2023-08-25T09:16:25.854 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2023-08-25T09:16:25.855 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2023-08-25T09:16:25.855 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:25.855 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:25.855 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:25.855 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:25.856 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8416, 2023-08-25T09:16:25.856 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 175, 2023-08-25T09:16:25.856 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:25.856 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:25.856 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:25.856 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2023-08-25T09:16:25.857 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 4949818, 2023-08-25T09:16:25.857 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 58509, 2023-08-25T09:16:25.857 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:25.857 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2023-08-25T09:16:25.857 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:25.858 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2023-08-25T09:16:25.858 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.858 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2023-08-25T09:16:25.858 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2023-08-25T09:16:25.858 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2023-08-25T09:16:25.859 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 96293, 2023-08-25T09:16:25.859 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 545507591, 2023-08-25T09:16:25.859 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 31, 2023-08-25T09:16:25.859 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 96293, 2023-08-25T09:16:25.859 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2023-08-25T09:16:25.860 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 96293, 2023-08-25T09:16:25.860 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1689.028470880, 2023-08-25T09:16:25.860 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.017540511 2023-08-25T09:16:25.860 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.860 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2023-08-25T09:16:25.861 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2023-08-25T09:16:25.861 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 96293, 2023-08-25T09:16:25.861 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 129917 2023-08-25T09:16:25.861 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.862 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 96293, 2023-08-25T09:16:25.862 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 848, 2023-08-25T09:16:25.862 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 95445, 2023-08-25T09:16:25.862 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2023-08-25T09:16:25.862 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2023-08-25T09:16:25.863 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 2466, 2023-08-25T09:16:25.863 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 13450, 2023-08-25T09:16:25.863 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 754, 2023-08-25T09:16:25.863 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 52646, 2023-08-25T09:16:25.863 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 718, 2023-08-25T09:16:25.863 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2023-08-25T09:16:25.864 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2023-08-25T09:16:25.864 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2023-08-25T09:16:25.864 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 16, 2023-08-25T09:16:25.864 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 25533, 2023-08-25T09:16:25.865 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2023-08-25T09:16:25.865 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2023-08-25T09:16:25.865 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2023-08-25T09:16:25.865 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 23, 2023-08-25T09:16:25.865 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 26900, 2023-08-25T09:16:25.866 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2023-08-25T09:16:25.866 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2023-08-25T09:16:25.866 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2023-08-25T09:16:25.866 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2023-08-25T09:16:25.867 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2023-08-25T09:16:25.867 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2023-08-25T09:16:25.867 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T09:16:25.867 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2023-08-25T09:16:25.867 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2023-08-25T09:16:25.868 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2023-08-25T09:16:25.868 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2023-08-25T09:16:25.868 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2023-08-25T09:16:25.868 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2023-08-25T09:16:25.868 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2023-08-25T09:16:25.869 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2023-08-25T09:16:25.869 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2023-08-25T09:16:25.869 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2023-08-25T09:16:25.869 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2023-08-25T09:16:25.869 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2023-08-25T09:16:25.870 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2023-08-25T09:16:25.870 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2023-08-25T09:16:25.870 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2023-08-25T09:16:25.870 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2023-08-25T09:16:25.870 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2023-08-25T09:16:25.871 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2023-08-25T09:16:25.871 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2023-08-25T09:16:25.871 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 112, 2023-08-25T09:16:25.871 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2023-08-25T09:16:25.871 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 30, 2023-08-25T09:16:25.872 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 12, 2023-08-25T09:16:25.872 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 12, 2023-08-25T09:16:25.872 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2023-08-25T09:16:25.872 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2023-08-25T09:16:25.872 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 5659, 2023-08-25T09:16:25.873 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 125, 2023-08-25T09:16:25.873 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 1627 2023-08-25T09:16:25.873 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.873 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2023-08-25T09:16:25.873 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2023-08-25T09:16:25.874 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2023-08-25T09:16:25.874 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2023-08-25T09:16:25.874 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2023-08-25T09:16:25.874 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.874 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2023-08-25T09:16:25.875 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2023-08-25T09:16:25.875 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.875 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2023-08-25T09:16:25.875 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2023-08-25T09:16:25.875 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 10, 2023-08-25T09:16:25.876 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2023-08-25T09:16:25.876 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 10, 2023-08-25T09:16:25.876 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 24688, 2023-08-25T09:16:25.876 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2023-08-25T09:16:25.877 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.877 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:25.877 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:25.877 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-08-25T09:16:25.877 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:25.878 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 573679, 2023-08-25T09:16:25.878 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 129736270, 2023-08-25T09:16:25.878 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:25.878 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 573679, 2023-08-25T09:16:25.878 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:25.879 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:25.879 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 573679, 2023-08-25T09:16:25.879 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 129736270, 2023-08-25T09:16:25.879 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:25.879 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.880 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.880 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.880 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.880 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.880 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2023-08-25T09:16:25.881 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:25.881 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-08-25T09:16:25.881 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:25.881 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:25.881 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:25.882 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:25.882 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:25.882 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 96293, 2023-08-25T09:16:25.882 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 548002145, 2023-08-25T09:16:25.883 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 96293, 2023-08-25T09:16:25.883 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 548002145, 2023-08-25T09:16:25.883 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:25.883 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.883 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.884 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.884 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.884 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.884 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2023-08-25T09:16:25.884 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:25.885 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2023-08-25T09:16:25.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:25.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:25.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:25.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:25.886 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:25.886 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 96293, 2023-08-25T09:16:25.886 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 96293, 2023-08-25T09:16:25.886 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 96293, 2023-08-25T09:16:25.886 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 96293, 2023-08-25T09:16:25.887 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:25.887 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.887 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.887 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.887 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.887 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.888 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2023-08-25T09:16:25.888 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:25.888 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-08-25T09:16:25.888 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:25.888 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 24688, 2023-08-25T09:16:25.889 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 2496912, 2023-08-25T09:16:25.889 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:25.889 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 24688, 2023-08-25T09:16:25.889 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:25.889 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:25.890 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 754, 2023-08-25T09:16:25.890 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2496912, 2023-08-25T09:16:25.890 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:25.890 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.890 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.891 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.891 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.891 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:25.891 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x556c06b5c6e0": { 2023-08-25T09:16:25.891 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:25.892 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-08-25T09:16:25.892 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:25.892 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 131590, 2023-08-25T09:16:25.892 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 517659894, 2023-08-25T09:16:25.892 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:25.893 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 131590, 2023-08-25T09:16:25.893 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:25.893 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:25.893 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 51892, 2023-08-25T09:16:25.893 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 517659894, 2023-08-25T09:16:25.894 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:25.894 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:25.894 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:25.894 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:25.894 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.895 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:25.895 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-08-25T09:16:25.896 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:25 smithi099 ceph-mon[97641]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 91 KiB/s wr, 15 op/s 2023-08-25T09:16:26.215 INFO:tasks.check_counter:Daemon mds.b mds.exported=16 2023-08-25T09:16:26.215 INFO:tasks.check_counter:Daemon mds.b mds.imported=18 2023-08-25T09:16:26.215 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-08-25T09:16:26.216 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=17 2023-08-25T09:16:26.216 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=11 2023-08-25T09:16:26.216 DEBUG:tasks.check_counter:Getting stats from e 2023-08-25T09:16:26.216 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.e perf dump 2023-08-25T09:16:26.769 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T09:16:27.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:27 smithi099 ceph-mon[97641]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 17 op/s 2023-08-25T09:16:27.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:27 smithi157 ceph-mon[130879]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 17 op/s 2023-08-25T09:16:27.888 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-08-25T09:16:27.888 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:27.889 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1294, 2023-08-25T09:16:27.889 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1296, 2023-08-25T09:16:27.889 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 193275895, 2023-08-25T09:16:27.889 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 402498, 2023-08-25T09:16:27.890 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2023-08-25T09:16:27.890 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-08-25T09:16:27.890 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.190505570, 2023-08-25T09:16:27.890 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.029082601, 2023-08-25T09:16:27.890 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.116330597, 2023-08-25T09:16:27.891 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.034823178, 2023-08-25T09:16:27.891 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:27.891 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1294, 2023-08-25T09:16:27.891 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.060234929, 2023-08-25T09:16:27.891 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000046549 2023-08-25T09:16:27.892 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.892 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:27.892 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.892 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.892 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.893 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.893 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T09:16:27.893 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T09:16:27.893 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.893 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:27.894 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1333, 2023-08-25T09:16:27.894 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1288, 2023-08-25T09:16:27.894 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 106852698, 2023-08-25T09:16:27.894 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 486882, 2023-08-25T09:16:27.894 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:27.895 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2023-08-25T09:16:27.895 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.191910974, 2023-08-25T09:16:27.895 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.037031099, 2023-08-25T09:16:27.895 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.121785124, 2023-08-25T09:16:27.895 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.034768794, 2023-08-25T09:16:27.896 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:27.896 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1288, 2023-08-25T09:16:27.896 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.057980290, 2023-08-25T09:16:27.896 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000045015 2023-08-25T09:16:27.897 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.897 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:27.897 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.897 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.897 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.898 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.898 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1748816, 2023-08-25T09:16:27.898 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 194848 2023-08-25T09:16:27.898 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.898 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:27.899 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2159, 2023-08-25T09:16:27.899 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2396, 2023-08-25T09:16:27.899 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 141733714, 2023-08-25T09:16:27.899 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 1240491, 2023-08-25T09:16:27.899 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2023-08-25T09:16:27.899 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2023-08-25T09:16:27.900 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.274405324, 2023-08-25T09:16:27.900 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.059100754, 2023-08-25T09:16:27.900 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.124577653, 2023-08-25T09:16:27.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.070197993, 2023-08-25T09:16:27.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:27.901 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2393, 2023-08-25T09:16:27.901 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.105057985, 2023-08-25T09:16:27.901 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000043902 2023-08-25T09:16:27.901 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.902 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:27.902 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.902 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.902 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.903 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-25T09:16:27.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 569664 2023-08-25T09:16:27.903 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.903 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2023-08-25T09:16:27.904 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2023-08-25T09:16:27.904 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2023-08-25T09:16:27.904 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.904 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2023-08-25T09:16:27.904 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-08-25T09:16:27.905 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-08-25T09:16:27.905 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7189, 2023-08-25T09:16:27.905 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.188065867, 2023-08-25T09:16:27.905 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000026160 2023-08-25T09:16:27.905 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:27.906 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.906 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2023-08-25T09:16:27.907 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-08-25T09:16:27.907 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-08-25T09:16:27.907 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.907 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.907 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.908 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:27.908 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.908 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2023-08-25T09:16:27.908 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2023-08-25T09:16:27.908 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2023-08-25T09:16:27.909 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2023-08-25T09:16:27.909 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.909 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.909 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.910 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.910 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2023-08-25T09:16:27.910 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2023-08-25T09:16:27.910 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2023-08-25T09:16:27.910 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2023-08-25T09:16:27.911 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2023-08-25T09:16:27.911 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2023-08-25T09:16:27.911 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2023-08-25T09:16:27.911 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 25252, 2023-08-25T09:16:27.911 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 1729, 2023-08-25T09:16:27.912 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 17760, 2023-08-25T09:16:27.912 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 5763, 2023-08-25T09:16:27.912 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 22782, 2023-08-25T09:16:27.913 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 95, 2023-08-25T09:16:27.913 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2023-08-25T09:16:27.913 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2023-08-25T09:16:27.913 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 6, 2023-08-25T09:16:27.913 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2023-08-25T09:16:27.913 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2023-08-25T09:16:27.914 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2023-08-25T09:16:27.914 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2023-08-25T09:16:27.914 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2023-08-25T09:16:27.914 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2023-08-25T09:16:27.914 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2023-08-25T09:16:27.915 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2023-08-25T09:16:27.915 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2023-08-25T09:16:27.915 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2023-08-25T09:16:27.915 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2023-08-25T09:16:27.915 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2023-08-25T09:16:27.916 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2023-08-25T09:16:27.916 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2023-08-25T09:16:27.916 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2023-08-25T09:16:27.916 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2023-08-25T09:16:27.916 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2023-08-25T09:16:27.917 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2023-08-25T09:16:27.917 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2023-08-25T09:16:27.917 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T09:16:27.917 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2023-08-25T09:16:27.918 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T09:16:27.918 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2023-08-25T09:16:27.918 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2023-08-25T09:16:27.918 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T09:16:27.918 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T09:16:27.919 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2023-08-25T09:16:27.919 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2023-08-25T09:16:27.919 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2023-08-25T09:16:27.919 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2023-08-25T09:16:27.919 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T09:16:27.920 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T09:16:27.920 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T09:16:27.920 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2023-08-25T09:16:27.920 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2023-08-25T09:16:27.920 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2023-08-25T09:16:27.921 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2023-08-25T09:16:27.921 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.921 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2023-08-25T09:16:27.921 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2023-08-25T09:16:27.921 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2023-08-25T09:16:27.922 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2023-08-25T09:16:27.922 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2023-08-25T09:16:27.922 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2023-08-25T09:16:27.922 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2023-08-25T09:16:27.922 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2023-08-25T09:16:27.923 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2023-08-25T09:16:27.923 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2023-08-25T09:16:27.923 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2023-08-25T09:16:27.923 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2023-08-25T09:16:27.923 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2023-08-25T09:16:27.924 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2023-08-25T09:16:27.924 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2023-08-25T09:16:27.924 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2023-08-25T09:16:27.924 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2023-08-25T09:16:27.924 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2023-08-25T09:16:27.925 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T09:16:27.925 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2023-08-25T09:16:27.925 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2023-08-25T09:16:27.925 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2023-08-25T09:16:27.925 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2023-08-25T09:16:27.926 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2023-08-25T09:16:27.926 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.926 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2023-08-25T09:16:27.926 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2023-08-25T09:16:27.926 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2023-08-25T09:16:27.927 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2023-08-25T09:16:27.927 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2023-08-25T09:16:27.927 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 112896, 2023-08-25T09:16:27.927 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2023-08-25T09:16:27.927 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2023-08-25T09:16:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2023-08-25T09:16:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2023-08-25T09:16:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2023-08-25T09:16:27.928 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 141, 2023-08-25T09:16:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 11, 2023-08-25T09:16:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2023-08-25T09:16:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2023-08-25T09:16:27.929 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 22248975, 2023-08-25T09:16:27.930 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 443069529, 2023-08-25T09:16:27.930 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 443063817, 2023-08-25T09:16:27.930 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2023-08-25T09:16:27.930 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.931 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.931 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.931 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.931 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 112896 2023-08-25T09:16:27.932 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.932 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2023-08-25T09:16:27.932 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 1333, 2023-08-25T09:16:27.932 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 25251, 2023-08-25T09:16:27.932 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 23918, 2023-08-25T09:16:27.933 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 83, 2023-08-25T09:16:27.933 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 162, 2023-08-25T09:16:27.933 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 79, 2023-08-25T09:16:27.933 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 25252, 2023-08-25T09:16:27.933 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 25485, 2023-08-25T09:16:27.934 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 233, 2023-08-25T09:16:27.934 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2023-08-25T09:16:27.934 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2023-08-25T09:16:27.934 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2023-08-25T09:16:27.935 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 78940, 2023-08-25T09:16:27.935 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207152 2023-08-25T09:16:27.935 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.935 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2023-08-25T09:16:27.935 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2023-08-25T09:16:27.936 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2023-08-25T09:16:27.936 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2023-08-25T09:16:27.936 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2023-08-25T09:16:27.936 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2023-08-25T09:16:27.937 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2023-08-25T09:16:27.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.937 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.938 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.938 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2023-08-25T09:16:27.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.938 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.939 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.939 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2023-08-25T09:16:27.939 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.939 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.940 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.940 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2023-08-25T09:16:27.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.941 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.941 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.941 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2023-08-25T09:16:27.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.942 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.942 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.942 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.942 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2023-08-25T09:16:27.942 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.943 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.943 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.943 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.943 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2023-08-25T09:16:27.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.944 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.944 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.945 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2023-08-25T09:16:27.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.945 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.946 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.946 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2023-08-25T09:16:27.946 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.946 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.946 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.947 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.947 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2023-08-25T09:16:27.947 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.947 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.947 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.948 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.948 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2023-08-25T09:16:27.948 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.948 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.949 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.949 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2023-08-25T09:16:27.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.949 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.950 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.950 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.950 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2023-08-25T09:16:27.950 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.950 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.950 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2023-08-25T09:16:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2023-08-25T09:16:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2023-08-25T09:16:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2023-08-25T09:16:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.955 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.955 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.955 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2023-08-25T09:16:27.956 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.956 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.956 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.956 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.957 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2023-08-25T09:16:27.957 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.957 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.957 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.957 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.958 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2023-08-25T09:16:27.958 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.958 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.958 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.958 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.959 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2023-08-25T09:16:27.959 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.959 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.959 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.959 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.960 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2023-08-25T09:16:27.960 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.960 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.960 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.961 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.961 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2023-08-25T09:16:27.961 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.961 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.962 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.962 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.962 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2023-08-25T09:16:27.962 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.963 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.963 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.963 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.963 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2023-08-25T09:16:27.963 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.964 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.964 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.964 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.964 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2023-08-25T09:16:27.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.965 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.965 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.965 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2023-08-25T09:16:27.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.966 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.966 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.967 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2023-08-25T09:16:27.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.967 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.968 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.968 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2023-08-25T09:16:27.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.969 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.970 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.970 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2023-08-25T09:16:27.970 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2023-08-25T09:16:27.970 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2023-08-25T09:16:27.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:27.971 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:27.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:27.971 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:27.972 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.972 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2023-08-25T09:16:27.972 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2023-08-25T09:16:27.972 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 2, 2023-08-25T09:16:27.973 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2023-08-25T09:16:27.973 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2023-08-25T09:16:27.973 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2023-08-25T09:16:27.973 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2023-08-25T09:16:27.973 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2023-08-25T09:16:27.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 4591 2023-08-25T09:16:27.974 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.974 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2023-08-25T09:16:27.974 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:27.974 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:27.975 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:27.975 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:27.975 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:27.975 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:27.975 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:27.975 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:27.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:27.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:27.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:27.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:27.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:27.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:27.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:27.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:27.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:27.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:27.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:27.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:27.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:27.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:27.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:27.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:27.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:27.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:27.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:27.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:27.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:27.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:27.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:27.982 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2023-08-25T09:16:27.982 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:27.982 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:27.983 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:27.983 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:27.984 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 291925, 2023-08-25T09:16:27.984 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 1581, 2023-08-25T09:16:27.985 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:27.985 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:27.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2023-08-25T09:16:27.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2023-08-25T09:16:27.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:27.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:27.987 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:27.987 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:27.987 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8416, 2023-08-25T09:16:27.987 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 175, 2023-08-25T09:16:27.988 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:27.988 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:27.988 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:27.988 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2023-08-25T09:16:27.988 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 21191233, 2023-08-25T09:16:27.989 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 148005, 2023-08-25T09:16:27.989 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:27.989 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2023-08-25T09:16:27.989 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:27.989 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2023-08-25T09:16:27.990 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.990 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2023-08-25T09:16:27.990 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2023-08-25T09:16:27.990 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2023-08-25T09:16:27.990 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 4390, 2023-08-25T09:16:27.991 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2023-08-25T09:16:27.991 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2023-08-25T09:16:27.991 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 4390, 2023-08-25T09:16:27.991 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2023-08-25T09:16:27.992 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4390, 2023-08-25T09:16:27.992 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 6.485994220, 2023-08-25T09:16:27.992 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001477447 2023-08-25T09:16:27.992 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.992 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2023-08-25T09:16:27.993 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2023-08-25T09:16:27.993 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4390, 2023-08-25T09:16:27.993 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 4391 2023-08-25T09:16:27.993 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:27.994 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 4390, 2023-08-25T09:16:27.994 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 4390, 2023-08-25T09:16:27.994 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2023-08-25T09:16:27.994 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2023-08-25T09:16:27.994 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2023-08-25T09:16:27.995 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 2547, 2023-08-25T09:16:27.995 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2023-08-25T09:16:27.995 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 1841, 2023-08-25T09:16:27.995 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2023-08-25T09:16:27.995 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2023-08-25T09:16:27.996 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2023-08-25T09:16:27.996 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2023-08-25T09:16:27.996 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2023-08-25T09:16:27.996 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2023-08-25T09:16:27.996 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2023-08-25T09:16:27.997 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2023-08-25T09:16:27.997 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2023-08-25T09:16:27.997 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2023-08-25T09:16:27.998 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2023-08-25T09:16:27.998 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2023-08-25T09:16:27.998 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2023-08-25T09:16:27.998 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2023-08-25T09:16:27.999 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2023-08-25T09:16:27.999 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2023-08-25T09:16:27.999 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2023-08-25T09:16:27.999 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2023-08-25T09:16:27.999 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T09:16:28.000 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2023-08-25T09:16:28.000 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2023-08-25T09:16:28.000 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2023-08-25T09:16:28.000 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2023-08-25T09:16:28.001 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2023-08-25T09:16:28.001 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2023-08-25T09:16:28.001 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2023-08-25T09:16:28.002 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2023-08-25T09:16:28.002 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2023-08-25T09:16:28.003 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2023-08-25T09:16:28.003 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2023-08-25T09:16:28.003 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2023-08-25T09:16:28.004 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2023-08-25T09:16:28.004 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2023-08-25T09:16:28.005 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2023-08-25T09:16:28.005 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2023-08-25T09:16:28.005 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2023-08-25T09:16:28.005 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2023-08-25T09:16:28.006 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2023-08-25T09:16:28.006 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 112, 2023-08-25T09:16:28.006 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2023-08-25T09:16:28.006 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 29, 2023-08-25T09:16:28.007 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 11, 2023-08-25T09:16:28.007 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 11, 2023-08-25T09:16:28.007 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2023-08-25T09:16:28.008 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2023-08-25T09:16:28.008 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2023-08-25T09:16:28.009 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2023-08-25T09:16:28.009 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2023-08-25T09:16:28.009 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:28.009 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2023-08-25T09:16:28.010 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2023-08-25T09:16:28.010 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2023-08-25T09:16:28.010 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2023-08-25T09:16:28.011 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2023-08-25T09:16:28.011 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:28.012 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2023-08-25T09:16:28.012 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2023-08-25T09:16:28.012 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:28.013 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2023-08-25T09:16:28.013 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2023-08-25T09:16:28.014 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2023-08-25T09:16:28.014 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2023-08-25T09:16:28.014 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2023-08-25T09:16:28.015 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2023-08-25T09:16:28.015 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2023-08-25T09:16:28.015 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:28.015 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:28.016 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:28.016 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-08-25T09:16:28.016 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:28.016 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 4782, 2023-08-25T09:16:28.016 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 441409364, 2023-08-25T09:16:28.017 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:28.017 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 4782, 2023-08-25T09:16:28.017 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:28.017 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:28.017 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 4782, 2023-08-25T09:16:28.018 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 441409364, 2023-08-25T09:16:28.018 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:28.018 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:28.018 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:28.019 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:28.019 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:28.019 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:28.019 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2023-08-25T09:16:28.020 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:28.020 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-08-25T09:16:28.020 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:28.020 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:28.020 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:28.021 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:28.021 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:28.021 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 4390, 2023-08-25T09:16:28.021 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 438902135, 2023-08-25T09:16:28.021 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 4390, 2023-08-25T09:16:28.022 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 438902135, 2023-08-25T09:16:28.022 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:28.022 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:28.022 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:28.022 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:28.023 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:28.023 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:28.023 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2023-08-25T09:16:28.023 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:28.023 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2023-08-25T09:16:28.023 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:28.024 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:28.024 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:28.024 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:28.024 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:28.024 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 4390, 2023-08-25T09:16:28.025 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 4390, 2023-08-25T09:16:28.025 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 4390, 2023-08-25T09:16:28.025 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 4390, 2023-08-25T09:16:28.025 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:28.025 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:28.026 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:28.026 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:28.026 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:28.026 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:28.026 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2023-08-25T09:16:28.027 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:28.027 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-08-25T09:16:28.027 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:28.027 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:28.027 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:28.028 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:28.028 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:28.028 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:28.028 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:28.028 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2023-08-25T09:16:28.029 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2023-08-25T09:16:28.029 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:28.029 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:28.029 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:28.029 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:28.029 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:28.030 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:28.030 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55cda2d90960": { 2023-08-25T09:16:28.030 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:28.030 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-08-25T09:16:28.030 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:28.031 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:28.031 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:28.031 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:28.031 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:28.031 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:28.032 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:28.032 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2023-08-25T09:16:28.032 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2023-08-25T09:16:28.032 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:28.032 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:28.033 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:28.033 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:28.033 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:28.033 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:28.033 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-08-25T09:16:28.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:27 smithi181 ceph-mon[131296]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 17 op/s 2023-08-25T09:16:28.311 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-08-25T09:16:28.311 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-08-25T09:16:28.312 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-08-25T09:16:28.312 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-08-25T09:16:28.312 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T09:16:28.312 DEBUG:tasks.check_counter:Getting stats from h 2023-08-25T09:16:28.313 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.h perf dump 2023-08-25T09:16:28.865 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T09:16:29.619 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:29 smithi157 ceph-mon[130879]: pgmap v546: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-08-25T09:16:29.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:29 smithi099 ceph-mon[97641]: pgmap v546: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-08-25T09:16:29.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:29 smithi181 ceph-mon[131296]: pgmap v546: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-08-25T09:16:30.080 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-08-25T09:16:30.080 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:30.080 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 235, 2023-08-25T09:16:30.081 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 230, 2023-08-25T09:16:30.081 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 19558674, 2023-08-25T09:16:30.081 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 76055, 2023-08-25T09:16:30.081 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2023-08-25T09:16:30.082 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2023-08-25T09:16:30.082 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.027836442, 2023-08-25T09:16:30.082 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.006197136, 2023-08-25T09:16:30.082 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.014633379, 2023-08-25T09:16:30.082 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.005983066, 2023-08-25T09:16:30.083 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:30.083 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 229, 2023-08-25T09:16:30.083 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.008285345, 2023-08-25T09:16:30.084 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000036180 2023-08-25T09:16:30.084 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.084 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:30.084 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.084 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.085 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.085 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.085 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 176224, 2023-08-25T09:16:30.085 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 14784 2023-08-25T09:16:30.085 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.086 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:30.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 39, 2023-08-25T09:16:30.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 56, 2023-08-25T09:16:30.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 11887699, 2023-08-25T09:16:30.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 121964, 2023-08-25T09:16:30.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:30.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2023-08-25T09:16:30.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.012006711, 2023-08-25T09:16:30.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.001178224, 2023-08-25T09:16:30.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.009360889, 2023-08-25T09:16:30.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000990058, 2023-08-25T09:16:30.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:30.088 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 54, 2023-08-25T09:16:30.088 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.007713330, 2023-08-25T09:16:30.088 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000142839 2023-08-25T09:16:30.089 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.089 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:30.090 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.090 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.091 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.091 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.091 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 160, 2023-08-25T09:16:30.092 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 110080 2023-08-25T09:16:30.092 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.093 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:30.093 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 88, 2023-08-25T09:16:30.094 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 88, 2023-08-25T09:16:30.094 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 25190804, 2023-08-25T09:16:30.095 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 26579, 2023-08-25T09:16:30.095 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:30.095 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2023-08-25T09:16:30.095 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.019815212, 2023-08-25T09:16:30.095 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.001328175, 2023-08-25T09:16:30.096 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.015891824, 2023-08-25T09:16:30.096 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.002031902, 2023-08-25T09:16:30.096 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:30.096 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 87, 2023-08-25T09:16:30.096 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006918232, 2023-08-25T09:16:30.097 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000079519 2023-08-25T09:16:30.097 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.097 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:30.097 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.097 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.098 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.098 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.098 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-25T09:16:30.098 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2023-08-25T09:16:30.098 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.099 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2023-08-25T09:16:30.099 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2023-08-25T09:16:30.099 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2023-08-25T09:16:30.099 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.099 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2023-08-25T09:16:30.100 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-08-25T09:16:30.100 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-08-25T09:16:30.100 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 515, 2023-08-25T09:16:30.100 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.038621882, 2023-08-25T09:16:30.100 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000074993 2023-08-25T09:16:30.101 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.101 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.101 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2023-08-25T09:16:30.101 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-08-25T09:16:30.101 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-08-25T09:16:30.102 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.102 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.102 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.102 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.102 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.103 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2023-08-25T09:16:30.103 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2023-08-25T09:16:30.103 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2023-08-25T09:16:30.103 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2023-08-25T09:16:30.103 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.104 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.104 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.104 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.104 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2023-08-25T09:16:30.105 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2023-08-25T09:16:30.105 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2023-08-25T09:16:30.105 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2023-08-25T09:16:30.105 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2023-08-25T09:16:30.105 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2023-08-25T09:16:30.106 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2023-08-25T09:16:30.106 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 4220, 2023-08-25T09:16:30.106 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 2592, 2023-08-25T09:16:30.106 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 1625, 2023-08-25T09:16:30.106 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 3, 2023-08-25T09:16:30.106 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 516, 2023-08-25T09:16:30.107 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 52, 2023-08-25T09:16:30.107 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2023-08-25T09:16:30.107 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2023-08-25T09:16:30.107 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 22, 2023-08-25T09:16:30.107 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2023-08-25T09:16:30.108 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2023-08-25T09:16:30.108 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2023-08-25T09:16:30.108 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2023-08-25T09:16:30.109 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2023-08-25T09:16:30.109 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2023-08-25T09:16:30.110 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2023-08-25T09:16:30.110 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2023-08-25T09:16:30.110 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2023-08-25T09:16:30.111 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2023-08-25T09:16:30.111 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2023-08-25T09:16:30.112 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2023-08-25T09:16:30.112 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2023-08-25T09:16:30.113 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2023-08-25T09:16:30.113 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2023-08-25T09:16:30.113 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2023-08-25T09:16:30.114 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 744, 2023-08-25T09:16:30.114 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 215161046, 2023-08-25T09:16:30.115 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2023-08-25T09:16:30.115 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T09:16:30.115 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2023-08-25T09:16:30.116 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T09:16:30.116 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2023-08-25T09:16:30.117 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2023-08-25T09:16:30.117 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T09:16:30.118 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T09:16:30.118 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2023-08-25T09:16:30.118 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2023-08-25T09:16:30.119 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2023-08-25T09:16:30.119 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2023-08-25T09:16:30.120 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T09:16:30.120 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T09:16:30.121 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T09:16:30.121 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2023-08-25T09:16:30.122 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2023-08-25T09:16:30.122 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2023-08-25T09:16:30.122 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2023-08-25T09:16:30.123 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.123 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2023-08-25T09:16:30.124 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2023-08-25T09:16:30.124 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2023-08-25T09:16:30.125 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2023-08-25T09:16:30.125 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2023-08-25T09:16:30.125 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2023-08-25T09:16:30.126 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2023-08-25T09:16:30.126 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2023-08-25T09:16:30.126 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2023-08-25T09:16:30.127 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2023-08-25T09:16:30.127 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2023-08-25T09:16:30.128 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2023-08-25T09:16:30.128 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2023-08-25T09:16:30.129 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2023-08-25T09:16:30.129 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2023-08-25T09:16:30.130 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2023-08-25T09:16:30.130 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2023-08-25T09:16:30.130 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2023-08-25T09:16:30.131 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T09:16:30.131 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2023-08-25T09:16:30.131 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2023-08-25T09:16:30.132 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2023-08-25T09:16:30.132 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2023-08-25T09:16:30.133 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2023-08-25T09:16:30.133 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.133 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2023-08-25T09:16:30.134 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2023-08-25T09:16:30.134 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2023-08-25T09:16:30.135 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2023-08-25T09:16:30.135 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2023-08-25T09:16:30.136 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 16987, 2023-08-25T09:16:30.136 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2023-08-25T09:16:30.137 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2023-08-25T09:16:30.137 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2023-08-25T09:16:30.137 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2023-08-25T09:16:30.137 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2023-08-25T09:16:30.137 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 3, 2023-08-25T09:16:30.138 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 5, 2023-08-25T09:16:30.138 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2023-08-25T09:16:30.138 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2023-08-25T09:16:30.138 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 519793639, 2023-08-25T09:16:30.138 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 522638994, 2023-08-25T09:16:30.139 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 522631112, 2023-08-25T09:16:30.139 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2023-08-25T09:16:30.139 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.139 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.140 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.140 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.140 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 16987 2023-08-25T09:16:30.140 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.141 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2023-08-25T09:16:30.141 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 451, 2023-08-25T09:16:30.141 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 4054, 2023-08-25T09:16:30.141 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 3603, 2023-08-25T09:16:30.141 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 97, 2023-08-25T09:16:30.141 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 119, 2023-08-25T09:16:30.142 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 22, 2023-08-25T09:16:30.142 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 4220, 2023-08-25T09:16:30.142 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 4272, 2023-08-25T09:16:30.142 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 52, 2023-08-25T09:16:30.142 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2023-08-25T09:16:30.143 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2023-08-25T09:16:30.143 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2023-08-25T09:16:30.143 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 55584, 2023-08-25T09:16:30.143 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207152 2023-08-25T09:16:30.143 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.144 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2023-08-25T09:16:30.144 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2023-08-25T09:16:30.144 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2023-08-25T09:16:30.144 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2023-08-25T09:16:30.144 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2023-08-25T09:16:30.145 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2023-08-25T09:16:30.145 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2023-08-25T09:16:30.145 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.145 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.145 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.146 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.146 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2023-08-25T09:16:30.146 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.146 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.147 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.147 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.147 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2023-08-25T09:16:30.147 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.147 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.148 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.148 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.148 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2023-08-25T09:16:30.149 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.149 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.149 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.149 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.150 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2023-08-25T09:16:30.150 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.150 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.150 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.151 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.151 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2023-08-25T09:16:30.151 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.151 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.151 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.152 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.152 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2023-08-25T09:16:30.153 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.153 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.154 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.154 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.154 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2023-08-25T09:16:30.155 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.155 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.156 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.156 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.157 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2023-08-25T09:16:30.157 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.158 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.158 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.158 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.158 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2023-08-25T09:16:30.159 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.159 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.159 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.159 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.159 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2023-08-25T09:16:30.160 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.160 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.160 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.160 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.161 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2023-08-25T09:16:30.161 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.161 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.161 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.161 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.162 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2023-08-25T09:16:30.162 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.162 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.162 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.162 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.163 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2023-08-25T09:16:30.163 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.163 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.163 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.164 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.164 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2023-08-25T09:16:30.164 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.164 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.165 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.165 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.165 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2023-08-25T09:16:30.166 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.166 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.166 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.166 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.166 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2023-08-25T09:16:30.167 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.167 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.167 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.167 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.167 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2023-08-25T09:16:30.168 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.168 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.168 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.168 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.169 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2023-08-25T09:16:30.169 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.169 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.169 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.170 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.170 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2023-08-25T09:16:30.170 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.170 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.170 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.171 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.171 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2023-08-25T09:16:30.171 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.171 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.171 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.172 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.172 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2023-08-25T09:16:30.172 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.172 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.172 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.173 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.173 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2023-08-25T09:16:30.173 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.173 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.173 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.174 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.174 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2023-08-25T09:16:30.174 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.174 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.175 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.175 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.175 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2023-08-25T09:16:30.175 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.176 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.176 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.176 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.176 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2023-08-25T09:16:30.177 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.177 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.177 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.177 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.177 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2023-08-25T09:16:30.178 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.178 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.178 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.178 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.179 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2023-08-25T09:16:30.179 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.179 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.179 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.179 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.180 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2023-08-25T09:16:30.180 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.180 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.180 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.180 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.180 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2023-08-25T09:16:30.181 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2023-08-25T09:16:30.181 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2023-08-25T09:16:30.181 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.181 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.181 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.182 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.182 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.182 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2023-08-25T09:16:30.182 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2023-08-25T09:16:30.182 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 2, 2023-08-25T09:16:30.183 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 0, 2023-08-25T09:16:30.183 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 0, 2023-08-25T09:16:30.183 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2023-08-25T09:16:30.183 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2023-08-25T09:16:30.183 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2023-08-25T09:16:30.184 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2812 2023-08-25T09:16:30.184 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.184 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2023-08-25T09:16:30.184 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:30.184 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:30.185 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:30.185 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:30.185 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:30.185 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:30.185 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:30.186 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:30.186 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:30.186 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:30.186 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:30.186 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:30.187 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:30.187 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:30.187 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:30.187 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:30.187 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:30.188 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:30.188 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:30.188 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:30.188 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:30.188 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:30.189 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:30.189 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:30.189 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:30.189 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:30.189 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:30.190 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:30.190 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:30.190 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:30.190 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:30.190 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2023-08-25T09:16:30.191 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:30.191 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:30.191 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:30.191 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:30.191 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 250712, 2023-08-25T09:16:30.192 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 679, 2023-08-25T09:16:30.192 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:30.192 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:30.192 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2023-08-25T09:16:30.192 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2023-08-25T09:16:30.193 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:30.193 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:30.193 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:30.193 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:30.193 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8416, 2023-08-25T09:16:30.193 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 175, 2023-08-25T09:16:30.194 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:30.194 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:30.194 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:30.194 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2023-08-25T09:16:30.194 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 4316278, 2023-08-25T09:16:30.195 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 37396, 2023-08-25T09:16:30.195 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:30.195 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2023-08-25T09:16:30.195 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:30.195 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2023-08-25T09:16:30.196 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.196 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2023-08-25T09:16:30.196 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2023-08-25T09:16:30.196 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2023-08-25T09:16:30.196 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 319, 2023-08-25T09:16:30.197 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 26, 2023-08-25T09:16:30.197 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 4, 2023-08-25T09:16:30.197 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 319, 2023-08-25T09:16:30.197 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2023-08-25T09:16:30.197 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 319, 2023-08-25T09:16:30.197 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.531999608, 2023-08-25T09:16:30.198 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001667710 2023-08-25T09:16:30.198 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.198 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2023-08-25T09:16:30.198 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2023-08-25T09:16:30.198 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 319, 2023-08-25T09:16:30.199 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 320 2023-08-25T09:16:30.199 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.199 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 319, 2023-08-25T09:16:30.199 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 319, 2023-08-25T09:16:30.199 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2023-08-25T09:16:30.200 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2023-08-25T09:16:30.200 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2023-08-25T09:16:30.200 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 172, 2023-08-25T09:16:30.200 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2023-08-25T09:16:30.200 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 144, 2023-08-25T09:16:30.201 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2023-08-25T09:16:30.201 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2023-08-25T09:16:30.201 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2023-08-25T09:16:30.201 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2023-08-25T09:16:30.201 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2023-08-25T09:16:30.201 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2023-08-25T09:16:30.202 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2023-08-25T09:16:30.202 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2023-08-25T09:16:30.202 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2023-08-25T09:16:30.202 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2023-08-25T09:16:30.202 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 2, 2023-08-25T09:16:30.203 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2023-08-25T09:16:30.203 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2023-08-25T09:16:30.203 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2023-08-25T09:16:30.203 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2023-08-25T09:16:30.203 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2023-08-25T09:16:30.204 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2023-08-25T09:16:30.204 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2023-08-25T09:16:30.204 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T09:16:30.204 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2023-08-25T09:16:30.204 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2023-08-25T09:16:30.205 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2023-08-25T09:16:30.205 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2023-08-25T09:16:30.205 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2023-08-25T09:16:30.205 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2023-08-25T09:16:30.205 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2023-08-25T09:16:30.206 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2023-08-25T09:16:30.206 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2023-08-25T09:16:30.206 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2023-08-25T09:16:30.206 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2023-08-25T09:16:30.206 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2023-08-25T09:16:30.207 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2023-08-25T09:16:30.207 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2023-08-25T09:16:30.207 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2023-08-25T09:16:30.207 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2023-08-25T09:16:30.207 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2023-08-25T09:16:30.207 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2023-08-25T09:16:30.208 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2023-08-25T09:16:30.208 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 112, 2023-08-25T09:16:30.208 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2023-08-25T09:16:30.208 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 2, 2023-08-25T09:16:30.208 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 10, 2023-08-25T09:16:30.209 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 10, 2023-08-25T09:16:30.209 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2023-08-25T09:16:30.209 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2023-08-25T09:16:30.209 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2023-08-25T09:16:30.209 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2023-08-25T09:16:30.210 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2023-08-25T09:16:30.210 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.210 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2023-08-25T09:16:30.210 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2023-08-25T09:16:30.210 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2023-08-25T09:16:30.211 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2023-08-25T09:16:30.211 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2023-08-25T09:16:30.211 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.211 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2023-08-25T09:16:30.211 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2023-08-25T09:16:30.212 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.212 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2023-08-25T09:16:30.212 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2023-08-25T09:16:30.212 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2023-08-25T09:16:30.212 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2023-08-25T09:16:30.213 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2023-08-25T09:16:30.213 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2023-08-25T09:16:30.213 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2023-08-25T09:16:30.213 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.213 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:30.214 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:30.214 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-08-25T09:16:30.214 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:30.214 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 358, 2023-08-25T09:16:30.214 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 56600128, 2023-08-25T09:16:30.215 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:30.215 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 358, 2023-08-25T09:16:30.215 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:30.215 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:30.215 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 358, 2023-08-25T09:16:30.216 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 56600128, 2023-08-25T09:16:30.216 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:30.216 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.216 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.216 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.217 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.217 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.217 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2023-08-25T09:16:30.217 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:30.217 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-08-25T09:16:30.217 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:30.218 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:30.218 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:30.218 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:30.218 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:30.218 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 319, 2023-08-25T09:16:30.219 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 56364741, 2023-08-25T09:16:30.219 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 319, 2023-08-25T09:16:30.219 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 56364741, 2023-08-25T09:16:30.219 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:30.219 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.220 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.220 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.220 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.220 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.220 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2023-08-25T09:16:30.221 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:30.221 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2023-08-25T09:16:30.221 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:30.221 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:30.221 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:30.222 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:30.222 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:30.222 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 319, 2023-08-25T09:16:30.222 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 319, 2023-08-25T09:16:30.222 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 319, 2023-08-25T09:16:30.223 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 319, 2023-08-25T09:16:30.223 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:30.223 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.223 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.224 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.224 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.224 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.224 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2023-08-25T09:16:30.224 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:30.225 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-08-25T09:16:30.225 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:30.225 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:30.225 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:30.225 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:30.225 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:30.226 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:30.226 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:30.226 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2023-08-25T09:16:30.226 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2023-08-25T09:16:30.226 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:30.227 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.227 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.227 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.227 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.227 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:30.228 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x559b21830960": { 2023-08-25T09:16:30.228 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:30.228 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-08-25T09:16:30.228 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:30.229 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-08-25T09:16:30.229 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-08-25T09:16:30.229 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:30.229 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:30.229 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:30.230 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:30.230 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2023-08-25T09:16:30.230 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2023-08-25T09:16:30.230 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:30.230 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:30.230 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:30.231 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:30.231 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.231 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:30.231 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-08-25T09:16:30.410 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-08-25T09:16:30.410 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-08-25T09:16:30.410 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-08-25T09:16:30.410 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-08-25T09:16:30.411 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T09:16:30.411 DEBUG:tasks.check_counter:Getting stats from k 2023-08-25T09:16:30.411 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.k perf dump 2023-08-25T09:16:30.967 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T09:16:31.789 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:31 smithi157 ceph-mon[130879]: pgmap v547: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-08-25T09:16:31.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:31 smithi099 ceph-mon[97641]: pgmap v547: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-08-25T09:16:31.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:31 smithi181 ceph-mon[131296]: pgmap v547: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-08-25T09:16:32.186 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-08-25T09:16:32.187 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:32.187 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 327, 2023-08-25T09:16:32.187 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 316, 2023-08-25T09:16:32.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1575904, 2023-08-25T09:16:32.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 190336, 2023-08-25T09:16:32.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2023-08-25T09:16:32.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 0, 2023-08-25T09:16:32.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.056007180, 2023-08-25T09:16:32.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.017780568, 2023-08-25T09:16:32.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.036228590, 2023-08-25T09:16:32.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.008437629, 2023-08-25T09:16:32.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:32.190 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 315, 2023-08-25T09:16:32.191 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.014041591, 2023-08-25T09:16:32.191 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000044576 2023-08-25T09:16:32.191 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.191 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:32.191 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:32.192 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:32.192 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:32.192 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.192 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1575904, 2023-08-25T09:16:32.192 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 190336 2023-08-25T09:16:32.193 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.193 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:32.193 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 3, 2023-08-25T09:16:32.193 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 242, 2023-08-25T09:16:32.193 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 480, 2023-08-25T09:16:32.193 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 271312, 2023-08-25T09:16:32.194 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:32.194 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2023-08-25T09:16:32.194 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.013643954, 2023-08-25T09:16:32.194 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.009092647, 2023-08-25T09:16:32.194 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.001776939, 2023-08-25T09:16:32.195 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:32.195 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:32.195 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 239, 2023-08-25T09:16:32.195 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.018697751, 2023-08-25T09:16:32.195 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000078233 2023-08-25T09:16:32.196 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.196 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:32.196 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:32.196 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:32.196 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:32.197 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.197 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T09:16:32.197 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 271312 2023-08-25T09:16:32.197 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.197 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:32.198 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4, 2023-08-25T09:16:32.198 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 4, 2023-08-25T09:16:32.198 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2944, 2023-08-25T09:16:32.198 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 448, 2023-08-25T09:16:32.198 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 2, 2023-08-25T09:16:32.199 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2023-08-25T09:16:32.199 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.000668068, 2023-08-25T09:16:32.199 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.000045192, 2023-08-25T09:16:32.199 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.000623091, 2023-08-25T09:16:32.199 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:32.200 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:32.200 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3, 2023-08-25T09:16:32.200 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000699249, 2023-08-25T09:16:32.200 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000233083 2023-08-25T09:16:32.200 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.201 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:32.201 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:32.201 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:32.201 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:32.201 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.201 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-25T09:16:32.202 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2023-08-25T09:16:32.202 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.202 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2023-08-25T09:16:32.202 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 0, 2023-08-25T09:16:32.202 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2023-08-25T09:16:32.203 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.203 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2023-08-25T09:16:32.203 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:32.203 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:32.203 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:32.204 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:32.204 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:32.204 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:32.204 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:32.204 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:32.205 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:32.205 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:32.205 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:32.205 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:32.205 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:32.206 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:32.206 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:32.206 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:32.206 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:32.206 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:32.207 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:32.207 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:32.207 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:32.207 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:32.207 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:32.208 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:32.208 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:32.208 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:32.208 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:32.208 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:32.209 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:32.209 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:32.209 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:32.209 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2023-08-25T09:16:32.209 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:32.210 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:32.210 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:32.210 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:32.210 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 148714, 2023-08-25T09:16:32.210 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 30, 2023-08-25T09:16:32.211 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:32.211 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:32.211 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2023-08-25T09:16:32.211 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2023-08-25T09:16:32.211 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:32.212 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:32.212 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:32.212 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:32.212 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 0, 2023-08-25T09:16:32.212 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 0, 2023-08-25T09:16:32.213 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:32.213 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:32.213 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:32.213 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2023-08-25T09:16:32.213 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 928, 2023-08-25T09:16:32.214 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2, 2023-08-25T09:16:32.214 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:32.214 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2023-08-25T09:16:32.214 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:32.214 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2023-08-25T09:16:32.215 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-08-25T09:16:32.215 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:32.215 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-08-25T09:16:32.215 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-08-25T09:16:32.215 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-08-25T09:16:32.216 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 330, 2023-08-25T09:16:32.216 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1529767, 2023-08-25T09:16:32.216 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:32.216 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 330, 2023-08-25T09:16:32.216 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-08-25T09:16:32.217 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-08-25T09:16:32.217 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 330, 2023-08-25T09:16:32.217 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1529767, 2023-08-25T09:16:32.217 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-08-25T09:16:32.217 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-08-25T09:16:32.218 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-08-25T09:16:32.218 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-08-25T09:16:32.218 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:32.218 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-08-25T09:16:32.218 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-08-25T09:16:32.522 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-08-25T09:16:32.523 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-08-25T09:16:32.523 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-08-25T09:16:32.523 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-08-25T09:16:32.523 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-08-25T09:16:32.523 DEBUG:tasks.check_counter:Getting stats from c 2023-08-25T09:16:32.524 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.c perf dump 2023-08-25T09:16:33.180 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T09:16:33.703 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:33 smithi181 ceph-mon[131296]: pgmap v548: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.5 MiB/s wr, 35 op/s 2023-08-25T09:16:33.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:33 smithi099 ceph-mon[97641]: pgmap v548: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.5 MiB/s wr, 35 op/s 2023-08-25T09:16:33.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:33 smithi157 ceph-mon[130879]: pgmap v548: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.5 MiB/s wr, 35 op/s 2023-08-25T09:16:34.636 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-08-25T09:16:34.637 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:34.637 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2750, 2023-08-25T09:16:34.637 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2750, 2023-08-25T09:16:34.637 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 181155689, 2023-08-25T09:16:34.637 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 854461, 2023-08-25T09:16:34.638 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 8, 2023-08-25T09:16:34.638 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2023-08-25T09:16:34.638 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.312372288, 2023-08-25T09:16:34.638 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.067279774, 2023-08-25T09:16:34.639 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.140171258, 2023-08-25T09:16:34.639 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.084104181, 2023-08-25T09:16:34.639 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:34.639 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2749, 2023-08-25T09:16:34.639 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.148588144, 2023-08-25T09:16:34.640 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000054051 2023-08-25T09:16:34.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.640 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:34.640 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.640 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.641 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.641 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-25T09:16:34.641 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2023-08-25T09:16:34.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.642 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:34.642 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1390, 2023-08-25T09:16:34.642 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1344, 2023-08-25T09:16:34.642 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 121503356, 2023-08-25T09:16:34.643 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 504529, 2023-08-25T09:16:34.643 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:34.643 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2023-08-25T09:16:34.643 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.200325503, 2023-08-25T09:16:34.643 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.035811083, 2023-08-25T09:16:34.644 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.129389978, 2023-08-25T09:16:34.644 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.035787562, 2023-08-25T09:16:34.644 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:34.644 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1343, 2023-08-25T09:16:34.644 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.062251569, 2023-08-25T09:16:34.645 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000046352 2023-08-25T09:16:34.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.645 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:34.645 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.645 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.646 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.646 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1755952, 2023-08-25T09:16:34.646 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 194400 2023-08-25T09:16:34.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.647 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:34.647 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 733, 2023-08-25T09:16:34.647 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 970, 2023-08-25T09:16:34.647 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 172154028, 2023-08-25T09:16:34.648 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 798573, 2023-08-25T09:16:34.648 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 9, 2023-08-25T09:16:34.648 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2023-08-25T09:16:34.648 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.158401874, 2023-08-25T09:16:34.648 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.021226706, 2023-08-25T09:16:34.649 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.109516725, 2023-08-25T09:16:34.649 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.020344387, 2023-08-25T09:16:34.649 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:34.649 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 967, 2023-08-25T09:16:34.649 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.092501047, 2023-08-25T09:16:34.650 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000095657 2023-08-25T09:16:34.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.650 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:34.650 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.650 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.651 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.651 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-25T09:16:34.651 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 571232 2023-08-25T09:16:34.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.652 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2023-08-25T09:16:34.652 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2023-08-25T09:16:34.652 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2023-08-25T09:16:34.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.652 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2023-08-25T09:16:34.653 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-08-25T09:16:34.653 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-08-25T09:16:34.653 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 7237, 2023-08-25T09:16:34.653 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.207160256, 2023-08-25T09:16:34.653 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000028625 2023-08-25T09:16:34.654 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.654 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2023-08-25T09:16:34.654 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-08-25T09:16:34.654 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-08-25T09:16:34.655 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.655 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.655 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.655 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.656 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2023-08-25T09:16:34.656 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2023-08-25T09:16:34.656 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2023-08-25T09:16:34.657 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2023-08-25T09:16:34.657 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.658 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.658 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.659 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2023-08-25T09:16:34.659 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2023-08-25T09:16:34.660 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2023-08-25T09:16:34.660 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2023-08-25T09:16:34.661 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2023-08-25T09:16:34.661 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2023-08-25T09:16:34.662 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2023-08-25T09:16:34.662 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 21248, 2023-08-25T09:16:34.663 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 184, 2023-08-25T09:16:34.663 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 20008, 2023-08-25T09:16:34.664 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 1056, 2023-08-25T09:16:34.664 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 20985, 2023-08-25T09:16:34.665 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 1381, 2023-08-25T09:16:34.665 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2023-08-25T09:16:34.665 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2023-08-25T09:16:34.666 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 18, 2023-08-25T09:16:34.666 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2023-08-25T09:16:34.666 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2023-08-25T09:16:34.667 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2023-08-25T09:16:34.667 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2023-08-25T09:16:34.668 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2023-08-25T09:16:34.668 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2023-08-25T09:16:34.668 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2023-08-25T09:16:34.669 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2023-08-25T09:16:34.669 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2023-08-25T09:16:34.669 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2023-08-25T09:16:34.669 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2023-08-25T09:16:34.669 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2023-08-25T09:16:34.670 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2023-08-25T09:16:34.670 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2023-08-25T09:16:34.670 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2023-08-25T09:16:34.670 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2023-08-25T09:16:34.671 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2023-08-25T09:16:34.671 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2023-08-25T09:16:34.671 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2023-08-25T09:16:34.671 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T09:16:34.671 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2023-08-25T09:16:34.672 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T09:16:34.672 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2023-08-25T09:16:34.672 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2023-08-25T09:16:34.672 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T09:16:34.673 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T09:16:34.673 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2023-08-25T09:16:34.673 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2023-08-25T09:16:34.673 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2023-08-25T09:16:34.673 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2023-08-25T09:16:34.674 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T09:16:34.674 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T09:16:34.674 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T09:16:34.674 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2023-08-25T09:16:34.674 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2023-08-25T09:16:34.675 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2023-08-25T09:16:34.675 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2023-08-25T09:16:34.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.675 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2023-08-25T09:16:34.676 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2023-08-25T09:16:34.676 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2023-08-25T09:16:34.676 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2023-08-25T09:16:34.676 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2023-08-25T09:16:34.676 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2023-08-25T09:16:34.677 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2023-08-25T09:16:34.677 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2023-08-25T09:16:34.677 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2023-08-25T09:16:34.677 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2023-08-25T09:16:34.677 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2023-08-25T09:16:34.678 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2023-08-25T09:16:34.678 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2023-08-25T09:16:34.678 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2023-08-25T09:16:34.678 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2023-08-25T09:16:34.679 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2023-08-25T09:16:34.679 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2023-08-25T09:16:34.679 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2023-08-25T09:16:34.679 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T09:16:34.679 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2023-08-25T09:16:34.680 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2023-08-25T09:16:34.680 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2023-08-25T09:16:34.680 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2023-08-25T09:16:34.680 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2023-08-25T09:16:34.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.681 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2023-08-25T09:16:34.681 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2023-08-25T09:16:34.681 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2023-08-25T09:16:34.681 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2023-08-25T09:16:34.682 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2023-08-25T09:16:34.682 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 117250, 2023-08-25T09:16:34.682 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2023-08-25T09:16:34.682 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2023-08-25T09:16:34.682 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2023-08-25T09:16:34.683 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2023-08-25T09:16:34.683 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2023-08-25T09:16:34.683 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 139, 2023-08-25T09:16:34.683 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 11, 2023-08-25T09:16:34.683 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2023-08-25T09:16:34.684 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2023-08-25T09:16:34.684 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 35246797, 2023-08-25T09:16:34.684 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 476009406, 2023-08-25T09:16:34.684 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 476003163, 2023-08-25T09:16:34.685 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2023-08-25T09:16:34.685 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.685 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.686 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.686 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 117250 2023-08-25T09:16:34.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.687 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2023-08-25T09:16:34.687 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1569, 2023-08-25T09:16:34.687 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 22783, 2023-08-25T09:16:34.688 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 21214, 2023-08-25T09:16:34.688 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 122, 2023-08-25T09:16:34.688 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 225, 2023-08-25T09:16:34.689 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 103, 2023-08-25T09:16:34.689 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 21248, 2023-08-25T09:16:34.690 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 23205, 2023-08-25T09:16:34.690 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 1957, 2023-08-25T09:16:34.690 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2023-08-25T09:16:34.691 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2023-08-25T09:16:34.691 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2023-08-25T09:16:34.692 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 77960, 2023-08-25T09:16:34.692 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 207152 2023-08-25T09:16:34.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.693 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2023-08-25T09:16:34.693 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2023-08-25T09:16:34.693 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2023-08-25T09:16:34.693 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2023-08-25T09:16:34.694 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2023-08-25T09:16:34.694 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2023-08-25T09:16:34.694 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2023-08-25T09:16:34.694 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.694 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.695 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.695 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2023-08-25T09:16:34.695 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.696 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.696 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.697 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2023-08-25T09:16:34.697 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.698 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.698 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.699 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.699 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2023-08-25T09:16:34.699 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.699 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.700 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.700 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.700 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2023-08-25T09:16:34.700 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.700 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.701 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.701 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.701 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2023-08-25T09:16:34.701 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.702 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.702 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.702 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.702 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2023-08-25T09:16:34.702 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.703 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.703 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.703 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.703 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2023-08-25T09:16:34.703 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.704 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.704 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.704 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.704 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2023-08-25T09:16:34.705 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.705 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.705 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.705 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.706 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2023-08-25T09:16:34.706 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.706 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.706 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.707 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.707 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2023-08-25T09:16:34.707 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.707 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.707 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.708 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.708 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2023-08-25T09:16:34.708 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.708 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.709 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.709 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.709 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2023-08-25T09:16:34.709 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2023-08-25T09:16:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2023-08-25T09:16:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2023-08-25T09:16:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2023-08-25T09:16:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2023-08-25T09:16:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2023-08-25T09:16:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2023-08-25T09:16:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2023-08-25T09:16:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2023-08-25T09:16:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2023-08-25T09:16:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2023-08-25T09:16:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2023-08-25T09:16:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2023-08-25T09:16:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2023-08-25T09:16:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2023-08-25T09:16:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2023-08-25T09:16:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2023-08-25T09:16:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2023-08-25T09:16:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2023-08-25T09:16:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2023-08-25T09:16:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 2, 2023-08-25T09:16:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 4, 2023-08-25T09:16:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 2, 2023-08-25T09:16:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 3, 2023-08-25T09:16:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2023-08-25T09:16:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2023-08-25T09:16:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2023-08-25T09:16:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 3756 2023-08-25T09:16:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2023-08-25T09:16:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2023-08-25T09:16:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 303881, 2023-08-25T09:16:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 1932, 2023-08-25T09:16:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2023-08-25T09:16:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2023-08-25T09:16:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8416, 2023-08-25T09:16:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 175, 2023-08-25T09:16:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2023-08-25T09:16:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 19054008, 2023-08-25T09:16:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 135480, 2023-08-25T09:16:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2023-08-25T09:16:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2023-08-25T09:16:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2023-08-25T09:16:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2023-08-25T09:16:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2023-08-25T09:16:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 4474, 2023-08-25T09:16:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 21, 2023-08-25T09:16:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 3, 2023-08-25T09:16:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 4474, 2023-08-25T09:16:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2023-08-25T09:16:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4474, 2023-08-25T09:16:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 6.925865528, 2023-08-25T09:16:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001548025 2023-08-25T09:16:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2023-08-25T09:16:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2023-08-25T09:16:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4474, 2023-08-25T09:16:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 4475 2023-08-25T09:16:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 4474, 2023-08-25T09:16:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 4474, 2023-08-25T09:16:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2023-08-25T09:16:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2023-08-25T09:16:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2023-08-25T09:16:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 2630, 2023-08-25T09:16:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2023-08-25T09:16:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 1842, 2023-08-25T09:16:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2023-08-25T09:16:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2023-08-25T09:16:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2023-08-25T09:16:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2023-08-25T09:16:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2023-08-25T09:16:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2023-08-25T09:16:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2023-08-25T09:16:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2023-08-25T09:16:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2023-08-25T09:16:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2023-08-25T09:16:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 1, 2023-08-25T09:16:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2023-08-25T09:16:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2023-08-25T09:16:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2023-08-25T09:16:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2023-08-25T09:16:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2023-08-25T09:16:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2023-08-25T09:16:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2023-08-25T09:16:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T09:16:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2023-08-25T09:16:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2023-08-25T09:16:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2023-08-25T09:16:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2023-08-25T09:16:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2023-08-25T09:16:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2023-08-25T09:16:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2023-08-25T09:16:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2023-08-25T09:16:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2023-08-25T09:16:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2023-08-25T09:16:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2023-08-25T09:16:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2023-08-25T09:16:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2023-08-25T09:16:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2023-08-25T09:16:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2023-08-25T09:16:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2023-08-25T09:16:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2023-08-25T09:16:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2023-08-25T09:16:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2023-08-25T09:16:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 112, 2023-08-25T09:16:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2023-08-25T09:16:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 29, 2023-08-25T09:16:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 11, 2023-08-25T09:16:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2023-08-25T09:16:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 1, 2023-08-25T09:16:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2023-08-25T09:16:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2023-08-25T09:16:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2023-08-25T09:16:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2023-08-25T09:16:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2023-08-25T09:16:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2023-08-25T09:16:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2023-08-25T09:16:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2023-08-25T09:16:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2023-08-25T09:16:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2023-08-25T09:16:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2023-08-25T09:16:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2023-08-25T09:16:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2023-08-25T09:16:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2023-08-25T09:16:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2023-08-25T09:16:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2023-08-25T09:16:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2023-08-25T09:16:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2023-08-25T09:16:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-08-25T09:16:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 4869, 2023-08-25T09:16:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 474352151, 2023-08-25T09:16:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 4869, 2023-08-25T09:16:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4869, 2023-08-25T09:16:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 474352151, 2023-08-25T09:16:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2023-08-25T09:16:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-08-25T09:16:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-08-25T09:16:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-08-25T09:16:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 4474, 2023-08-25T09:16:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 471820721, 2023-08-25T09:16:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4474, 2023-08-25T09:16:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 471820721, 2023-08-25T09:16:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2023-08-25T09:16:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2023-08-25T09:16:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-08-25T09:16:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-08-25T09:16:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 4474, 2023-08-25T09:16:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 4474, 2023-08-25T09:16:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4474, 2023-08-25T09:16:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 4474, 2023-08-25T09:16:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2023-08-25T09:16:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-08-25T09:16:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-08-25T09:16:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-08-25T09:16:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2023-08-25T09:16:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2023-08-25T09:16:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x562158014960": { 2023-08-25T09:16:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-08-25T09:16:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-08-25T09:16:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-08-25T09:16:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2023-08-25T09:16:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2023-08-25T09:16:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.790 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:34.790 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-08-25T09:16:35.035 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-08-25T09:16:35.036 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-08-25T09:16:35.036 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2023-08-25T09:16:35.037 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-08-25T09:16:35.037 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T09:16:35.037 DEBUG:tasks.check_counter:Getting stats from f 2023-08-25T09:16:35.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.f perf dump 2023-08-25T09:16:35.594 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T09:16:35.630 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:35 smithi157 ceph-mon[130879]: pgmap v549: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.5 MiB/s wr, 28 op/s 2023-08-25T09:16:35.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:35 smithi181 ceph-mon[131296]: pgmap v549: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.5 MiB/s wr, 28 op/s 2023-08-25T09:16:35.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:35 smithi099 ceph-mon[97641]: pgmap v549: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.5 MiB/s wr, 28 op/s 2023-08-25T09:16:36.835 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-08-25T09:16:36.836 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:36.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 229992, 2023-08-25T09:16:36.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 364619, 2023-08-25T09:16:36.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 77353568, 2023-08-25T09:16:36.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 436944997, 2023-08-25T09:16:36.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 15, 2023-08-25T09:16:36.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2023-08-25T09:16:36.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 13.498035574, 2023-08-25T09:16:36.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 5.821746471, 2023-08-25T09:16:36.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 8.832653401, 2023-08-25T09:16:36.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.815861179, 2023-08-25T09:16:36.839 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:36.839 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 364616, 2023-08-25T09:16:36.839 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 39.023442766, 2023-08-25T09:16:36.840 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000107026 2023-08-25T09:16:36.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.840 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:36.840 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 459061, 2023-08-25T09:16:36.841 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.089159213, 2023-08-25T09:16:36.841 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000194 2023-08-25T09:16:36.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.841 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1758320, 2023-08-25T09:16:36.842 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 201472 2023-08-25T09:16:36.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.842 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:36.843 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 52376, 2023-08-25T09:16:36.843 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 55074, 2023-08-25T09:16:36.843 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 16361277, 2023-08-25T09:16:36.844 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 179830506, 2023-08-25T09:16:36.844 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 10, 2023-08-25T09:16:36.844 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2023-08-25T09:16:36.844 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 3.884305327, 2023-08-25T09:16:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 1.625403078, 2023-08-25T09:16:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 1.551984380, 2023-08-25T09:16:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.850901868, 2023-08-25T09:16:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 55073, 2023-08-25T09:16:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 5.777497263, 2023-08-25T09:16:36.846 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000104906 2023-08-25T09:16:36.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.846 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:36.846 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 61954, 2023-08-25T09:16:36.847 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.011290495, 2023-08-25T09:16:36.847 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000182 2023-08-25T09:16:36.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.848 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 2320, 2023-08-25T09:16:36.848 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 122656 2023-08-25T09:16:36.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.849 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:36.849 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 136224, 2023-08-25T09:16:36.850 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 140930, 2023-08-25T09:16:36.850 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 44599121, 2023-08-25T09:16:36.850 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 151578833, 2023-08-25T09:16:36.851 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 8, 2023-08-25T09:16:36.851 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2023-08-25T09:16:36.852 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 7.005588276, 2023-08-25T09:16:36.852 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 2.674730710, 2023-08-25T09:16:36.852 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 4.219446560, 2023-08-25T09:16:36.853 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.709478641, 2023-08-25T09:16:36.853 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:36.854 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 140929, 2023-08-25T09:16:36.854 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 8.703247723, 2023-08-25T09:16:36.854 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000061756 2023-08-25T09:16:36.855 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.855 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:36.855 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 233328, 2023-08-25T09:16:36.856 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.035225649, 2023-08-25T09:16:36.856 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000150 2023-08-25T09:16:36.857 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.857 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-08-25T09:16:36.857 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 452176 2023-08-25T09:16:36.858 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.858 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2023-08-25T09:16:36.858 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2023-08-25T09:16:36.859 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2023-08-25T09:16:36.859 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.860 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2023-08-25T09:16:36.860 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-08-25T09:16:36.860 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-08-25T09:16:36.861 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 60696, 2023-08-25T09:16:36.861 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 84.537453253, 2023-08-25T09:16:36.861 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001392801 2023-08-25T09:16:36.862 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.862 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.862 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2023-08-25T09:16:36.863 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-08-25T09:16:36.863 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-08-25T09:16:36.864 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 37094, 2023-08-25T09:16:36.864 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 9.862593930, 2023-08-25T09:16:36.864 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000265881 2023-08-25T09:16:36.865 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.865 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.865 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2023-08-25T09:16:36.866 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 142919, 2023-08-25T09:16:36.866 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 138444, 2023-08-25T09:16:36.867 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2023-08-25T09:16:36.867 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 138444, 2023-08-25T09:16:36.867 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 530.902552346, 2023-08-25T09:16:36.868 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003834781 2023-08-25T09:16:36.868 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.869 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2023-08-25T09:16:36.869 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 4475, 2023-08-25T09:16:36.869 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 6, 2023-08-25T09:16:36.870 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2023-08-25T09:16:36.870 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 116, 2023-08-25T09:16:36.870 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 12, 2023-08-25T09:16:36.871 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 8, 2023-08-25T09:16:36.871 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 2781, 2023-08-25T09:16:36.872 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 1085, 2023-08-25T09:16:36.872 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 1680, 2023-08-25T09:16:36.872 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 16, 2023-08-25T09:16:36.873 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 1221, 2023-08-25T09:16:36.873 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 215, 2023-08-25T09:16:36.873 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 322, 2023-08-25T09:16:36.874 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 322, 2023-08-25T09:16:36.874 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 12, 2023-08-25T09:16:36.875 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 234149, 2023-08-25T09:16:36.875 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 125936, 2023-08-25T09:16:36.875 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 4473, 2023-08-25T09:16:36.876 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 27, 2023-08-25T09:16:36.876 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 5, 2023-08-25T09:16:36.876 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2023-08-25T09:16:36.877 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 1981, 2023-08-25T09:16:36.877 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 10590, 2023-08-25T09:16:36.878 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2023-08-25T09:16:36.878 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 3, 2023-08-25T09:16:36.878 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 158, 2023-08-25T09:16:36.878 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 7, 2023-08-25T09:16:36.879 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 703, 2023-08-25T09:16:36.879 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2023-08-25T09:16:36.879 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2023-08-25T09:16:36.879 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2023-08-25T09:16:36.879 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 744, 2023-08-25T09:16:36.880 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 215997246, 2023-08-25T09:16:36.880 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 6, 2023-08-25T09:16:36.880 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T09:16:36.881 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2023-08-25T09:16:36.881 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T09:16:36.881 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2023-08-25T09:16:36.881 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2023-08-25T09:16:36.882 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T09:16:36.882 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T09:16:36.882 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2023-08-25T09:16:36.882 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 4, 2023-08-25T09:16:36.882 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 24914, 2023-08-25T09:16:36.883 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 97862, 2023-08-25T09:16:36.883 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T09:16:36.883 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 2082, 2023-08-25T09:16:36.883 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T09:16:36.883 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 52449, 2023-08-25T09:16:36.884 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 20946, 2023-08-25T09:16:36.884 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 6126, 2023-08-25T09:16:36.884 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 104616 2023-08-25T09:16:36.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.884 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2023-08-25T09:16:36.885 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 28, 2023-08-25T09:16:36.885 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 82, 2023-08-25T09:16:36.885 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 48, 2023-08-25T09:16:36.885 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 96, 2023-08-25T09:16:36.885 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 123, 2023-08-25T09:16:36.885 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 105, 2023-08-25T09:16:36.886 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2023-08-25T09:16:36.886 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2023-08-25T09:16:36.886 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 23910, 2023-08-25T09:16:36.886 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 23805, 2023-08-25T09:16:36.886 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2023-08-25T09:16:36.887 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2023-08-25T09:16:36.887 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2023-08-25T09:16:36.887 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2023-08-25T09:16:36.887 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2023-08-25T09:16:36.887 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2023-08-25T09:16:36.888 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2023-08-25T09:16:36.888 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T09:16:36.888 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 3, 2023-08-25T09:16:36.888 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2023-08-25T09:16:36.888 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 20, 2023-08-25T09:16:36.889 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2023-08-25T09:16:36.889 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2023-08-25T09:16:36.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.889 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2023-08-25T09:16:36.889 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2023-08-25T09:16:36.890 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 113001, 2023-08-25T09:16:36.890 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 23680, 2023-08-25T09:16:36.890 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 12291, 2023-08-25T09:16:36.890 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 100710, 2023-08-25T09:16:36.890 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2023-08-25T09:16:36.890 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 11389, 2023-08-25T09:16:36.891 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 155, 2023-08-25T09:16:36.891 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 27, 2023-08-25T09:16:36.891 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 14, 2023-08-25T09:16:36.891 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 141, 2023-08-25T09:16:36.891 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 9, 2023-08-25T09:16:36.892 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2023-08-25T09:16:36.892 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 13, 2023-08-25T09:16:36.892 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 22248975, 2023-08-25T09:16:36.892 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 443632318, 2023-08-25T09:16:36.892 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2023-08-25T09:16:36.893 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2023-08-25T09:16:36.893 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 33341, 2023-08-25T09:16:36.893 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 154.116325378, 2023-08-25T09:16:36.893 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004622426 2023-08-25T09:16:36.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.894 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2023-08-25T09:16:36.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.894 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2023-08-25T09:16:36.894 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1434, 2023-08-25T09:16:36.894 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 25392, 2023-08-25T09:16:36.895 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 23958, 2023-08-25T09:16:36.895 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 152, 2023-08-25T09:16:36.895 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 297, 2023-08-25T09:16:36.895 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 145, 2023-08-25T09:16:36.895 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 2781, 2023-08-25T09:16:36.895 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 44564, 2023-08-25T09:16:36.896 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 41783, 2023-08-25T09:16:36.896 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 317, 2023-08-25T09:16:36.896 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 25731, 2023-08-25T09:16:36.896 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 25414, 2023-08-25T09:16:36.896 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 82356, 2023-08-25T09:16:36.897 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 198960 2023-08-25T09:16:36.897 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.897 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2023-08-25T09:16:36.897 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 219611, 2023-08-25T09:16:36.897 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 172, 2023-08-25T09:16:36.898 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 142919, 2023-08-25T09:16:36.898 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 3199, 2023-08-25T09:16:36.898 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 10380, 2023-08-25T09:16:36.898 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2023-08-25T09:16:36.898 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 23908, 2023-08-25T09:16:36.899 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 88.833018665, 2023-08-25T09:16:36.899 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003715618 2023-08-25T09:16:36.899 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.899 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2023-08-25T09:16:36.899 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1003, 2023-08-25T09:16:36.900 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 9.457653827, 2023-08-25T09:16:36.900 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.009429365 2023-08-25T09:16:36.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.900 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2023-08-25T09:16:36.900 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.901 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.901 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.901 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2023-08-25T09:16:36.901 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.901 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.902 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.902 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2023-08-25T09:16:36.902 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 39035, 2023-08-25T09:16:36.902 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 92.808165743, 2023-08-25T09:16:36.903 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002377562 2023-08-25T09:16:36.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.903 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2023-08-25T09:16:36.903 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.903 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.904 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.904 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2023-08-25T09:16:36.904 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.904 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.905 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.905 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2023-08-25T09:16:36.905 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.905 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.906 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.906 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2023-08-25T09:16:36.906 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.906 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.907 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2023-08-25T09:16:36.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.907 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.908 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2023-08-25T09:16:36.908 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.908 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.908 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.909 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2023-08-25T09:16:36.909 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.909 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.909 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.910 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2023-08-25T09:16:36.910 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.910 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.910 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.911 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2023-08-25T09:16:36.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.911 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.912 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2023-08-25T09:16:36.912 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.912 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.912 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.913 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2023-08-25T09:16:36.913 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 45629, 2023-08-25T09:16:36.913 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 246.357426096, 2023-08-25T09:16:36.913 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005399141 2023-08-25T09:16:36.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.914 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2023-08-25T09:16:36.914 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4446, 2023-08-25T09:16:36.914 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 17.384613537, 2023-08-25T09:16:36.914 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003910169 2023-08-25T09:16:36.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.915 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2023-08-25T09:16:36.915 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.915 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.915 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.916 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2023-08-25T09:16:36.916 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.916 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.916 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.916 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.917 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2023-08-25T09:16:36.917 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.917 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.917 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.917 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.918 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2023-08-25T09:16:36.918 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.918 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.918 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.918 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.918 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2023-08-25T09:16:36.919 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.919 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.919 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.919 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2023-08-25T09:16:36.920 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 17, 2023-08-25T09:16:36.920 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.090586505, 2023-08-25T09:16:36.920 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005328617 2023-08-25T09:16:36.920 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.920 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2023-08-25T09:16:36.921 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.921 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.921 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.921 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2023-08-25T09:16:36.922 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 496, 2023-08-25T09:16:36.922 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.085494044, 2023-08-25T09:16:36.922 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002188496 2023-08-25T09:16:36.922 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.922 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2023-08-25T09:16:36.923 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.923 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.923 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.923 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2023-08-25T09:16:36.923 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.924 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.924 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.924 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.924 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2023-08-25T09:16:36.924 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.925 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.925 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.925 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2023-08-25T09:16:36.925 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 23910, 2023-08-25T09:16:36.926 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 74.885593929, 2023-08-25T09:16:36.926 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003131977 2023-08-25T09:16:36.926 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.926 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2023-08-25T09:16:36.926 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2023-08-25T09:16:36.927 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2023-08-25T09:16:36.927 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.927 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.927 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.927 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.928 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2023-08-25T09:16:36.928 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2023-08-25T09:16:36.928 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 2, 2023-08-25T09:16:36.928 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 1, 2023-08-25T09:16:36.929 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2023-08-25T09:16:36.929 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2023-08-25T09:16:36.929 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 35581, 2023-08-25T09:16:36.929 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 35581, 2023-08-25T09:16:36.929 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 1052 2023-08-25T09:16:36.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.930 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2023-08-25T09:16:36.930 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:36.930 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:36.930 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:36.930 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:36.931 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:36.931 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:36.931 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:36.931 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:36.931 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:36.932 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:36.932 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:36.932 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:36.932 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:36.932 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:36.933 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:36.933 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:36.933 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:36.933 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:36.933 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:36.933 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:36.934 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:36.934 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:36.934 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:36.934 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:36.934 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:36.935 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:36.935 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:36.935 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:36.935 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:36.935 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:36.936 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:36.936 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2023-08-25T09:16:36.936 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:36.936 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:36.936 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:36.937 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:36.937 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 1579881, 2023-08-25T09:16:36.937 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 1361, 2023-08-25T09:16:36.937 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 264, 2023-08-25T09:16:36.937 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 3, 2023-08-25T09:16:36.938 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2023-08-25T09:16:36.938 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2023-08-25T09:16:36.938 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:36.938 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:36.938 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:36.938 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:36.939 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8416, 2023-08-25T09:16:36.939 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 175, 2023-08-25T09:16:36.939 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:36.939 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:36.939 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:36.940 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2023-08-25T09:16:36.940 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 6071895, 2023-08-25T09:16:36.940 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 71051, 2023-08-25T09:16:36.940 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:36.940 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2023-08-25T09:16:36.941 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:36.941 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2023-08-25T09:16:36.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.941 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2023-08-25T09:16:36.941 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2023-08-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2023-08-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 60094, 2023-08-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 444511323, 2023-08-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2023-08-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 60094, 2023-08-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2023-08-25T09:16:36.943 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 60094, 2023-08-25T09:16:36.943 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 303.188126785, 2023-08-25T09:16:36.943 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005045231 2023-08-25T09:16:36.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.943 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2023-08-25T09:16:36.944 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2023-08-25T09:16:36.944 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 60094, 2023-08-25T09:16:36.944 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 60654 2023-08-25T09:16:36.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.944 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 60094, 2023-08-25T09:16:36.945 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 951, 2023-08-25T09:16:36.945 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 59143, 2023-08-25T09:16:36.945 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2023-08-25T09:16:36.945 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2023-08-25T09:16:36.945 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2023-08-25T09:16:36.946 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 21, 2023-08-25T09:16:36.946 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 940, 2023-08-25T09:16:36.946 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 34281, 2023-08-25T09:16:36.946 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 522, 2023-08-25T09:16:36.946 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2023-08-25T09:16:36.947 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2023-08-25T09:16:36.947 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2023-08-25T09:16:36.947 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 12, 2023-08-25T09:16:36.947 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 23989, 2023-08-25T09:16:36.947 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2023-08-25T09:16:36.947 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2023-08-25T09:16:36.948 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2023-08-25T09:16:36.948 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 6, 2023-08-25T09:16:36.948 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 42, 2023-08-25T09:16:36.948 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2023-08-25T09:16:36.948 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2023-08-25T09:16:36.949 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2023-08-25T09:16:36.949 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2023-08-25T09:16:36.949 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2023-08-25T09:16:36.949 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2023-08-25T09:16:36.949 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T09:16:36.950 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2023-08-25T09:16:36.950 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2023-08-25T09:16:36.950 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2023-08-25T09:16:36.950 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2023-08-25T09:16:36.950 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2023-08-25T09:16:36.951 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2023-08-25T09:16:36.951 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2023-08-25T09:16:36.951 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2023-08-25T09:16:36.951 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2023-08-25T09:16:36.951 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2023-08-25T09:16:36.951 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2023-08-25T09:16:36.952 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2023-08-25T09:16:36.952 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2023-08-25T09:16:36.952 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2023-08-25T09:16:36.952 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2023-08-25T09:16:36.952 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2023-08-25T09:16:36.953 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2023-08-25T09:16:36.953 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2023-08-25T09:16:36.953 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2023-08-25T09:16:36.953 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 112, 2023-08-25T09:16:36.953 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2023-08-25T09:16:36.954 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 29, 2023-08-25T09:16:36.954 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2023-08-25T09:16:36.954 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2023-08-25T09:16:36.954 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2023-08-25T09:16:36.954 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2023-08-25T09:16:36.954 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 615, 2023-08-25T09:16:36.955 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 17, 2023-08-25T09:16:36.955 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 209 2023-08-25T09:16:36.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.955 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2023-08-25T09:16:36.955 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2023-08-25T09:16:36.956 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2023-08-25T09:16:36.956 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2023-08-25T09:16:36.956 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2023-08-25T09:16:36.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.956 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2023-08-25T09:16:36.957 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 2 2023-08-25T09:16:36.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.957 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2023-08-25T09:16:36.957 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2023-08-25T09:16:36.957 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 10, 2023-08-25T09:16:36.958 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2023-08-25T09:16:36.958 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 10, 2023-08-25T09:16:36.958 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 23805, 2023-08-25T09:16:36.958 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2023-08-25T09:16:36.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.958 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:36.959 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:36.959 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-08-25T09:16:36.959 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:36.959 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 418588, 2023-08-25T09:16:36.959 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 100616539, 2023-08-25T09:16:36.960 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:36.960 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 418588, 2023-08-25T09:16:36.960 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:36.960 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:36.960 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 418588, 2023-08-25T09:16:36.961 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 100616539, 2023-08-25T09:16:36.961 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:36.961 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.961 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.961 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.962 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.962 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2023-08-25T09:16:36.962 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:36.962 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-08-25T09:16:36.962 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:36.963 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-08-25T09:16:36.963 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-08-25T09:16:36.963 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:36.963 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:36.963 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 60094, 2023-08-25T09:16:36.964 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 446918150, 2023-08-25T09:16:36.964 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 60094, 2023-08-25T09:16:36.964 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 446918150, 2023-08-25T09:16:36.964 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:36.964 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.965 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.965 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.965 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.965 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2023-08-25T09:16:36.966 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:36.966 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2023-08-25T09:16:36.966 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:36.966 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-08-25T09:16:36.966 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-08-25T09:16:36.967 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:36.967 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:36.967 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 60094, 2023-08-25T09:16:36.967 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 60094, 2023-08-25T09:16:36.967 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 60094, 2023-08-25T09:16:36.968 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 60094, 2023-08-25T09:16:36.968 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:36.968 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.968 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.968 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.969 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.969 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2023-08-25T09:16:36.969 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:36.969 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-08-25T09:16:36.969 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:36.970 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 23805, 2023-08-25T09:16:36.970 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 2407081, 2023-08-25T09:16:36.970 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:36.970 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 23805, 2023-08-25T09:16:36.970 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:36.971 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:36.971 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 940, 2023-08-25T09:16:36.971 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2407081, 2023-08-25T09:16:36.971 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:36.971 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.972 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.972 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.972 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:36.972 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x559a6577e6e0": { 2023-08-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 6564, 2023-08-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-08-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 113001, 2023-08-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 439438014, 2023-08-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:36.974 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 113001, 2023-08-25T09:16:36.974 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:36.974 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:36.974 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 33341, 2023-08-25T09:16:36.974 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 439431450, 2023-08-25T09:16:36.975 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:36.975 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:36.975 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:36.975 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:36.975 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.976 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:36.976 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-08-25T09:16:37.248 INFO:tasks.check_counter:Daemon mds.f mds.exported=3 2023-08-25T09:16:37.249 INFO:tasks.check_counter:Daemon mds.f mds.imported=7 2023-08-25T09:16:37.249 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2023-08-25T09:16:37.249 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-08-25T09:16:37.250 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T09:16:37.250 DEBUG:tasks.check_counter:Getting stats from i 2023-08-25T09:16:37.250 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.i perf dump 2023-08-25T09:16:37.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:37 smithi181 ceph-mon[131296]: pgmap v550: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2023-08-25T09:16:37.805 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T09:16:37.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:37 smithi099 ceph-mon[97641]: pgmap v550: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2023-08-25T09:16:37.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:37 smithi157 ceph-mon[130879]: pgmap v550: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2023-08-25T09:16:39.151 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-08-25T09:16:39.152 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:39.152 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 352961, 2023-08-25T09:16:39.152 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 461900, 2023-08-25T09:16:39.152 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 119545086, 2023-08-25T09:16:39.153 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 418507670, 2023-08-25T09:16:39.153 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 14, 2023-08-25T09:16:39.153 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2023-08-25T09:16:39.153 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 18.052875944, 2023-08-25T09:16:39.153 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 7.669394604, 2023-08-25T09:16:39.154 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 13.237716423, 2023-08-25T09:16:39.154 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.531184667, 2023-08-25T09:16:39.154 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:39.154 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 461899, 2023-08-25T09:16:39.154 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 53.588356603, 2023-08-25T09:16:39.155 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000116017 2023-08-25T09:16:39.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.155 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:39.155 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 672945, 2023-08-25T09:16:39.155 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.123857749, 2023-08-25T09:16:39.156 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000184 2023-08-25T09:16:39.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.156 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1761936, 2023-08-25T09:16:39.156 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 201280 2023-08-25T09:16:39.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.157 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:39.157 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 17002, 2023-08-25T09:16:39.157 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 17297, 2023-08-25T09:16:39.157 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 4312236, 2023-08-25T09:16:39.157 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 180936465, 2023-08-25T09:16:39.158 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 13, 2023-08-25T09:16:39.158 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2023-08-25T09:16:39.158 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 1.567620618, 2023-08-25T09:16:39.158 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.535187917, 2023-08-25T09:16:39.159 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.257106643, 2023-08-25T09:16:39.159 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.691509014, 2023-08-25T09:16:39.159 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:39.159 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 17294, 2023-08-25T09:16:39.160 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2.731582505, 2023-08-25T09:16:39.160 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000157949 2023-08-25T09:16:39.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.161 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:39.161 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 107, 2023-08-25T09:16:39.161 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000017279, 2023-08-25T09:16:39.161 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000161 2023-08-25T09:16:39.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.162 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 6816, 2023-08-25T09:16:39.162 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 595664 2023-08-25T09:16:39.162 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.162 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:39.162 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 54502, 2023-08-25T09:16:39.163 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 51826, 2023-08-25T09:16:39.163 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 20361622, 2023-08-25T09:16:39.163 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 210075761, 2023-08-25T09:16:39.163 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 8, 2023-08-25T09:16:39.163 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2023-08-25T09:16:39.164 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 3.738176482, 2023-08-25T09:16:39.164 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 1.613382064, 2023-08-25T09:16:39.164 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 1.579950113, 2023-08-25T09:16:39.164 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.725630848, 2023-08-25T09:16:39.164 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:39.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 51825, 2023-08-25T09:16:39.165 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 5.912798007, 2023-08-25T09:16:39.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000114091 2023-08-25T09:16:39.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.165 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:39.166 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 62143, 2023-08-25T09:16:39.166 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.010769393, 2023-08-25T09:16:39.166 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000173 2023-08-25T09:16:39.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.166 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-25T09:16:39.166 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2023-08-25T09:16:39.167 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.167 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2023-08-25T09:16:39.167 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2023-08-25T09:16:39.167 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2023-08-25T09:16:39.167 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.168 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2023-08-25T09:16:39.168 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-08-25T09:16:39.168 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-08-25T09:16:39.168 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 48075, 2023-08-25T09:16:39.168 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 93.348002332, 2023-08-25T09:16:39.169 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001941716 2023-08-25T09:16:39.169 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.169 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.169 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2023-08-25T09:16:39.169 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-08-25T09:16:39.170 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-08-25T09:16:39.170 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 30658, 2023-08-25T09:16:39.170 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 13.689606692, 2023-08-25T09:16:39.170 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000446526 2023-08-25T09:16:39.170 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.171 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2023-08-25T09:16:39.171 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 157957, 2023-08-25T09:16:39.171 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 147349, 2023-08-25T09:16:39.171 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2023-08-25T09:16:39.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 147349, 2023-08-25T09:16:39.172 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 917.831714820, 2023-08-25T09:16:39.172 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006228964 2023-08-25T09:16:39.172 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.172 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2023-08-25T09:16:39.172 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 10608, 2023-08-25T09:16:39.173 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 26, 2023-08-25T09:16:39.173 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2023-08-25T09:16:39.173 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 147, 2023-08-25T09:16:39.173 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 15, 2023-08-25T09:16:39.173 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 15, 2023-08-25T09:16:39.174 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 3325, 2023-08-25T09:16:39.174 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 1097, 2023-08-25T09:16:39.174 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 1992, 2023-08-25T09:16:39.174 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 236, 2023-08-25T09:16:39.174 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 1758, 2023-08-25T09:16:39.175 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 832, 2023-08-25T09:16:39.175 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 352, 2023-08-25T09:16:39.175 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 353, 2023-08-25T09:16:39.175 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 22, 2023-08-25T09:16:39.175 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 215595, 2023-08-25T09:16:39.175 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 133572, 2023-08-25T09:16:39.176 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 10588, 2023-08-25T09:16:39.176 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 88, 2023-08-25T09:16:39.176 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 8, 2023-08-25T09:16:39.176 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2023-08-25T09:16:39.176 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 1159, 2023-08-25T09:16:39.177 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 21955, 2023-08-25T09:16:39.177 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2023-08-25T09:16:39.177 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 17, 2023-08-25T09:16:39.177 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 574, 2023-08-25T09:16:39.177 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 11, 2023-08-25T09:16:39.178 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 1193, 2023-08-25T09:16:39.178 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2023-08-25T09:16:39.178 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2023-08-25T09:16:39.178 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2023-08-25T09:16:39.178 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 744, 2023-08-25T09:16:39.179 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 215997246, 2023-08-25T09:16:39.179 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 6, 2023-08-25T09:16:39.179 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T09:16:39.179 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2023-08-25T09:16:39.179 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T09:16:39.180 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2023-08-25T09:16:39.180 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2023-08-25T09:16:39.180 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T09:16:39.180 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T09:16:39.180 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2023-08-25T09:16:39.181 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 12, 2023-08-25T09:16:39.181 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 36769, 2023-08-25T09:16:39.181 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 100595, 2023-08-25T09:16:39.181 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T09:16:39.181 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 2195, 2023-08-25T09:16:39.182 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T09:16:39.182 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 64274, 2023-08-25T09:16:39.182 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 20344, 2023-08-25T09:16:39.182 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 6822, 2023-08-25T09:16:39.182 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 106327 2023-08-25T09:16:39.182 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.183 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2023-08-25T09:16:39.183 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 29, 2023-08-25T09:16:39.183 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 80, 2023-08-25T09:16:39.183 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 49, 2023-08-25T09:16:39.183 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 146, 2023-08-25T09:16:39.184 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 149, 2023-08-25T09:16:39.184 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 68, 2023-08-25T09:16:39.184 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2023-08-25T09:16:39.184 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2023-08-25T09:16:39.184 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 20778, 2023-08-25T09:16:39.185 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 20710, 2023-08-25T09:16:39.185 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2023-08-25T09:16:39.185 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2023-08-25T09:16:39.185 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2023-08-25T09:16:39.185 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2023-08-25T09:16:39.186 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2023-08-25T09:16:39.186 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2023-08-25T09:16:39.186 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2023-08-25T09:16:39.186 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T09:16:39.186 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 31, 2023-08-25T09:16:39.187 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2023-08-25T09:16:39.187 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 30, 2023-08-25T09:16:39.187 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2023-08-25T09:16:39.187 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2023-08-25T09:16:39.187 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.187 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2023-08-25T09:16:39.188 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2023-08-25T09:16:39.188 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 117283, 2023-08-25T09:16:39.188 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 33348, 2023-08-25T09:16:39.188 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 24581, 2023-08-25T09:16:39.188 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 92702, 2023-08-25T09:16:39.189 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2023-08-25T09:16:39.189 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 8767, 2023-08-25T09:16:39.189 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 170, 2023-08-25T09:16:39.189 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 42, 2023-08-25T09:16:39.189 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 31, 2023-08-25T09:16:39.190 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 139, 2023-08-25T09:16:39.190 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 8, 2023-08-25T09:16:39.190 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2023-08-25T09:16:39.190 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 11, 2023-08-25T09:16:39.190 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 35246797, 2023-08-25T09:16:39.190 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 476159339, 2023-08-25T09:16:39.191 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2023-08-25T09:16:39.191 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2023-08-25T09:16:39.191 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 28587, 2023-08-25T09:16:39.191 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 211.643386834, 2023-08-25T09:16:39.192 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.007403483 2023-08-25T09:16:39.192 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.192 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2023-08-25T09:16:39.192 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.192 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2023-08-25T09:16:39.193 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1610, 2023-08-25T09:16:39.193 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 22936, 2023-08-25T09:16:39.193 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 21326, 2023-08-25T09:16:39.193 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 189, 2023-08-25T09:16:39.193 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 388, 2023-08-25T09:16:39.193 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 199, 2023-08-25T09:16:39.194 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 3326, 2023-08-25T09:16:39.194 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 44360, 2023-08-25T09:16:39.194 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 41034, 2023-08-25T09:16:39.194 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 341, 2023-08-25T09:16:39.194 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 22729, 2023-08-25T09:16:39.195 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 22388, 2023-08-25T09:16:39.195 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 78464, 2023-08-25T09:16:39.195 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 198960 2023-08-25T09:16:39.195 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.195 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2023-08-25T09:16:39.196 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 206921, 2023-08-25T09:16:39.196 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 2441, 2023-08-25T09:16:39.196 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 157957, 2023-08-25T09:16:39.196 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 2856, 2023-08-25T09:16:39.196 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 7988, 2023-08-25T09:16:39.197 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2023-08-25T09:16:39.197 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 20845, 2023-08-25T09:16:39.197 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 121.974646696, 2023-08-25T09:16:39.197 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005851506 2023-08-25T09:16:39.197 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.198 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2023-08-25T09:16:39.198 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2801, 2023-08-25T09:16:39.198 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 58.073583800, 2023-08-25T09:16:39.198 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.020733160 2023-08-25T09:16:39.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.198 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2023-08-25T09:16:39.199 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.199 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.199 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.199 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2023-08-25T09:16:39.200 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.200 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.200 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.200 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2023-08-25T09:16:39.201 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 32214, 2023-08-25T09:16:39.201 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 185.052603425, 2023-08-25T09:16:39.201 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005744477 2023-08-25T09:16:39.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.201 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2023-08-25T09:16:39.202 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.202 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.202 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.202 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2023-08-25T09:16:39.203 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2023-08-25T09:16:39.203 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000107208, 2023-08-25T09:16:39.203 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000107208 2023-08-25T09:16:39.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.203 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2023-08-25T09:16:39.203 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.204 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.204 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.204 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2023-08-25T09:16:39.204 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.205 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.205 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.205 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2023-08-25T09:16:39.205 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.206 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.206 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.206 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2023-08-25T09:16:39.206 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.207 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.207 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.207 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.207 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2023-08-25T09:16:39.207 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.208 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.208 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.208 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2023-08-25T09:16:39.208 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.208 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.209 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.209 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2023-08-25T09:16:39.209 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.209 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.210 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.210 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2023-08-25T09:16:39.210 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.210 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.211 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.211 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2023-08-25T09:16:39.211 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 65915, 2023-08-25T09:16:39.211 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 379.532600115, 2023-08-25T09:16:39.212 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005757909 2023-08-25T09:16:39.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.212 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2023-08-25T09:16:39.212 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4120, 2023-08-25T09:16:39.212 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 55.445131241, 2023-08-25T09:16:39.212 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.013457556 2023-08-25T09:16:39.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.213 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2023-08-25T09:16:39.213 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.213 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.213 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.214 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2023-08-25T09:16:39.214 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.214 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.214 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.215 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2023-08-25T09:16:39.215 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.215 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.215 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.216 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2023-08-25T09:16:39.216 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.216 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.216 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.217 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2023-08-25T09:16:39.217 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.217 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.217 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.218 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2023-08-25T09:16:39.218 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 31, 2023-08-25T09:16:39.218 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.734801343, 2023-08-25T09:16:39.218 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.023703269 2023-08-25T09:16:39.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.219 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2023-08-25T09:16:39.219 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.219 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.219 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.220 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2023-08-25T09:16:39.220 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 644, 2023-08-25T09:16:39.220 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 3.099690913, 2023-08-25T09:16:39.220 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004813184 2023-08-25T09:16:39.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.221 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2023-08-25T09:16:39.221 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.221 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.221 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.222 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2023-08-25T09:16:39.222 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.222 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.222 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.223 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2023-08-25T09:16:39.223 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.223 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.223 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.223 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2023-08-25T09:16:39.224 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 20778, 2023-08-25T09:16:39.224 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 113.918550079, 2023-08-25T09:16:39.224 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005482652 2023-08-25T09:16:39.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.224 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2023-08-25T09:16:39.225 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2023-08-25T09:16:39.225 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2023-08-25T09:16:39.225 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.225 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.225 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.226 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.226 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2023-08-25T09:16:39.226 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 2, 2023-08-25T09:16:39.226 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 4, 2023-08-25T09:16:39.227 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 2, 2023-08-25T09:16:39.227 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 2, 2023-08-25T09:16:39.227 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2023-08-25T09:16:39.227 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 36942, 2023-08-25T09:16:39.227 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 18471, 2023-08-25T09:16:39.228 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 1017 2023-08-25T09:16:39.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.228 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2023-08-25T09:16:39.228 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:39.228 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:39.229 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:39.229 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:39.229 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:39.229 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:39.229 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:39.229 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:39.230 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:39.230 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:39.230 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:39.230 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:39.230 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:39.231 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:39.231 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:39.231 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:39.231 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:39.231 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:39.232 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:39.232 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:39.232 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:39.232 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:39.232 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:39.233 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:39.233 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:39.233 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:39.233 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:39.233 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:39.233 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:39.234 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:39.234 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:39.234 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2023-08-25T09:16:39.234 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:39.234 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:39.235 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:39.235 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:39.235 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 482651, 2023-08-25T09:16:39.235 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 1219, 2023-08-25T09:16:39.235 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 176, 2023-08-25T09:16:39.236 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 2, 2023-08-25T09:16:39.236 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2023-08-25T09:16:39.236 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2023-08-25T09:16:39.236 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:39.236 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:39.237 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:39.237 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:39.237 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8416, 2023-08-25T09:16:39.237 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 175, 2023-08-25T09:16:39.237 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:39.237 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:39.238 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:39.238 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2023-08-25T09:16:39.238 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 6825961, 2023-08-25T09:16:39.238 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 75365, 2023-08-25T09:16:39.238 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:39.239 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2023-08-25T09:16:39.239 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:39.239 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2023-08-25T09:16:39.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.239 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2023-08-25T09:16:39.240 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2023-08-25T09:16:39.240 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2023-08-25T09:16:39.240 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 51534, 2023-08-25T09:16:39.240 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 476821031, 2023-08-25T09:16:39.240 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 27, 2023-08-25T09:16:39.240 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 51534, 2023-08-25T09:16:39.241 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2023-08-25T09:16:39.241 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 51534, 2023-08-25T09:16:39.241 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 331.887585475, 2023-08-25T09:16:39.241 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006440167 2023-08-25T09:16:39.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.242 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2023-08-25T09:16:39.242 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2023-08-25T09:16:39.242 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 51534, 2023-08-25T09:16:39.242 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 52116 2023-08-25T09:16:39.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.243 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 51534, 2023-08-25T09:16:39.243 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 678, 2023-08-25T09:16:39.243 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 50856, 2023-08-25T09:16:39.243 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2023-08-25T09:16:39.243 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2023-08-25T09:16:39.244 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2023-08-25T09:16:39.244 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 21, 2023-08-25T09:16:39.244 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 624, 2023-08-25T09:16:39.244 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 29211, 2023-08-25T09:16:39.244 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 350, 2023-08-25T09:16:39.244 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2023-08-25T09:16:39.245 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2023-08-25T09:16:39.245 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2023-08-25T09:16:39.245 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 15, 2023-08-25T09:16:39.245 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 20913, 2023-08-25T09:16:39.245 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2023-08-25T09:16:39.246 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2023-08-25T09:16:39.246 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2023-08-25T09:16:39.246 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 9, 2023-08-25T09:16:39.246 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 42, 2023-08-25T09:16:39.246 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2023-08-25T09:16:39.247 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2023-08-25T09:16:39.247 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2023-08-25T09:16:39.247 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2023-08-25T09:16:39.247 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2023-08-25T09:16:39.247 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2023-08-25T09:16:39.247 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T09:16:39.248 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2023-08-25T09:16:39.248 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2023-08-25T09:16:39.248 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2023-08-25T09:16:39.248 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2023-08-25T09:16:39.248 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2023-08-25T09:16:39.249 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2023-08-25T09:16:39.249 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2023-08-25T09:16:39.249 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2023-08-25T09:16:39.249 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2023-08-25T09:16:39.249 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2023-08-25T09:16:39.250 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2023-08-25T09:16:39.250 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2023-08-25T09:16:39.250 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2023-08-25T09:16:39.250 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2023-08-25T09:16:39.250 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2023-08-25T09:16:39.251 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2023-08-25T09:16:39.251 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2023-08-25T09:16:39.251 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2023-08-25T09:16:39.251 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2023-08-25T09:16:39.251 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 112, 2023-08-25T09:16:39.251 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2023-08-25T09:16:39.252 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 29, 2023-08-25T09:16:39.252 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2023-08-25T09:16:39.252 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2023-08-25T09:16:39.252 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2023-08-25T09:16:39.252 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2023-08-25T09:16:39.253 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 667, 2023-08-25T09:16:39.253 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 80, 2023-08-25T09:16:39.253 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 184 2023-08-25T09:16:39.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.253 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2023-08-25T09:16:39.254 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2023-08-25T09:16:39.254 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2023-08-25T09:16:39.254 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2023-08-25T09:16:39.254 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2023-08-25T09:16:39.254 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2023-08-25T09:16:39.255 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 2 2023-08-25T09:16:39.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.255 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2023-08-25T09:16:39.255 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2023-08-25T09:16:39.255 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 10, 2023-08-25T09:16:39.256 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2023-08-25T09:16:39.256 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 10, 2023-08-25T09:16:39.256 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 20710, 2023-08-25T09:16:39.256 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2023-08-25T09:16:39.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.257 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:39.257 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:39.257 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-08-25T09:16:39.257 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:39.257 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 424461, 2023-08-25T09:16:39.258 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 105992337, 2023-08-25T09:16:39.258 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:39.258 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 424461, 2023-08-25T09:16:39.258 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:39.258 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:39.259 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 424461, 2023-08-25T09:16:39.259 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 105992337, 2023-08-25T09:16:39.259 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:39.259 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.259 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.259 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.260 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.260 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2023-08-25T09:16:39.260 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:39.260 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-08-25T09:16:39.261 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:39.261 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-08-25T09:16:39.261 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-08-25T09:16:39.261 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:39.261 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:39.262 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 51534, 2023-08-25T09:16:39.262 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 478914476, 2023-08-25T09:16:39.262 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 51534, 2023-08-25T09:16:39.262 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 478914476, 2023-08-25T09:16:39.262 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:39.263 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.263 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.263 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.263 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.263 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.263 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2023-08-25T09:16:39.264 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:39.264 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2023-08-25T09:16:39.264 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:39.264 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-08-25T09:16:39.264 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-08-25T09:16:39.265 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:39.265 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-08-25T09:16:39.265 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 51534, 2023-08-25T09:16:39.265 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 51534, 2023-08-25T09:16:39.265 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 51534, 2023-08-25T09:16:39.266 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 51534, 2023-08-25T09:16:39.266 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:39.266 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.266 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.266 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.267 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.267 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2023-08-25T09:16:39.267 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:39.267 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-08-25T09:16:39.268 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:39.268 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 20710, 2023-08-25T09:16:39.268 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 2094742, 2023-08-25T09:16:39.268 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:39.268 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 20710, 2023-08-25T09:16:39.268 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:39.269 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:39.269 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 624, 2023-08-25T09:16:39.269 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2094742, 2023-08-25T09:16:39.269 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:39.269 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.270 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.270 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.270 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:39.270 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x563975bee6e0": { 2023-08-25T09:16:39.271 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 70551, 2023-08-25T09:16:39.271 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-08-25T09:16:39.271 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:39.271 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 117283, 2023-08-25T09:16:39.271 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 471965035, 2023-08-25T09:16:39.272 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:39.272 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 117283, 2023-08-25T09:16:39.272 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:39.272 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:39.272 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 28587, 2023-08-25T09:16:39.273 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 471894484, 2023-08-25T09:16:39.273 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:39.273 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:39.273 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:39.273 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:39.273 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.274 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:39.274 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-08-25T09:16:39.565 INFO:tasks.check_counter:Daemon mds.i mds.exported=17 2023-08-25T09:16:39.566 INFO:tasks.check_counter:Daemon mds.i mds.imported=11 2023-08-25T09:16:39.566 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-08-25T09:16:39.566 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-08-25T09:16:39.566 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T09:16:39.567 DEBUG:tasks.check_counter:Getting stats from l 2023-08-25T09:16:39.567 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c -- ceph daemon mds.l perf dump 2023-08-25T09:16:39.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:39 smithi181 ceph-mon[131296]: pgmap v551: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.6 MiB/s wr, 30 op/s 2023-08-25T09:16:39.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:39 smithi099 ceph-mon[97641]: pgmap v551: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.6 MiB/s wr, 30 op/s 2023-08-25T09:16:39.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:39 smithi157 ceph-mon[130879]: pgmap v551: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.6 MiB/s wr, 30 op/s 2023-08-25T09:16:40.122 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T09:16:41.352 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-08-25T09:16:41.353 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T09:16:41.353 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 330, 2023-08-25T09:16:41.353 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 319, 2023-08-25T09:16:41.354 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1576720, 2023-08-25T09:16:41.354 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 192208, 2023-08-25T09:16:41.354 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2023-08-25T09:16:41.354 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2023-08-25T09:16:41.354 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.055061357, 2023-08-25T09:16:41.355 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.017693310, 2023-08-25T09:16:41.355 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.035236525, 2023-08-25T09:16:41.355 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.008199875, 2023-08-25T09:16:41.356 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:41.356 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 318, 2023-08-25T09:16:41.356 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.015875851, 2023-08-25T09:16:41.356 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000049924 2023-08-25T09:16:41.357 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.357 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:41.357 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:41.357 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:41.357 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:41.358 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.359 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1576720, 2023-08-25T09:16:41.359 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 192208 2023-08-25T09:16:41.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.360 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T09:16:41.360 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3, 2023-08-25T09:16:41.360 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 245, 2023-08-25T09:16:41.360 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 480, 2023-08-25T09:16:41.360 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 271840, 2023-08-25T09:16:41.361 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2023-08-25T09:16:41.361 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2023-08-25T09:16:41.361 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.013758709, 2023-08-25T09:16:41.361 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.009165686, 2023-08-25T09:16:41.361 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.001751041, 2023-08-25T09:16:41.362 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:41.362 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:41.362 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 242, 2023-08-25T09:16:41.362 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.019546564, 2023-08-25T09:16:41.362 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000080770 2023-08-25T09:16:41.363 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.363 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:41.363 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:41.363 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:41.363 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:41.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.364 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T09:16:41.364 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 271840 2023-08-25T09:16:41.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.364 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T09:16:41.364 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 4, 2023-08-25T09:16:41.365 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 4, 2023-08-25T09:16:41.365 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 2944, 2023-08-25T09:16:41.365 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 448, 2023-08-25T09:16:41.365 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 2, 2023-08-25T09:16:41.365 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 1, 2023-08-25T09:16:41.366 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.000670858, 2023-08-25T09:16:41.366 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000038615, 2023-08-25T09:16:41.366 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.000615049, 2023-08-25T09:16:41.366 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T09:16:41.366 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T09:16:41.367 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3, 2023-08-25T09:16:41.367 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000711645, 2023-08-25T09:16:41.367 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000237215 2023-08-25T09:16:41.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.367 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-08-25T09:16:41.368 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:41.368 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:41.368 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:41.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.368 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-25T09:16:41.369 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2023-08-25T09:16:41.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.369 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2023-08-25T09:16:41.369 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 0, 2023-08-25T09:16:41.369 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2023-08-25T09:16:41.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.370 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2023-08-25T09:16:41.370 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2023-08-25T09:16:41.370 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2023-08-25T09:16:41.370 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T09:16:41.370 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2023-08-25T09:16:41.371 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T09:16:41.371 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2023-08-25T09:16:41.371 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T09:16:41.371 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T09:16:41.371 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T09:16:41.372 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T09:16:41.372 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T09:16:41.372 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2023-08-25T09:16:41.372 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T09:16:41.372 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T09:16:41.373 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2023-08-25T09:16:41.373 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2023-08-25T09:16:41.373 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2023-08-25T09:16:41.373 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2023-08-25T09:16:41.373 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T09:16:41.374 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T09:16:41.374 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T09:16:41.374 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T09:16:41.374 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T09:16:41.374 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2023-08-25T09:16:41.375 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2023-08-25T09:16:41.375 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2023-08-25T09:16:41.375 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T09:16:41.375 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T09:16:41.375 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2023-08-25T09:16:41.375 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2023-08-25T09:16:41.376 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2023-08-25T09:16:41.376 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2023-08-25T09:16:41.376 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T09:16:41.376 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2023-08-25T09:16:41.376 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T09:16:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2023-08-25T09:16:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 148731, 2023-08-25T09:16:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 31, 2023-08-25T09:16:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2023-08-25T09:16:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2023-08-25T09:16:41.378 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2023-08-25T09:16:41.378 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2023-08-25T09:16:41.378 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2023-08-25T09:16:41.378 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2023-08-25T09:16:41.378 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2023-08-25T09:16:41.379 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2023-08-25T09:16:41.379 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 0, 2023-08-25T09:16:41.379 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 0, 2023-08-25T09:16:41.379 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T09:16:41.379 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2023-08-25T09:16:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2023-08-25T09:16:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2023-08-25T09:16:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 928, 2023-08-25T09:16:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 2, 2023-08-25T09:16:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2023-08-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2023-08-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2023-08-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2023-08-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-08-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-08-25T09:16:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-08-25T09:16:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-08-25T09:16:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 333, 2023-08-25T09:16:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1530157, 2023-08-25T09:16:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-08-25T09:16:41.383 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 333, 2023-08-25T09:16:41.383 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-08-25T09:16:41.383 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-08-25T09:16:41.383 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 333, 2023-08-25T09:16:41.383 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1530157, 2023-08-25T09:16:41.384 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-08-25T09:16:41.384 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-08-25T09:16:41.384 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-08-25T09:16:41.384 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-08-25T09:16:41.384 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:41.385 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-08-25T09:16:41.385 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-08-25T09:16:41.779 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:41 smithi181 ceph-mon[131296]: pgmap v552: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.6 MiB/s wr, 30 op/s 2023-08-25T09:16:41.781 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-08-25T09:16:41.781 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-08-25T09:16:41.781 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-08-25T09:16:41.782 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-08-25T09:16:41.782 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-08-25T09:16:41.782 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-08-25T09:16:41.793 INFO:tasks.kclient:Unmounting kernel clients... 2023-08-25T09:16:41.794 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T09:16:41.794 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:16:41.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:41 smithi099 ceph-mon[97641]: pgmap v552: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.6 MiB/s wr, 30 op/s 2023-08-25T09:16:41.844 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-08-25T09:16:41.844 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:16:41.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:41 smithi157 ceph-mon[130879]: pgmap v552: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.6 MiB/s wr, 30 op/s 2023-08-25T09:16:41.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:16:42.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 -- 172.21.15.99:0/4172430271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe301068b0 msgr2=0x7fbe30106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 --2- 172.21.15.99:0/4172430271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe301068b0 0x7fbe30106c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009990 tx=0x7fbe20009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 -- 172.21.15.99:0/4172430271 shutdown_connections 2023-08-25T09:16:42.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 --2- 172.21.15.99:0/4172430271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe30101260 0x7fbe3010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 --2- 172.21.15.99:0/4172430271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe301068b0 0x7fbe30106c90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 --2- 172.21.15.99:0/4172430271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe301008a0 0x7fbe30100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 -- 172.21.15.99:0/4172430271 >> 172.21.15.99:0/4172430271 conn(0x7fbe300fc3d0 msgr2=0x7fbe300fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:42.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 -- 172.21.15.99:0/4172430271 shutdown_connections 2023-08-25T09:16:42.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.173+0000 7fbe35bff700 1 -- 172.21.15.99:0/4172430271 wait complete. 2023-08-25T09:16:42.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.174+0000 7fbe35bff700 1 Processor -- start 2023-08-25T09:16:42.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.174+0000 7fbe35bff700 1 -- start start 2023-08-25T09:16:42.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe35bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe301008a0 0x7fbe30072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:42.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe35bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe30101260 0x7fbe30073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:42.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe35bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe301068b0 0x7fbe301a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:42.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe35bff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30073990 con 0x7fbe30101260 2023-08-25T09:16:42.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe35bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301a5040 con 0x7fbe301068b0 2023-08-25T09:16:42.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe35bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301a5180 con 0x7fbe301008a0 2023-08-25T09:16:42.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe30101260 0x7fbe30073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:42.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe30101260 0x7fbe30073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36708/0 (socket says 172.21.15.99:36708) 2023-08-25T09:16:42.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2effd700 1 -- 172.21.15.99:0/355579592 learned_addr learned my addr 172.21.15.99:0/355579592 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:42.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2f7fe700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe301008a0 0x7fbe30072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:42.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2ffff700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe301068b0 0x7fbe301a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:42.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2effd700 1 -- 172.21.15.99:0/355579592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe301008a0 msgr2=0x7fbe30072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2effd700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe301008a0 0x7fbe30072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2effd700 1 -- 172.21.15.99:0/355579592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe301068b0 msgr2=0x7fbe301a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2effd700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe301068b0 0x7fbe301a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.175+0000 7fbe2effd700 1 -- 172.21.15.99:0/355579592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe200095f0 con 0x7fbe30101260 2023-08-25T09:16:42.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.176+0000 7fbe2effd700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe30101260 0x7fbe30073160 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400e9a0 tx=0x7fbe2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:42.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.176+0000 7fbe2cff9700 1 -- 172.21.15.99:0/355579592 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24009790 con 0x7fbe30101260 2023-08-25T09:16:42.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.176+0000 7fbe2cff9700 1 -- 172.21.15.99:0/355579592 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe24010430 con 0x7fbe30101260 2023-08-25T09:16:42.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.176+0000 7fbe2cff9700 1 -- 172.21.15.99:0/355579592 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24018680 con 0x7fbe30101260 2023-08-25T09:16:42.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.176+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe301a53e0 con 0x7fbe30101260 2023-08-25T09:16:42.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.176+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe301a5870 con 0x7fbe30101260 2023-08-25T09:16:42.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.177+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe30067710 con 0x7fbe30101260 2023-08-25T09:16:42.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.179+0000 7fbe2cff9700 1 -- 172.21.15.99:0/355579592 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fbe240105a0 con 0x7fbe30101260 2023-08-25T09:16:42.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.181+0000 7fbe2cff9700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbe180a0100 0x7fbe180a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:42.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.181+0000 7fbe2cff9700 1 -- 172.21.15.99:0/355579592 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbe24014070 con 0x7fbe30101260 2023-08-25T09:16:42.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.182+0000 7fbe2f7fe700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbe180a0100 0x7fbe180a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:42.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.182+0000 7fbe2f7fe700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbe180a0100 0x7fbe180a25c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000b2e0 tx=0x7fbe20005960 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:42.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.185+0000 7fbe2cff9700 1 -- 172.21.15.99:0/355579592 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fbe24051e00 con 0x7fbe30101260 2023-08-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.387+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe30074150 con 0x7fbe30101260 2023-08-25T09:16:42.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.389+0000 7fbe2cff9700 1 -- 172.21.15.99:0/355579592 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+23210 (secure 0 0 0) 0x7fbe24095260 con 0x7fbe30101260 2023-08-25T09:16:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T09:16:42.391 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":112,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T09:16:14.347830+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2892690228":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/3418985282":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6800/4068336524":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/90449605":"2023-08-26T08:59:10.145783+0000","172.21.15.99:6801/4068336524":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1861696476":"2023-08-26T08:59:10.145783+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:16:42.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbe180a0100 msgr2=0x7fbe180a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbe180a0100 0x7fbe180a25c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000b2e0 tx=0x7fbe20005960 comp rx=0 tx=0).stop 2023-08-25T09:16:42.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe30101260 msgr2=0x7fbe30073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe30101260 0x7fbe30073160 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400e9a0 tx=0x7fbe2400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 shutdown_connections 2023-08-25T09:16:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fbe180a0100 0x7fbe180a25c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe301068b0 0x7fbe301a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe301008a0 0x7fbe30072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 --2- 172.21.15.99:0/355579592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe30101260 0x7fbe30073160 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.391+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 >> 172.21.15.99:0/355579592 conn(0x7fbe300fc3d0 msgr2=0x7fbe300fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:42.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.392+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 shutdown_connections 2023-08-25T09:16:42.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.392+0000 7fbe35bff700 1 -- 172.21.15.99:0/355579592 wait complete. 2023-08-25T09:16:42.412 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T08:53:08.946525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T08:57:17.389400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T08:57:18.695578+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:16:42.413 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:16:42.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 -- 172.21.15.99:0/2325104768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344106880 msgr2=0x7fe344106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/2325104768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344106880 0x7fe344106c60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe334009990 tx=0x7fe334009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 -- 172.21.15.99:0/2325104768 shutdown_connections 2023-08-25T09:16:42.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/2325104768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe344101230 0x7fe34410d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/2325104768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344106880 0x7fe344106c60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/2325104768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe344100870 0x7fe344100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 -- 172.21.15.99:0/2325104768 >> 172.21.15.99:0/2325104768 conn(0x7fe3440fc3a0 msgr2=0x7fe3440fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:42.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 -- 172.21.15.99:0/2325104768 shutdown_connections 2023-08-25T09:16:42.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.613+0000 7fe34a7ff700 1 -- 172.21.15.99:0/2325104768 wait complete. 2023-08-25T09:16:42.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.614+0000 7fe34a7ff700 1 Processor -- start 2023-08-25T09:16:42.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.614+0000 7fe34a7ff700 1 -- start start 2023-08-25T09:16:42.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe34a7ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344100870 0x7fe344196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:42.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe34a7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe344101230 0x7fe344196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:42.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe34a7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe344106880 0x7fe34419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:42.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe34a7ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34419b900 con 0x7fe344100870 2023-08-25T09:16:42.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe34a7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34419ba70 con 0x7fe344101230 2023-08-25T09:16:42.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe34a7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34419bc30 con 0x7fe344106880 2023-08-25T09:16:42.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe343fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344100870 0x7fe344196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:42.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe3437fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe344101230 0x7fe344196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:42.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe343fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344100870 0x7fe344196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36722/0 (socket says 172.21.15.99:36722) 2023-08-25T09:16:42.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe3437fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe344101230 0x7fe344196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:60460/0 (socket says 172.21.15.99:60460) 2023-08-25T09:16:42.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe343fff700 1 -- 172.21.15.99:0/3829765371 learned_addr learned my addr 172.21.15.99:0/3829765371 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:42.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.615+0000 7fe348d9c700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe344106880 0x7fe34419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:42.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3437fe700 1 -- 172.21.15.99:0/3829765371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe344106880 msgr2=0x7fe34419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3437fe700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe344106880 0x7fe34419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3437fe700 1 -- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344100870 msgr2=0x7fe344196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3437fe700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344100870 0x7fe344196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3437fe700 1 -- 172.21.15.99:0/3829765371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3340095f0 con 0x7fe344101230 2023-08-25T09:16:42.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe348d9c700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe344106880 0x7fe34419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:42.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3437fe700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe344101230 0x7fe344196ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe33800e9f0 tx=0x7fe33800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:42.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3417fa700 1 -- 172.21.15.99:0/3829765371 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33800c9c0 con 0x7fe344101230 2023-08-25T09:16:42.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34419bf10 con 0x7fe344101230 2023-08-25T09:16:42.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3417fa700 1 -- 172.21.15.99:0/3829765371 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe338010430 con 0x7fe344101230 2023-08-25T09:16:42.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe3417fa700 1 -- 172.21.15.99:0/3829765371 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338018680 con 0x7fe344101230 2023-08-25T09:16:42.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.616+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe344072d30 con 0x7fe344101230 2023-08-25T09:16:42.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.617+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34419c180 con 0x7fe344101230 2023-08-25T09:16:42.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.619+0000 7fe3417fa700 1 -- 172.21.15.99:0/3829765371 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fe338010860 con 0x7fe344101230 2023-08-25T09:16:42.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.621+0000 7fe3417fa700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe32c0a00c0 0x7fe32c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:42.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.621+0000 7fe3417fa700 1 -- 172.21.15.99:0/3829765371 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fe338014070 con 0x7fe344101230 2023-08-25T09:16:42.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.621+0000 7fe343fff700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe32c0a00c0 0x7fe32c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:42.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.621+0000 7fe343fff700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe32c0a00c0 0x7fe32c0a2580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe334006010 tx=0x7fe33400b260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:42.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.623+0000 7fe3417fa700 1 -- 172.21.15.99:0/3829765371 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fe338095220 con 0x7fe344101230 2023-08-25T09:16:42.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:42 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/355579592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.826+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3441108a0 con 0x7fe344101230 2023-08-25T09:16:42.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.826+0000 7fe3417fa700 1 -- 172.21.15.99:0/3829765371 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe338098840 con 0x7fe344101230 2023-08-25T09:16:42.828 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-08-25T09:16:42.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe32c0a00c0 msgr2=0x7fe32c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe32c0a00c0 0x7fe32c0a2580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe334006010 tx=0x7fe33400b260 comp rx=0 tx=0).stop 2023-08-25T09:16:42.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe344101230 msgr2=0x7fe344196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:42.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe344101230 0x7fe344196ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe33800e9f0 tx=0x7fe33800ed00 comp rx=0 tx=0).stop 2023-08-25T09:16:42.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 shutdown_connections 2023-08-25T09:16:42.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fe32c0a00c0 0x7fe32c0a2580 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe344106880 0x7fe34419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344100870 0x7fe344196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 --2- 172.21.15.99:0/3829765371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe344101230 0x7fe344196ea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:42.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 >> 172.21.15.99:0/3829765371 conn(0x7fe3440fc3a0 msgr2=0x7fe3440fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 shutdown_connections 2023-08-25T09:16:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:42.829+0000 7fe34a7ff700 1 -- 172.21.15.99:0/3829765371 wait complete. 2023-08-25T09:16:42.849 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:16:42.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:42 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/355579592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:42.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:42 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/355579592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:43.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.049+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/368309995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8100890 msgr2=0x7fb0c8100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.049+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/368309995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8100890 0x7fb0c8100d10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009990 tx=0x7fb0bc009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.049+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/368309995 shutdown_connections 2023-08-25T09:16:43.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.049+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/368309995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c8101250 0x7fb0c810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.049+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/368309995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c81068a0 0x7fb0c8106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.049+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/368309995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8100890 0x7fb0c8100d10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.049+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/368309995 >> 172.21.15.99:0/368309995 conn(0x7fb0c80fc380 msgr2=0x7fb0c80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:43.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.050+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/368309995 shutdown_connections 2023-08-25T09:16:43.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.050+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/368309995 wait complete. 2023-08-25T09:16:43.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.051+0000 7fb0ce3ff700 1 Processor -- start 2023-08-25T09:16:43.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.051+0000 7fb0ce3ff700 1 -- start start 2023-08-25T09:16:43.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0ce3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c8100890 0x7fb0c819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0ce3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8101250 0x7fb0c819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0ce3ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c81068a0 0x7fb0c8196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0ce3ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c8196df0 con 0x7fb0c81068a0 2023-08-25T09:16:43.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0ce3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c8196f60 con 0x7fb0c8100890 2023-08-25T09:16:43.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0ce3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c8197120 con 0x7fb0c8101250 2023-08-25T09:16:43.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0cc99c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c81068a0 0x7fb0c8196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0c7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c8100890 0x7fb0c819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0cc99c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c81068a0 0x7fb0c8196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36738/0 (socket says 172.21.15.99:36738) 2023-08-25T09:16:43.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0cc99c700 1 -- 172.21.15.99:0/1993756973 learned_addr learned my addr 172.21.15.99:0/1993756973 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:43.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0c77fe700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8101250 0x7fb0c819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0cc99c700 1 -- 172.21.15.99:0/1993756973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8101250 msgr2=0x7fb0c819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0cc99c700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8101250 0x7fb0c819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0cc99c700 1 -- 172.21.15.99:0/1993756973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c8100890 msgr2=0x7fb0c819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0cc99c700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c8100890 0x7fb0c819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0cc99c700 1 -- 172.21.15.99:0/1993756973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0095f0 con 0x7fb0c81068a0 2023-08-25T09:16:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.052+0000 7fb0c77fe700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8101250 0x7fb0c819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:16:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.053+0000 7fb0cc99c700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c81068a0 0x7fb0c8196780 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000ef30 tx=0x7fb0c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:43.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.053+0000 7fb0c57fa700 1 -- 172.21.15.99:0/1993756973 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0018070 con 0x7fb0c81068a0 2023-08-25T09:16:43.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.053+0000 7fb0c57fa700 1 -- 172.21.15.99:0/1993756973 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0c000f040 con 0x7fb0c81068a0 2023-08-25T09:16:43.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.053+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c81973a0 con 0x7fb0c81068a0 2023-08-25T09:16:43.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.053+0000 7fb0c57fa700 1 -- 172.21.15.99:0/1993756973 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000cdb0 con 0x7fb0c81068a0 2023-08-25T09:16:43.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.053+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c81a3700 con 0x7fb0c81068a0 2023-08-25T09:16:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.054+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c80676a0 con 0x7fb0c81068a0 2023-08-25T09:16:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.056+0000 7fb0c57fa700 1 -- 172.21.15.99:0/1993756973 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fb0c0003a00 con 0x7fb0c81068a0 2023-08-25T09:16:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.057+0000 7fb0c57fa700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb0b00a00c0 0x7fb0b00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.057+0000 7fb0c57fa700 1 -- 172.21.15.99:0/1993756973 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb0c00cbcc0 con 0x7fb0c81068a0 2023-08-25T09:16:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.058+0000 7fb0c7fff700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb0b00a00c0 0x7fb0b00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.058+0000 7fb0c7fff700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb0b00a00c0 0x7fb0b00a2580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009be0 tx=0x7fb0b8009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:43.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.059+0000 7fb0c57fa700 1 -- 172.21.15.99:0/1993756973 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb0c00956d0 con 0x7fb0c81068a0 2023-08-25T09:16:43.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.260+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0c8110910 con 0x7fb0c81068a0 2023-08-25T09:16:43.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.261+0000 7fb0c57fa700 1 -- 172.21.15.99:0/1993756973 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb0c0098cf0 con 0x7fb0c81068a0 2023-08-25T09:16:43.263 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-08-25T09:16:43.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.263+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb0b00a00c0 msgr2=0x7fb0b00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.263+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb0b00a00c0 0x7fb0b00a2580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009be0 tx=0x7fb0b8009260 comp rx=0 tx=0).stop 2023-08-25T09:16:43.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.263+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c81068a0 msgr2=0x7fb0c8196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.263+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c81068a0 0x7fb0c8196780 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000ef30 tx=0x7fb0c000c3a0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.264+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 shutdown_connections 2023-08-25T09:16:43.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.264+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb0b00a00c0 0x7fb0b00a2580 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.264+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c81068a0 0x7fb0c8196780 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.264+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c8100890 0x7fb0c819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.264+0000 7fb0ce3ff700 1 --2- 172.21.15.99:0/1993756973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c8101250 0x7fb0c819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.264+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 >> 172.21.15.99:0/1993756973 conn(0x7fb0c80fc380 msgr2=0x7fb0c80fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:43.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.264+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 shutdown_connections 2023-08-25T09:16:43.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.264+0000 7fb0ce3ff700 1 -- 172.21.15.99:0/1993756973 wait complete. 2023-08-25T09:16:43.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:16:43.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/3077185907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b4100890 msgr2=0x7fc0b4100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/3077185907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b4100890 0x7fc0b4100d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009990 tx=0x7fc0a4009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/3077185907 shutdown_connections 2023-08-25T09:16:43.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/3077185907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b4101250 0x7fc0b410d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/3077185907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0b41068a0 0x7fc0b4106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/3077185907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b4100890 0x7fc0b4100d10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/3077185907 >> 172.21.15.99:0/3077185907 conn(0x7fc0b40fc380 msgr2=0x7fc0b40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:43.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/3077185907 shutdown_connections 2023-08-25T09:16:43.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.484+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/3077185907 wait complete. 2023-08-25T09:16:43.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.485+0000 7fc0bc1ff700 1 Processor -- start 2023-08-25T09:16:43.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.485+0000 7fc0bc1ff700 1 -- start start 2023-08-25T09:16:43.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0bc1ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b4100890 0x7fc0b419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0bc1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b4101250 0x7fc0b419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0bc1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0b41068a0 0x7fc0b4196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0bc1ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b4196e70 con 0x7fc0b4100890 2023-08-25T09:16:43.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0bc1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b4196fe0 con 0x7fc0b4101250 2023-08-25T09:16:43.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0bc1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41971a0 con 0x7fc0b41068a0 2023-08-25T09:16:43.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b9f9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b4100890 0x7fc0b419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b9f9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b4100890 0x7fc0b419c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36754/0 (socket says 172.21.15.99:36754) 2023-08-25T09:16:43.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b9f9b700 1 -- 172.21.15.99:0/888870065 learned_addr learned my addr 172.21.15.99:0/888870065 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:43.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0ba79c700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0b41068a0 0x7fc0b4196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b979a700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b4101250 0x7fc0b419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b9f9b700 1 -- 172.21.15.99:0/888870065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0b41068a0 msgr2=0x7fc0b4196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b9f9b700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0b41068a0 0x7fc0b4196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b9f9b700 1 -- 172.21.15.99:0/888870065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b4101250 msgr2=0x7fc0b419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b9f9b700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b4101250 0x7fc0b419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0b9f9b700 1 -- 172.21.15.99:0/888870065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a40095f0 con 0x7fc0b4100890 2023-08-25T09:16:43.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.486+0000 7fc0ba79c700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0b41068a0 0x7fc0b4196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:16:43.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.487+0000 7fc0b9f9b700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b4100890 0x7fc0b419c790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000e9a0 tx=0x7fc0b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:43.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.487+0000 7fc0aaffd700 1 -- 172.21.15.99:0/888870065 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0009790 con 0x7fc0b4100890 2023-08-25T09:16:43.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.487+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b4197480 con 0x7fc0b4100890 2023-08-25T09:16:43.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.487+0000 7fc0aaffd700 1 -- 172.21.15.99:0/888870065 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b000cbd0 con 0x7fc0b4100890 2023-08-25T09:16:43.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.487+0000 7fc0aaffd700 1 -- 172.21.15.99:0/888870065 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0018680 con 0x7fc0b4100890 2023-08-25T09:16:43.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.487+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b41a3780 con 0x7fc0b4100890 2023-08-25T09:16:43.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.488+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b40676a0 con 0x7fc0b4100890 2023-08-25T09:16:43.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.490+0000 7fc0aaffd700 1 -- 172.21.15.99:0/888870065 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fc0b000cd40 con 0x7fc0b4100890 2023-08-25T09:16:43.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.491+0000 7fc0aaffd700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc0a00a00c0 0x7fc0a00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.491+0000 7fc0aaffd700 1 -- 172.21.15.99:0/888870065 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fc0b0014070 con 0x7fc0b4100890 2023-08-25T09:16:43.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.492+0000 7fc0b979a700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc0a00a00c0 0x7fc0a00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.492+0000 7fc0b979a700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc0a00a00c0 0x7fc0a00a2580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4105fc0 tx=0x7fc0a4005fd0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:43.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.495+0000 7fc0aaffd700 1 -- 172.21.15.99:0/888870065 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc0b0051e00 con 0x7fc0b4100890 2023-08-25T09:16:43.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:43 smithi099 ceph-mon[97641]: pgmap v553: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2023-08-25T09:16:43.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:43 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3829765371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T09:16:43.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:43 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1993756973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T09:16:43.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.700+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0b4197d10 con 0x7fc0b4100890 2023-08-25T09:16:43.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.700+0000 7fc0aaffd700 1 -- 172.21.15.99:0/888870065 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0b00952f0 con 0x7fc0b4100890 2023-08-25T09:16:43.702 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-08-25T09:16:43.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc0a00a00c0 msgr2=0x7fc0a00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc0a00a00c0 0x7fc0a00a2580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4105fc0 tx=0x7fc0a4005fd0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b4100890 msgr2=0x7fc0b419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b4100890 0x7fc0b419c790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000e9a0 tx=0x7fc0b000ecb0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 shutdown_connections 2023-08-25T09:16:43.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fc0a00a00c0 0x7fc0a00a2580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0b41068a0 0x7fc0b4196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b4100890 0x7fc0b419c790 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 --2- 172.21.15.99:0/888870065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b4101250 0x7fc0b419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 >> 172.21.15.99:0/888870065 conn(0x7fc0b40fc380 msgr2=0x7fc0b40fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:43.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 shutdown_connections 2023-08-25T09:16:43.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.703+0000 7fc0bc1ff700 1 -- 172.21.15.99:0/888870065 wait complete. 2023-08-25T09:16:43.724 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:16:43.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:43 smithi157 ceph-mon[130879]: pgmap v553: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2023-08-25T09:16:43.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:43 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3829765371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T09:16:43.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:43 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1993756973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T09:16:43.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 -- 172.21.15.99:0/903371527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52840ff760 msgr2=0x7f52840ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 --2- 172.21.15.99:0/903371527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52840ff760 0x7f52840ffbe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5278009990 tx=0x7f5278009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 -- 172.21.15.99:0/903371527 shutdown_connections 2023-08-25T09:16:43.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 --2- 172.21.15.99:0/903371527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5284100120 0x7f528410c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 --2- 172.21.15.99:0/903371527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5284105770 0x7f5284105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 --2- 172.21.15.99:0/903371527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52840ff760 0x7f52840ffbe0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 -- 172.21.15.99:0/903371527 >> 172.21.15.99:0/903371527 conn(0x7f52840fb270 msgr2=0x7f52840fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:43.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 -- 172.21.15.99:0/903371527 shutdown_connections 2023-08-25T09:16:43.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.921+0000 7f52889ff700 1 -- 172.21.15.99:0/903371527 wait complete. 2023-08-25T09:16:43.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.922+0000 7f52889ff700 1 Processor -- start 2023-08-25T09:16:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.922+0000 7f52889ff700 1 -- start start 2023-08-25T09:16:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f52889ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52840ff760 0x7f528419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f52889ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5284100120 0x7f528419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f52889ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284105770 0x7f5284198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f52889ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5284199230 con 0x7f5284105770 2023-08-25T09:16:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f52889ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52841993a0 con 0x7f5284100120 2023-08-25T09:16:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f52889ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5284199560 con 0x7f52840ff760 2023-08-25T09:16:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f5282d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284105770 0x7f5284198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f5281d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5284100120 0x7f528419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f5282d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284105770 0x7f5284198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36772/0 (socket says 172.21.15.99:36772) 2023-08-25T09:16:43.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f5282d9d700 1 -- 172.21.15.99:0/353477825 learned_addr learned my addr 172.21.15.99:0/353477825 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:43.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f528259c700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52840ff760 0x7f528419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.923+0000 7f5282d9d700 1 -- 172.21.15.99:0/353477825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52840ff760 msgr2=0x7f528419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f5282d9d700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52840ff760 0x7f528419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f5282d9d700 1 -- 172.21.15.99:0/353477825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5284100120 msgr2=0x7f528419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:43.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f5282d9d700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5284100120 0x7f528419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:43.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f5282d9d700 1 -- 172.21.15.99:0/353477825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780095f0 con 0x7f5284105770 2023-08-25T09:16:43.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f5282d9d700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284105770 0x7f5284198a60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f527c007ee0 tx=0x7f527c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:43.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f52737fe700 1 -- 172.21.15.99:0/353477825 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c017070 con 0x7f5284105770 2023-08-25T09:16:43.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f52737fe700 1 -- 172.21.15.99:0/353477825 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f527c00f040 con 0x7f5284105770 2023-08-25T09:16:43.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5284199840 con 0x7f5284105770 2023-08-25T09:16:43.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f52737fe700 1 -- 172.21.15.99:0/353477825 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c005580 con 0x7f5284105770 2023-08-25T09:16:43.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.924+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52841a5a00 con 0x7f5284105770 2023-08-25T09:16:43.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.925+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52840676a0 con 0x7f5284105770 2023-08-25T09:16:43.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.927+0000 7f52737fe700 1 -- 172.21.15.99:0/353477825 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f527c00db20 con 0x7f5284105770 2023-08-25T09:16:43.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.928+0000 7f52737fe700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f526c0a0140 0x7f526c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:43.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.928+0000 7f52737fe700 1 -- 172.21.15.99:0/353477825 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f527c0cadf0 con 0x7f5284105770 2023-08-25T09:16:43.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.929+0000 7f528259c700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f526c0a0140 0x7f526c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:43.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.929+0000 7f528259c700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f526c0a0140 0x7f526c0a2600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5274009be0 tx=0x7f5274009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:43.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:43.932+0000 7f52737fe700 1 -- 172.21.15.99:0/353477825 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f527c0947d0 con 0x7f5284105770 2023-08-25T09:16:43.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:43 smithi181 ceph-mon[131296]: pgmap v553: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2023-08-25T09:16:43.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:43 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3829765371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T09:16:43.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:43 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1993756973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T09:16:44.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.155+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5284110960 con 0x7f5284105770 2023-08-25T09:16:44.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.156+0000 7f52737fe700 1 -- 172.21.15.99:0/353477825 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v112) v1 ==== 81+0+942 (secure 0 0 0) 0x7f527c097df0 con 0x7f5284105770 2023-08-25T09:16:44.158 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2892690228 2023-08-26T08:59:10.145783+0000 2023-08-25T09:16:44.158 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2769940141 2023-08-26T08:50:54.351659+0000 2023-08-25T09:16:44.158 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/3418985282 2023-08-26T08:59:10.145783+0000 2023-08-25T09:16:44.158 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/124662375 2023-08-26T08:50:54.351659+0000 2023-08-25T09:16:44.159 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/4068336524 2023-08-26T08:59:10.145783+0000 2023-08-25T09:16:44.159 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/483386971 2023-08-26T08:50:54.351659+0000 2023-08-25T09:16:44.159 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/2336486699 2023-08-26T08:50:54.351659+0000 2023-08-25T09:16:44.159 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/1268974739 2023-08-26T08:50:28.397071+0000 2023-08-25T09:16:44.159 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/90449605 2023-08-26T08:59:10.145783+0000 2023-08-25T09:16:44.160 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/4068336524 2023-08-26T08:59:10.145783+0000 2023-08-25T09:16:44.160 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/1871470354 2023-08-26T08:50:28.397071+0000 2023-08-25T09:16:44.160 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/1861696476 2023-08-26T08:59:10.145783+0000 2023-08-25T09:16:44.160 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/3349539225 2023-08-26T08:50:28.397071+0000 2023-08-25T09:16:44.160 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/3349539225 2023-08-26T08:50:28.397071+0000 2023-08-25T09:16:44.161 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2636132577 2023-08-26T08:50:28.397071+0000 2023-08-25T09:16:44.161 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/2336486699 2023-08-26T08:50:54.351659+0000 2023-08-25T09:16:44.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f526c0a0140 msgr2=0x7f526c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:44.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f526c0a0140 0x7f526c0a2600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5274009be0 tx=0x7f5274009260 comp rx=0 tx=0).stop 2023-08-25T09:16:44.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284105770 msgr2=0x7f5284198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:44.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284105770 0x7f5284198a60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f527c007ee0 tx=0x7f527c00d3b0 comp rx=0 tx=0).stop 2023-08-25T09:16:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 shutdown_connections 2023-08-25T09:16:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f526c0a0140 0x7f526c0a2600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284105770 0x7f5284198a60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52840ff760 0x7f528419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 --2- 172.21.15.99:0/353477825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5284100120 0x7f528419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 >> 172.21.15.99:0/353477825 conn(0x7f52840fb270 msgr2=0x7f528410aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 shutdown_connections 2023-08-25T09:16:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:44.158+0000 7f52889ff700 1 -- 172.21.15.99:0/353477825 wait complete. 2023-08-25T09:16:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:listed 16 entries 2023-08-25T09:16:44.178 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-25T09:16:44.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:16:44.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-25T09:16:44.696 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T09:16:44.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:16:44.697 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:16:44.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:16:44.852 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-08-25T09:16:44.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:44 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/888870065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T09:16:44.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:44 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/353477825' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T09:16:44.959 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:16:44.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:16:44.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:16:44.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:44 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/888870065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T09:16:44.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:44 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/353477825' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T09:16:45.035 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:44 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/888870065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T09:16:45.036 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:44 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/353477825' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T09:16:45.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:16:45.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2023-08-25T09:16:45.160 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-08-25T09:16:45.171 INFO:tasks.cephadm:Teardown begin 2023-08-25T09:16:45.172 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:16:45.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:16:45.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.371+0000 7f0712dff700 1 -- 172.21.15.99:0/2830488298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f070c1008a0 msgr2=0x7f070c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:45.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.371+0000 7f0712dff700 1 --2- 172.21.15.99:0/2830488298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f070c1008a0 0x7f070c100d20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f06f80098f0 tx=0x7f06f8009c00 comp rx=0 tx=0).stop 2023-08-25T09:16:45.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.371+0000 7f0712dff700 1 -- 172.21.15.99:0/2830488298 shutdown_connections 2023-08-25T09:16:45.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.371+0000 7f0712dff700 1 --2- 172.21.15.99:0/2830488298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c101260 0x7f070c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.371+0000 7f0712dff700 1 --2- 172.21.15.99:0/2830488298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1068b0 0x7f070c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.371+0000 7f0712dff700 1 --2- 172.21.15.99:0/2830488298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f070c1008a0 0x7f070c100d20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.371+0000 7f0712dff700 1 -- 172.21.15.99:0/2830488298 >> 172.21.15.99:0/2830488298 conn(0x7f070c0fc3d0 msgr2=0x7f070c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:45.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.372+0000 7f0712dff700 1 -- 172.21.15.99:0/2830488298 shutdown_connections 2023-08-25T09:16:45.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.372+0000 7f0712dff700 1 -- 172.21.15.99:0/2830488298 wait complete. 2023-08-25T09:16:45.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0712dff700 1 Processor -- start 2023-08-25T09:16:45.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0712dff700 1 -- start start 2023-08-25T09:16:45.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0712dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c1008a0 0x7f070c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:45.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0712dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f070c101260 0x7f070c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:45.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0712dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1068b0 0x7f070c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:45.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0712dff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c199180 con 0x7f070c1068b0 2023-08-25T09:16:45.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0712dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c1992f0 con 0x7f070c1008a0 2023-08-25T09:16:45.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0712dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c1994b0 con 0x7f070c101260 2023-08-25T09:16:45.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0710b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c1008a0 0x7f070c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:45.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f071139c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1068b0 0x7f070c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:45.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0710b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c1008a0 0x7f070c19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.99:60502/0 (socket says 172.21.15.99:60502) 2023-08-25T09:16:45.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.373+0000 7f0710b9b700 1 -- 172.21.15.99:0/2245154196 learned_addr learned my addr 172.21.15.99:0/2245154196 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:45.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f070bfff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f070c101260 0x7f070c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:45.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0710b9b700 1 -- 172.21.15.99:0/2245154196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f070c101260 msgr2=0x7f070c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:45.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0710b9b700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f070c101260 0x7f070c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0710b9b700 1 -- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1068b0 msgr2=0x7f070c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:45.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0710b9b700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1068b0 0x7f070c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0710b9b700 1 -- 172.21.15.99:0/2245154196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f8009520 con 0x7f070c1008a0 2023-08-25T09:16:45.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0710b9b700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c1008a0 0x7f070c19e9d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f070000e9c0 tx=0x7f070000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:45.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0709ffb700 1 -- 172.21.15.99:0/2245154196 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700004a30 con 0x7f070c1008a0 2023-08-25T09:16:45.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0709ffb700 1 -- 172.21.15.99:0/2245154196 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f070000cc80 con 0x7f070c1008a0 2023-08-25T09:16:45.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0709ffb700 1 -- 172.21.15.99:0/2245154196 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000054e0 con 0x7f070c1008a0 2023-08-25T09:16:45.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c199790 con 0x7f070c1008a0 2023-08-25T09:16:45.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.374+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c1a5950 con 0x7f070c1008a0 2023-08-25T09:16:45.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.375+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c067710 con 0x7f070c1008a0 2023-08-25T09:16:45.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.377+0000 7f0709ffb700 1 -- 172.21.15.99:0/2245154196 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f0700004b90 con 0x7f070c1008a0 2023-08-25T09:16:45.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.378+0000 7f0709ffb700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f06fc0a0100 0x7f06fc0a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:45.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.379+0000 7f0709ffb700 1 -- 172.21.15.99:0/2245154196 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0700014070 con 0x7f070c1008a0 2023-08-25T09:16:45.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.379+0000 7f070bfff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f06fc0a0100 0x7f06fc0a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:45.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.379+0000 7f070bfff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f06fc0a0100 0x7f06fc0a25c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06f8005e90 tx=0x7f06f8019040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:45.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.382+0000 7f0709ffb700 1 -- 172.21.15.99:0/2245154196 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f07000950c0 con 0x7f070c1008a0 2023-08-25T09:16:45.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.626+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f070c110910 con 0x7f070c1008a0 2023-08-25T09:16:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.627+0000 7f0709ffb700 1 -- 172.21.15.99:0/2245154196 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+11286 (secure 0 0 0) 0x7f07000986e0 con 0x7f070c1008a0 2023-08-25T09:16:45.629 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T09:16:45.629 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T08:57:19.244738+0000","modified":"2023-08-25T09:16:00.500538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24547,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26552":{"gid":26552,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/985582767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":985582767},{"type":"v1","addr":"172.21.15.157:6837","nonce":985582767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:16:45.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.629+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f06fc0a0100 msgr2=0x7f06fc0a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:45.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.629+0000 7f0712dff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f06fc0a0100 0x7f06fc0a25c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06f8005e90 tx=0x7f06f8019040 comp rx=0 tx=0).stop 2023-08-25T09:16:45.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c1008a0 msgr2=0x7f070c19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:45.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c1008a0 0x7f070c19e9d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f070000e9c0 tx=0x7f070000ecd0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 shutdown_connections 2023-08-25T09:16:45.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f06fc0a0100 0x7f06fc0a25c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1068b0 0x7f070c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c1008a0 0x7f070c19e9d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 --2- 172.21.15.99:0/2245154196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f070c101260 0x7f070c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 >> 172.21.15.99:0/2245154196 conn(0x7f070c0fc3d0 msgr2=0x7f070c10bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:45.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 shutdown_connections 2023-08-25T09:16:45.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.630+0000 7f0712dff700 1 -- 172.21.15.99:0/2245154196 wait complete. 2023-08-25T09:16:45.634 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2023-08-25T09:16:45.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:16:45.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 -- 172.21.15.99:0/884542012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941068b0 msgr2=0x7f1994106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:45.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 --2- 172.21.15.99:0/884542012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941068b0 0x7f1994106c90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f19840098f0 tx=0x7f1984009c00 comp rx=0 tx=0).stop 2023-08-25T09:16:45.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 -- 172.21.15.99:0/884542012 shutdown_connections 2023-08-25T09:16:45.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 --2- 172.21.15.99:0/884542012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1994101260 0x7f199410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 --2- 172.21.15.99:0/884542012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941068b0 0x7f1994106c90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 --2- 172.21.15.99:0/884542012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19941008a0 0x7f1994100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 -- 172.21.15.99:0/884542012 >> 172.21.15.99:0/884542012 conn(0x7f19940fc3d0 msgr2=0x7f19940fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:45.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 -- 172.21.15.99:0/884542012 shutdown_connections 2023-08-25T09:16:45.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.846+0000 7f19995ff700 1 -- 172.21.15.99:0/884542012 wait complete. 2023-08-25T09:16:45.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.847+0000 7f19995ff700 1 Processor -- start 2023-08-25T09:16:45.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.847+0000 7f19995ff700 1 -- start start 2023-08-25T09:16:45.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f19995ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941008a0 0x7f199419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:45.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f19995ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1994101260 0x7f199419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:45.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f19995ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19941068b0 0x7f1994196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:45.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f19995ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1994196e70 con 0x7f19941068b0 2023-08-25T09:16:45.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f19995ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1994196fe0 con 0x7f1994101260 2023-08-25T09:16:45.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f19995ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941971a0 con 0x7f19941008a0 2023-08-25T09:16:45.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941008a0 0x7f199419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:45.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941008a0 0x7f199419c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:56082/0 (socket says 172.21.15.99:56082) 2023-08-25T09:16:45.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 -- 172.21.15.99:0/1418723062 learned_addr learned my addr 172.21.15.99:0/1418723062 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:45.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f19937fe700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19941068b0 0x7f1994196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:45.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f198bfff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1994101260 0x7f199419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:45.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 -- 172.21.15.99:0/1418723062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1994101260 msgr2=0x7f199419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:45.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1994101260 0x7f199419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 -- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19941068b0 msgr2=0x7f1994196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:45.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19941068b0 0x7f1994196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:45.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 -- 172.21.15.99:0/1418723062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0095f0 con 0x7f19941008a0 2023-08-25T09:16:45.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f198bfff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1994101260 0x7f199419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:16:45.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.848+0000 7f1992ffd700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941008a0 0x7f199419c790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1984009810 tx=0x7f198400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:45.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.849+0000 7f1990ff9700 1 -- 172.21.15.99:0/1418723062 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198401c070 con 0x7f19941008a0 2023-08-25T09:16:45.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.849+0000 7f1990ff9700 1 -- 172.21.15.99:0/1418723062 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f198400bbd0 con 0x7f19941008a0 2023-08-25T09:16:45.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.849+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1984009520 con 0x7f19941008a0 2023-08-25T09:16:45.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.849+0000 7f1990ff9700 1 -- 172.21.15.99:0/1418723062 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840049c0 con 0x7f19941008a0 2023-08-25T09:16:45.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.849+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19941a35a0 con 0x7f19941008a0 2023-08-25T09:16:45.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.850+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1994067710 con 0x7f19941008a0 2023-08-25T09:16:45.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.852+0000 7f1990ff9700 1 -- 172.21.15.99:0/1418723062 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f1984020070 con 0x7f19941008a0 2023-08-25T09:16:45.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.853+0000 7f1990ff9700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f19800a0100 0x7f19800a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:45.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.853+0000 7f1990ff9700 1 -- 172.21.15.99:0/1418723062 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f19840cc840 con 0x7f19941008a0 2023-08-25T09:16:45.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.853+0000 7f198bfff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f19800a0100 0x7f19800a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:45.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.854+0000 7f198bfff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f19800a0100 0x7f19800a25c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f19940fecf0 tx=0x7f197c009310 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:45.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:45.855+0000 7f1990ff9700 1 -- 172.21.15.99:0/1418723062 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1984096250 con 0x7f19941008a0 2023-08-25T09:16:45.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:45 smithi157 ceph-mon[130879]: pgmap v554: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-08-25T09:16:45.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:45 smithi181 ceph-mon[131296]: pgmap v554: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-08-25T09:16:46.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:45 smithi099 ceph-mon[97641]: pgmap v554: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-08-25T09:16:46.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.102+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1994197d10 con 0x7f19941008a0 2023-08-25T09:16:46.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.103+0000 7f1990ff9700 1 -- 172.21.15.99:0/1418723062 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+11286 (secure 0 0 0) 0x7f1984099870 con 0x7f19941008a0 2023-08-25T09:16:46.105 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T09:16:46.106 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2749583644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2749583644},{"type":"v1","addr":"172.21.15.99:6835","nonce":2749583644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2687363843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2687363843},{"type":"v1","addr":"172.21.15.99:6839","nonce":2687363843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1596550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1596550306},{"type":"v1","addr":"172.21.15.99:6841","nonce":1596550306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/829780458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":829780458},{"type":"v1","addr":"172.21.15.181:6833","nonce":829780458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3715294590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3715294590},{"type":"v1","addr":"172.21.15.157:6833","nonce":3715294590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/654200682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":654200682},{"type":"v1","addr":"172.21.15.99:6837","nonce":654200682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T08:57:19.244738+0000","modified":"2023-08-25T09:16:00.500538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24547,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/1963370106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1963370106},{"type":"v1","addr":"172.21.15.157:6835","nonce":1963370106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/3995966648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3995966648},{"type":"v1","addr":"172.21.15.181:6835","nonce":3995966648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2291370984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2291370984},{"type":"v1","addr":"172.21.15.181:6837","nonce":2291370984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/2903255438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2903255438},{"type":"v1","addr":"172.21.15.181:6839","nonce":2903255438}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6839/1062424301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1062424301},{"type":"v1","addr":"172.21.15.157:6839","nonce":1062424301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26552":{"gid":26552,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/985582767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":985582767},{"type":"v1","addr":"172.21.15.157:6837","nonce":985582767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:16:46.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f19800a0100 msgr2=0x7f19800a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f19800a0100 0x7f19800a25c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f19940fecf0 tx=0x7f197c009310 comp rx=0 tx=0).stop 2023-08-25T09:16:46.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941008a0 msgr2=0x7f199419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941008a0 0x7f199419c790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1984009810 tx=0x7f198400fa00 comp rx=0 tx=0).stop 2023-08-25T09:16:46.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 shutdown_connections 2023-08-25T09:16:46.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f19800a0100 0x7f19800a25c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19941068b0 0x7f1994196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19941008a0 0x7f199419c790 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 --2- 172.21.15.99:0/1418723062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1994101260 0x7f199419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.105+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 >> 172.21.15.99:0/1418723062 conn(0x7f19940fc3d0 msgr2=0x7f19940fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:46.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.106+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 shutdown_connections 2023-08-25T09:16:46.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.106+0000 7f19995ff700 1 -- 172.21.15.99:0/1418723062 wait complete. 2023-08-25T09:16:46.110 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2023-08-25T09:16:46.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:16:46.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 -- 172.21.15.99:0/1374993984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 msgr2=0x7f110810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 --2- 172.21.15.99:0/1374993984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 0x7f110810d770 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f110000b0a0 tx=0x7f110000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 -- 172.21.15.99:0/1374993984 shutdown_connections 2023-08-25T09:16:46.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 --2- 172.21.15.99:0/1374993984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 0x7f110810d770 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 --2- 172.21.15.99:0/1374993984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11081068b0 0x7f1108106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 --2- 172.21.15.99:0/1374993984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11081008a0 0x7f1108100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 -- 172.21.15.99:0/1374993984 >> 172.21.15.99:0/1374993984 conn(0x7f11080fc3d0 msgr2=0x7f11080fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:46.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 -- 172.21.15.99:0/1374993984 shutdown_connections 2023-08-25T09:16:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.325+0000 7f110cbff700 1 -- 172.21.15.99:0/1374993984 wait complete. 2023-08-25T09:16:46.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.326+0000 7f110cbff700 1 Processor -- start 2023-08-25T09:16:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.326+0000 7f110cbff700 1 -- start start 2023-08-25T09:16:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110cbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11081008a0 0x7f110819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110cbff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 0x7f110819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110cbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11081068b0 0x7f1108198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110cbff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1108199120 con 0x7f1108101260 2023-08-25T09:16:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110cbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1108199290 con 0x7f11081068b0 2023-08-25T09:16:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110cbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1108199450 con 0x7f11081008a0 2023-08-25T09:16:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 0x7f110819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11081008a0 0x7f110819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11081008a0 0x7f110819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:56096/0 (socket says 172.21.15.99:56096) 2023-08-25T09:16:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f110659c700 1 -- 172.21.15.99:0/3309158584 learned_addr learned my addr 172.21.15.99:0/3309158584 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f1106d9d700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11081068b0 0x7f1108198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f1105d9b700 1 -- 172.21.15.99:0/3309158584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11081008a0 msgr2=0x7f110819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f1105d9b700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11081008a0 0x7f110819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f1105d9b700 1 -- 172.21.15.99:0/3309158584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11081068b0 msgr2=0x7f1108198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f1105d9b700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11081068b0 0x7f1108198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f1105d9b700 1 -- 172.21.15.99:0/3309158584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1100009d00 con 0x7f1108101260 2023-08-25T09:16:46.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.327+0000 7f1106d9d700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11081068b0 0x7f1108198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:16:46.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.328+0000 7f1105d9b700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 0x7f110819eeb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00e9f0 tx=0x7f10fc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:46.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.328+0000 7f10f77fe700 1 -- 172.21.15.99:0/3309158584 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00eee0 con 0x7f1108101260 2023-08-25T09:16:46.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.328+0000 7f10f77fe700 1 -- 172.21.15.99:0/3309158584 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10fc00cbd0 con 0x7f1108101260 2023-08-25T09:16:46.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.328+0000 7f10f77fe700 1 -- 172.21.15.99:0/3309158584 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc018680 con 0x7f1108101260 2023-08-25T09:16:46.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.328+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1108199730 con 0x7f1108101260 2023-08-25T09:16:46.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.328+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11081a58f0 con 0x7f1108101260 2023-08-25T09:16:46.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.329+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1108067710 con 0x7f1108101260 2023-08-25T09:16:46.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.331+0000 7f10f77fe700 1 -- 172.21.15.99:0/3309158584 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f10fc00cd40 con 0x7f1108101260 2023-08-25T09:16:46.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.332+0000 7f10f77fe700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f10f00a0180 0x7f10f00a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:46.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.332+0000 7f10f77fe700 1 -- 172.21.15.99:0/3309158584 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f10fc014070 con 0x7f1108101260 2023-08-25T09:16:46.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.332+0000 7f110659c700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f10f00a0180 0x7f10f00a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:46.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.333+0000 7f110659c700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f10f00a0180 0x7f10f00a2640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009ca0 tx=0x7f10f8009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:46.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.336+0000 7f10f77fe700 1 -- 172.21.15.99:0/3309158584 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f10fc051e00 con 0x7f1108101260 2023-08-25T09:16:46.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.543+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1108110ac0 con 0x7f1108101260 2023-08-25T09:16:46.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.545+0000 7f10f77fe700 1 -- 172.21.15.99:0/3309158584 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+23210 (secure 0 0 0) 0x7f10fc0951e0 con 0x7f1108101260 2023-08-25T09:16:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-08-25T09:16:46.547 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":112,"fsid":"3d4188ec-4324-11ee-9b3d-001a4aab830c","created":"2023-08-25T08:49:52.251496+0000","modified":"2023-08-25T09:16:14.347830+0000","last_up_change":"2023-08-25T08:56:30.922558+0000","last_in_change":"2023-08-25T08:56:14.354624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T08:53:08.946525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T08:57:17.389400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T08:57:18.695578+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fd38b95e-eec4-4af7-91aa-73e6e78584e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6803","nonce":3269986106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6805","nonce":3269986106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6809","nonce":3269986106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3269986106},{"type":"v1","addr":"172.21.15.99:6807","nonce":3269986106}]},"public_addr":"172.21.15.99:6803/3269986106","cluster_addr":"172.21.15.99:6805/3269986106","heartbeat_back_addr":"172.21.15.99:6809/3269986106","heartbeat_front_addr":"172.21.15.99:6807/3269986106","state":["exists","up"]},{"osd":1,"uuid":"6b212381-183b-4878-be8a-e088b3e62122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6801","nonce":2722577134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6803","nonce":2722577134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6807","nonce":2722577134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2722577134},{"type":"v1","addr":"172.21.15.157:6805","nonce":2722577134}]},"public_addr":"172.21.15.157:6801/2722577134","cluster_addr":"172.21.15.157:6803/2722577134","heartbeat_back_addr":"172.21.15.157:6807/2722577134","heartbeat_front_addr":"172.21.15.157:6805/2722577134","state":["exists","up"]},{"osd":2,"uuid":"2577a270-c667-48bf-9382-d1d5d2ec37df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6801","nonce":4155797556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6803","nonce":4155797556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6807","nonce":4155797556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4155797556},{"type":"v1","addr":"172.21.15.181:6805","nonce":4155797556}]},"public_addr":"172.21.15.181:6801/4155797556","cluster_addr":"172.21.15.181:6803/4155797556","heartbeat_back_addr":"172.21.15.181:6807/4155797556","heartbeat_front_addr":"172.21.15.181:6805/4155797556","state":["exists","up"]},{"osd":3,"uuid":"ae5e672d-2f3e-4afd-8207-bcc4f2aa6530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6811","nonce":2656518905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6813","nonce":2656518905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6817","nonce":2656518905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2656518905},{"type":"v1","addr":"172.21.15.99:6815","nonce":2656518905}]},"public_addr":"172.21.15.99:6811/2656518905","cluster_addr":"172.21.15.99:6813/2656518905","heartbeat_back_addr":"172.21.15.99:6817/2656518905","heartbeat_front_addr":"172.21.15.99:6815/2656518905","state":["exists","up"]},{"osd":4,"uuid":"91518e8f-f259-4741-9f5b-17b1e1d1b662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6809","nonce":488498023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6811","nonce":488498023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6815","nonce":488498023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":488498023},{"type":"v1","addr":"172.21.15.157:6813","nonce":488498023}]},"public_addr":"172.21.15.157:6809/488498023","cluster_addr":"172.21.15.157:6811/488498023","heartbeat_back_addr":"172.21.15.157:6815/488498023","heartbeat_front_addr":"172.21.15.157:6813/488498023","state":["exists","up"]},{"osd":5,"uuid":"c17bcccb-4491-4d2c-8864-f2e744438d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6809","nonce":1806112824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6811","nonce":1806112824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6815","nonce":1806112824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1806112824},{"type":"v1","addr":"172.21.15.181:6813","nonce":1806112824}]},"public_addr":"172.21.15.181:6809/1806112824","cluster_addr":"172.21.15.181:6811/1806112824","heartbeat_back_addr":"172.21.15.181:6815/1806112824","heartbeat_front_addr":"172.21.15.181:6813/1806112824","state":["exists","up"]},{"osd":6,"uuid":"8629c9d9-a959-4532-94c1-9b923a0760a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6819","nonce":2954530435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6821","nonce":2954530435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6825","nonce":2954530435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2954530435},{"type":"v1","addr":"172.21.15.99:6823","nonce":2954530435}]},"public_addr":"172.21.15.99:6819/2954530435","cluster_addr":"172.21.15.99:6821/2954530435","heartbeat_back_addr":"172.21.15.99:6825/2954530435","heartbeat_front_addr":"172.21.15.99:6823/2954530435","state":["exists","up"]},{"osd":7,"uuid":"c726dc23-b1c2-4bc2-9c89-85b26fc9f3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6817","nonce":3957644340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6819","nonce":3957644340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6823","nonce":3957644340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3957644340},{"type":"v1","addr":"172.21.15.157:6821","nonce":3957644340}]},"public_addr":"172.21.15.157:6817/3957644340","cluster_addr":"172.21.15.157:6819/3957644340","heartbeat_back_addr":"172.21.15.157:6823/3957644340","heartbeat_front_addr":"172.21.15.157:6821/3957644340","state":["exists","up"]},{"osd":8,"uuid":"91174f3e-a7c9-496c-a2c7-bdf90a0d59f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6817","nonce":411715472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6819","nonce":411715472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6823","nonce":411715472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":411715472},{"type":"v1","addr":"172.21.15.181:6821","nonce":411715472}]},"public_addr":"172.21.15.181:6817/411715472","cluster_addr":"172.21.15.181:6819/411715472","heartbeat_back_addr":"172.21.15.181:6823/411715472","heartbeat_front_addr":"172.21.15.181:6821/411715472","state":["exists","up"]},{"osd":9,"uuid":"7256a919-8afa-4675-867e-880a61ae8a4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6827","nonce":797626209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6829","nonce":797626209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6833","nonce":797626209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":797626209},{"type":"v1","addr":"172.21.15.99:6831","nonce":797626209}]},"public_addr":"172.21.15.99:6827/797626209","cluster_addr":"172.21.15.99:6829/797626209","heartbeat_back_addr":"172.21.15.99:6833/797626209","heartbeat_front_addr":"172.21.15.99:6831/797626209","state":["exists","up"]},{"osd":10,"uuid":"8274d02b-096a-4240-ac33-738e885e7451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6825","nonce":3880870693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6827","nonce":3880870693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6831","nonce":3880870693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3880870693},{"type":"v1","addr":"172.21.15.157:6829","nonce":3880870693}]},"public_addr":"172.21.15.157:6825/3880870693","cluster_addr":"172.21.15.157:6827/3880870693","heartbeat_back_addr":"172.21.15.157:6831/3880870693","heartbeat_front_addr":"172.21.15.157:6829/3880870693","state":["exists","up"]},{"osd":11,"uuid":"4f3eeca9-42b8-49b0-bc5c-ac8e741d48f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6825","nonce":1448660331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6827","nonce":1448660331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6831","nonce":1448660331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1448660331},{"type":"v1","addr":"172.21.15.181:6829","nonce":1448660331}]},"public_addr":"172.21.15.181:6825/1448660331","cluster_addr":"172.21.15.181:6827/1448660331","heartbeat_back_addr":"172.21.15.181:6831/1448660331","heartbeat_front_addr":"172.21.15.181:6829/1448660331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:24.462945+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:52:45.883106+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:06.795621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:33.058902+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:53:54.892290+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:15.339804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:54:40.880691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:01.847187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:22.980576+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:55:47.833309+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:09.397561+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T08:56:29.808948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/2892690228":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/2769940141":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/3418985282":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/124662375":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6800/4068336524":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/483386971":"2023-08-26T08:50:54.351659+0000","172.21.15.99:6801/2336486699":"2023-08-26T08:50:54.351659+0000","172.21.15.99:0/1268974739":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/90449605":"2023-08-26T08:59:10.145783+0000","172.21.15.99:6801/4068336524":"2023-08-26T08:59:10.145783+0000","172.21.15.99:0/1871470354":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/1861696476":"2023-08-26T08:59:10.145783+0000","172.21.15.99:6801/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/3349539225":"2023-08-26T08:50:28.397071+0000","172.21.15.99:0/2636132577":"2023-08-26T08:50:28.397071+0000","172.21.15.99:6800/2336486699":"2023-08-26T08:50:54.351659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:16:46.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f10f00a0180 msgr2=0x7f10f00a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f10f00a0180 0x7f10f00a2640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009ca0 tx=0x7f10f8009260 comp rx=0 tx=0).stop 2023-08-25T09:16:46.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 msgr2=0x7f110819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 0x7f110819eeb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00e9f0 tx=0x7f10fc00ed00 comp rx=0 tx=0).stop 2023-08-25T09:16:46.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 shutdown_connections 2023-08-25T09:16:46.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f10f00a0180 0x7f10f00a2640 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11081068b0 0x7f1108198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11081008a0 0x7f110819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 --2- 172.21.15.99:0/3309158584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1108101260 0x7f110819eeb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.547+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 >> 172.21.15.99:0/3309158584 conn(0x7f11080fc3d0 msgr2=0x7f110810bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:46.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.548+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 shutdown_connections 2023-08-25T09:16:46.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.548+0000 7f110cbff700 1 -- 172.21.15.99:0/3309158584 wait complete. 2023-08-25T09:16:46.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:16:46.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 -- 172.21.15.99:0/519575336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048101250 msgr2=0x7f804810d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 --2- 172.21.15.99:0/519575336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048101250 0x7f804810d760 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8040009f90 tx=0x7f804000b330 comp rx=0 tx=0).stop 2023-08-25T09:16:46.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 -- 172.21.15.99:0/519575336 shutdown_connections 2023-08-25T09:16:46.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 --2- 172.21.15.99:0/519575336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048101250 0x7f804810d760 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 --2- 172.21.15.99:0/519575336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80481068a0 0x7f8048106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 --2- 172.21.15.99:0/519575336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048100890 0x7f8048100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 -- 172.21.15.99:0/519575336 >> 172.21.15.99:0/519575336 conn(0x7f80480fc380 msgr2=0x7f80480fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:46.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 -- 172.21.15.99:0/519575336 shutdown_connections 2023-08-25T09:16:46.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.767+0000 7f804f9ff700 1 -- 172.21.15.99:0/519575336 wait complete. 2023-08-25T09:16:46.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.769+0000 7f804f9ff700 1 Processor -- start 2023-08-25T09:16:46.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.769+0000 7f804f9ff700 1 -- start start 2023-08-25T09:16:46.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804f9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048100890 0x7f804819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:46.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804f9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048101250 0x7f804819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:46.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804f9ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80481068a0 0x7f80481966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:46.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804f9ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8048196d50 con 0x7f80481068a0 2023-08-25T09:16:46.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804f9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8048196ec0 con 0x7f8048101250 2023-08-25T09:16:46.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804f9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8048197080 con 0x7f8048100890 2023-08-25T09:16:46.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804d79b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048100890 0x7f804819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:46.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804cf9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048101250 0x7f804819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:46.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804d79b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048100890 0x7f804819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.99:56102/0 (socket says 172.21.15.99:56102) 2023-08-25T09:16:46.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804d79b700 1 -- 172.21.15.99:0/1351284145 learned_addr learned my addr 172.21.15.99:0/1351284145 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:46.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804df9c700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80481068a0 0x7f80481966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:46.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804df9c700 1 -- 172.21.15.99:0/1351284145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048100890 msgr2=0x7f804819c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804df9c700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048100890 0x7f804819c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804df9c700 1 -- 172.21.15.99:0/1351284145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048101250 msgr2=0x7f804819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:46.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804df9c700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048101250 0x7f804819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:46.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804df9c700 1 -- 172.21.15.99:0/1351284145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440095f0 con 0x7f80481068a0 2023-08-25T09:16:46.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.770+0000 7f804d79b700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048100890 0x7f804819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:46.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.771+0000 7f804df9c700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80481068a0 0x7f80481966e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f804000b6c0 tx=0x7f8040004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:46.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.771+0000 7f803e7fc700 1 -- 172.21.15.99:0/1351284145 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804000e040 con 0x7f80481068a0 2023-08-25T09:16:46.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.771+0000 7f803e7fc700 1 -- 172.21.15.99:0/1351284145 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8040004e30 con 0x7f80481068a0 2023-08-25T09:16:46.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.771+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8040009c40 con 0x7f80481068a0 2023-08-25T09:16:46.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.771+0000 7f803e7fc700 1 -- 172.21.15.99:0/1351284145 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400052a0 con 0x7f80481068a0 2023-08-25T09:16:46.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.771+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80481a3480 con 0x7f80481068a0 2023-08-25T09:16:46.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.772+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80480676a0 con 0x7f80481068a0 2023-08-25T09:16:46.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.774+0000 7f803e7fc700 1 -- 172.21.15.99:0/1351284145 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f804001d470 con 0x7f80481068a0 2023-08-25T09:16:46.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.775+0000 7f803e7fc700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f80340a0140 0x7f80340a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:46.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.775+0000 7f803e7fc700 1 -- 172.21.15.99:0/1351284145 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f80400ccae0 con 0x7f80481068a0 2023-08-25T09:16:46.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.776+0000 7f804d79b700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f80340a0140 0x7f80340a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:46.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.776+0000 7f804d79b700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f80340a0140 0x7f80340a2600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8044005d10 tx=0x7f8044009310 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:46.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:46.779+0000 7f803e7fc700 1 -- 172.21.15.99:0/1351284145 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f80400964f0 con 0x7f80481068a0 2023-08-25T09:16:46.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:46 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/2245154196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T09:16:46.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:46 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1418723062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T09:16:46.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:46 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3309158584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:46.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:46 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/2245154196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T09:16:46.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:46 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1418723062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T09:16:46.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:46 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3309158584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:47.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.030+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8048197bf0 con 0x7f80481068a0 2023-08-25T09:16:47.034 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:46 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/2245154196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T09:16:47.034 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:46 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1418723062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T09:16:47.034 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:46 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3309158584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T09:16:47.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.037+0000 7f803e7fc700 1 -- 172.21.15.99:0/1351284145 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f804002d2f0 con 0x7f80481068a0 2023-08-25T09:16:47.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f80340a0140 msgr2=0x7f80340a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:47.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f80340a0140 0x7f80340a2600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8044005d10 tx=0x7f8044009310 comp rx=0 tx=0).stop 2023-08-25T09:16:47.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80481068a0 msgr2=0x7f80481966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:47.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80481068a0 0x7f80481966e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f804000b6c0 tx=0x7f8040004a30 comp rx=0 tx=0).stop 2023-08-25T09:16:47.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 shutdown_connections 2023-08-25T09:16:47.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f80340a0140 0x7f80340a2600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80481068a0 0x7f80481966e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8048100890 0x7f804819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 --2- 172.21.15.99:0/1351284145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048101250 0x7f804819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 >> 172.21.15.99:0/1351284145 conn(0x7f80480fc380 msgr2=0x7f80481043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:47.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 shutdown_connections 2023-08-25T09:16:47.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.040+0000 7f804f9ff700 1 -- 172.21.15.99:0/1351284145 wait complete. 2023-08-25T09:16:47.044 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:16:47.061 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:16:47.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/3721759617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b4100890 msgr2=0x7fb7b4100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:47.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/3721759617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b4100890 0x7fb7b4100d10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009990 tx=0x7fb7a8009ca0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/3721759617 shutdown_connections 2023-08-25T09:16:47.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/3721759617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4101250 0x7fb7b410d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/3721759617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b41068a0 0x7fb7b4106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/3721759617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b4100890 0x7fb7b4100d10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/3721759617 >> 172.21.15.99:0/3721759617 conn(0x7fb7b40fc380 msgr2=0x7fb7b40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:47.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/3721759617 shutdown_connections 2023-08-25T09:16:47.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.259+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/3721759617 wait complete. 2023-08-25T09:16:47.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.260+0000 7fb7ba5ff700 1 Processor -- start 2023-08-25T09:16:47.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7ba5ff700 1 -- start start 2023-08-25T09:16:47.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7ba5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4100890 0x7fb7b4072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:47.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7ba5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b4101250 0x7fb7b4073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:47.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7ba5ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b41068a0 0x7fb7b41a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:47.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7ba5ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4073990 con 0x7fb7b41068a0 2023-08-25T09:16:47.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7ba5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41a5080 con 0x7fb7b4101250 2023-08-25T09:16:47.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7ba5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41a51c0 con 0x7fb7b4100890 2023-08-25T09:16:47.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7b8b9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b41068a0 0x7fb7b41a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:47.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7b3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4100890 0x7fb7b4072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:47.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7b37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b4101250 0x7fb7b4073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:47.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7b8b9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b41068a0 0x7fb7b41a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38038/0 (socket says 172.21.15.99:38038) 2023-08-25T09:16:47.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.261+0000 7fb7b8b9c700 1 -- 172.21.15.99:0/529378788 learned_addr learned my addr 172.21.15.99:0/529378788 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:47.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b3fff700 1 -- 172.21.15.99:0/529378788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b4101250 msgr2=0x7fb7b4073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:47.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b3fff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b4101250 0x7fb7b4073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b3fff700 1 -- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b41068a0 msgr2=0x7fb7b41a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:47.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b3fff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b41068a0 0x7fb7b41a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:47.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b3fff700 1 -- 172.21.15.99:0/529378788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80095f0 con 0x7fb7b4100890 2023-08-25T09:16:47.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b37fe700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b4101250 0x7fb7b4073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:47.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b8b9c700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b41068a0 0x7fb7b41a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:47.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b3fff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4100890 0x7fb7b4072c20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e9f0 tx=0x7fb7a400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:47.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b17fa700 1 -- 172.21.15.99:0/529378788 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400c9c0 con 0x7fb7b4100890 2023-08-25T09:16:47.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b17fa700 1 -- 172.21.15.99:0/529378788 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7a4010430 con 0x7fb7b4100890 2023-08-25T09:16:47.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b41a5420 con 0x7fb7b4100890 2023-08-25T09:16:47.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7b17fa700 1 -- 172.21.15.99:0/529378788 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4018680 con 0x7fb7b4100890 2023-08-25T09:16:47.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.262+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b41a5900 con 0x7fb7b4100890 2023-08-25T09:16:47.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.263+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b40676a0 con 0x7fb7b4100890 2023-08-25T09:16:47.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.265+0000 7fb7b17fa700 1 -- 172.21.15.99:0/529378788 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fb7a4010860 con 0x7fb7b4100890 2023-08-25T09:16:47.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.267+0000 7fb7b17fa700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb79c0a00c0 0x7fb79c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:47.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.267+0000 7fb7b17fa700 1 -- 172.21.15.99:0/529378788 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8843+0+0 (secure 0 0 0) 0x7fb7a4014070 con 0x7fb7b4100890 2023-08-25T09:16:47.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.267+0000 7fb7b37fe700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb79c0a00c0 0x7fb79c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:47.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.268+0000 7fb7b37fe700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb79c0a00c0 0x7fb79c0a2580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800b2e0 tx=0x7fb7a8005960 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:47.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.269+0000 7fb7b17fa700 1 -- 172.21.15.99:0/529378788 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb7a40951e0 con 0x7fb7b4100890 2023-08-25T09:16:47.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-a[97616]: 2023-08-25T09:16:47.031+0000 7fd31b090700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:16:47.380 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-b[155907]: ignoring --setuser ceph since I am not root 2023-08-25T09:16:47.380 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-b[155907]: ignoring --setgroup ceph since I am not root 2023-08-25T09:16:47.381 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mds[155931]: -- 172.21.15.157:0/1981818480 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cba9f3b1e0 con 0x55cba9f10c00 2023-08-25T09:16:47.381 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-b[155907]: starting mds.b at 2023-08-25T09:16:47.381 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-e[155225]: ignoring --setuser ceph since I am not root 2023-08-25T09:16:47.382 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-e[155225]: ignoring --setgroup ceph since I am not root 2023-08-25T09:16:47.382 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mds[155249]: -- 172.21.15.157:0/915835008 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e9a61c51e0 con 0x55e9a619a800 2023-08-25T09:16:47.382 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-e[155225]: starting mds.e at 2023-08-25T09:16:47.382 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[162818]: ignoring --setuser ceph since I am not root 2023-08-25T09:16:47.383 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[162818]: ignoring --setgroup ceph since I am not root 2023-08-25T09:16:47.383 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mds[162844]: -- 172.21.15.157:0/1673678546 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0ea1571e0 con 0x55d0ea12d000 2023-08-25T09:16:47.383 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[162818]: starting mds.h at 2023-08-25T09:16:47.494 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-c[157078]: ignoring --setuser ceph since I am not root 2023-08-25T09:16:47.494 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-c[157078]: ignoring --setgroup ceph since I am not root 2023-08-25T09:16:47.494 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mds[157102]: -- 172.21.15.181:0/1004039602 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646d34871e0 con 0x5646d345c800 2023-08-25T09:16:47.495 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-c[157078]: starting mds.c at 2023-08-25T09:16:47.496 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-f[156735]: ignoring --setuser ceph since I am not root 2023-08-25T09:16:47.496 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-f[156735]: ignoring --setgroup ceph since I am not root 2023-08-25T09:16:47.496 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mds[156759]: -- 172.21.15.181:0/1012487516 <== mon.2 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ad5c1891e0 con 0x55ad5c15f000 2023-08-25T09:16:47.497 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-f[156735]: starting mds.f at 2023-08-25T09:16:47.497 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-i[157419]: ignoring --setuser ceph since I am not root 2023-08-25T09:16:47.498 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-i[157419]: ignoring --setgroup ceph since I am not root 2023-08-25T09:16:47.498 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mds[157443]: -- 172.21.15.181:0/2973242154 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56537cb0d1e0 con 0x56537cae2c00 2023-08-25T09:16:47.499 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-i[157419]: starting mds.i at 2023-08-25T09:16:47.499 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-i[157419]: 2023-08-25T09:16:47.096+0000 7f3fc2a5b700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6836/0: (98) Address already in use 2023-08-25T09:16:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:47.523+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb7b4110a40 con 0x7fb7b4100890 2023-08-25T09:16:47.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: pgmap v555: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 410 KiB/s wr, 38 op/s 2023-08-25T09:16:47.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1351284145' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-25T09:16:47.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-25T09:16:47.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:16:47.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: osdmap e113: 12 total, 12 up, 12 in 2023-08-25T09:16:47.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/1351284145' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-25T09:16:47.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-08-25T09:16:47.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/529378788' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T09:16:47.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T09:16:47.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:47 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:16:47.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: pgmap v555: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 410 KiB/s wr, 38 op/s 2023-08-25T09:16:47.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1351284145' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-25T09:16:47.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-25T09:16:47.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:16:47.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: osdmap e113: 12 total, 12 up, 12 in 2023-08-25T09:16:47.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/1351284145' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-25T09:16:47.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-08-25T09:16:47.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/529378788' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T09:16:47.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T09:16:47.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:47 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:16:48.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.039+0000 7fb7b17fa700 1 -- 172.21.15.99:0/529378788 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb7a4098800 con 0x7fb7b4100890 2023-08-25T09:16:48.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb79c0a00c0 msgr2=0x7fb79c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:48.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb79c0a00c0 0x7fb79c0a2580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800b2e0 tx=0x7fb7a8005960 comp rx=0 tx=0).stop 2023-08-25T09:16:48.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4100890 msgr2=0x7fb7b4072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:48.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4100890 0x7fb7b4072c20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e9f0 tx=0x7fb7a400ed00 comp rx=0 tx=0).stop 2023-08-25T09:16:48.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 shutdown_connections 2023-08-25T09:16:48.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb79c0a00c0 0x7fb79c0a2580 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b41068a0 0x7fb7b41a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4100890 0x7fb7b4072c20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 --2- 172.21.15.99:0/529378788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b4101250 0x7fb7b4073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 >> 172.21.15.99:0/529378788 conn(0x7fb7b40fc380 msgr2=0x7fb7b40fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:48.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 shutdown_connections 2023-08-25T09:16:48.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.042+0000 7fb7ba5ff700 1 -- 172.21.15.99:0/529378788 wait complete. 2023-08-25T09:16:48.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: pgmap v555: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 410 KiB/s wr, 38 op/s 2023-08-25T09:16:48.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1351284145' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-25T09:16:48.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-25T09:16:48.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:16:48.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: osdmap e113: 12 total, 12 up, 12 in 2023-08-25T09:16:48.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/1351284145' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-25T09:16:48.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-08-25T09:16:48.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/529378788' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T09:16:48.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T09:16:48.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:47 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T09:16:48.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:16:48.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.278+0000 7fb61ffff700 1 -- 172.21.15.99:0/2705873523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 msgr2=0x7fb61810d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:48.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.278+0000 7fb61ffff700 1 --2- 172.21.15.99:0/2705873523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 0x7fb61810d740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb61000b0a0 tx=0x7fb61000b3b0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.278+0000 7fb61ffff700 1 -- 172.21.15.99:0/2705873523 shutdown_connections 2023-08-25T09:16:48.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.278+0000 7fb61ffff700 1 --2- 172.21.15.99:0/2705873523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 0x7fb61810d740 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.278+0000 7fb61ffff700 1 --2- 172.21.15.99:0/2705873523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618106880 0x7fb618106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.278+0000 7fb61ffff700 1 --2- 172.21.15.99:0/2705873523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb618100870 0x7fb618100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.278+0000 7fb61ffff700 1 -- 172.21.15.99:0/2705873523 >> 172.21.15.99:0/2705873523 conn(0x7fb6180fc3a0 msgr2=0x7fb6180fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:48.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.279+0000 7fb61ffff700 1 -- 172.21.15.99:0/2705873523 shutdown_connections 2023-08-25T09:16:48.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.279+0000 7fb61ffff700 1 -- 172.21.15.99:0/2705873523 wait complete. 2023-08-25T09:16:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.280+0000 7fb61ffff700 1 Processor -- start 2023-08-25T09:16:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.280+0000 7fb61ffff700 1 -- start start 2023-08-25T09:16:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.280+0000 7fb61ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb618100870 0x7fb61819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:48.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.280+0000 7fb61ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 0x7fb61819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:48.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.280+0000 7fb61ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618106880 0x7fb618198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:48.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61ffff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb618199120 con 0x7fb618101230 2023-08-25T09:16:48.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61ffff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb618199290 con 0x7fb618100870 2023-08-25T09:16:48.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb618199450 con 0x7fb618106880 2023-08-25T09:16:48.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61d59a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 0x7fb61819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:48.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61d59a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 0x7fb61819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38050/0 (socket says 172.21.15.99:38050) 2023-08-25T09:16:48.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61d59a700 1 -- 172.21.15.99:0/3811792325 learned_addr learned my addr 172.21.15.99:0/3811792325 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:48.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61dd9b700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb618100870 0x7fb61819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:48.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61e59c700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618106880 0x7fb618198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:48.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61dd9b700 1 -- 172.21.15.99:0/3811792325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618106880 msgr2=0x7fb618198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:48.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61dd9b700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618106880 0x7fb618198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61dd9b700 1 -- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 msgr2=0x7fb61819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:48.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61dd9b700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 0x7fb61819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:48.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61dd9b700 1 -- 172.21.15.99:0/3811792325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610009d00 con 0x7fb618100870 2023-08-25T09:16:48.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61e59c700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618106880 0x7fb618198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:48.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.281+0000 7fb61d59a700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 0x7fb61819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:16:48.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.282+0000 7fb61dd9b700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb618100870 0x7fb61819e970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb61400e9f0 tx=0x7fb61400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:48.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.282+0000 7fb60effd700 1 -- 172.21.15.99:0/3811792325 <== mon.2 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400c9c0 con 0x7fb618100870 2023-08-25T09:16:48.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.282+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb618199730 con 0x7fb618100870 2023-08-25T09:16:48.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.282+0000 7fb60effd700 1 -- 172.21.15.99:0/3811792325 <== mon.2 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb614010430 con 0x7fb618100870 2023-08-25T09:16:48.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.282+0000 7fb60effd700 1 -- 172.21.15.99:0/3811792325 <== mon.2 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614018680 con 0x7fb618100870 2023-08-25T09:16:48.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.282+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6181a5940 con 0x7fb618100870 2023-08-25T09:16:48.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.283+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb618067af0 con 0x7fb618100870 2023-08-25T09:16:48.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.285+0000 7fb60effd700 1 -- 172.21.15.99:0/3811792325 <== mon.2 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fb614010860 con 0x7fb618100870 2023-08-25T09:16:48.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.286+0000 7fb60effd700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb6040a0140 0x7fb6040a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:48.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.286+0000 7fb60effd700 1 -- 172.21.15.99:0/3811792325 <== mon.2 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8843+0+0 (secure 0 0 0) 0x7fb614014070 con 0x7fb618100870 2023-08-25T09:16:48.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.287+0000 7fb61d59a700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb6040a0140 0x7fb6040a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:48.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.287+0000 7fb61d59a700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb6040a0140 0x7fb6040a2600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb61819a180 tx=0x7fb608009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:48.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.290+0000 7fb60effd700 1 -- 172.21.15.99:0/3811792325 <== mon.2 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb614051e00 con 0x7fb618100870 2023-08-25T09:16:48.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:48.503+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb6180008d0 con 0x7fb618100870 2023-08-25T09:16:48.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-25T09:16:48.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-25T09:16:48.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: Cluster is now healthy 2023-08-25T09:16:48.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6834/1543732207,v1:172.21.15.157:6835/1543732207] up:boot 2023-08-25T09:16:48.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6834/355054271,v1:172.21.15.181:6835/355054271] up:boot 2023-08-25T09:16:48.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6837/3012800952,v1:172.21.15.181:6839/3012800952] up:boot 2023-08-25T09:16:48.881 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6836/3628358732,v1:172.21.15.157:6837/3628358732] up:boot 2023-08-25T09:16:48.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.157:6838/3963514040,v1:172.21.15.157:6839/3963514040] up:boot 2023-08-25T09:16:48.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: mds.? [v2:172.21.15.181:6836/4273543020,v1:172.21.15.181:6838/4273543020] up:boot 2023-08-25T09:16:48.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-25T09:16:48.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: fsmap 12 up:standby 2023-08-25T09:16:48.882 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T09:16:48.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T09:16:48.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T09:16:48.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T09:16:48.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:16:48.883 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T09:16:48.884 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:16:48.884 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:16:48.884 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:16:48.884 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3811792325' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:48.884 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:48 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:48.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-25T09:16:48.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-25T09:16:48.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: Cluster is now healthy 2023-08-25T09:16:48.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6834/1543732207,v1:172.21.15.157:6835/1543732207] up:boot 2023-08-25T09:16:48.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6834/355054271,v1:172.21.15.181:6835/355054271] up:boot 2023-08-25T09:16:48.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6837/3012800952,v1:172.21.15.181:6839/3012800952] up:boot 2023-08-25T09:16:48.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6836/3628358732,v1:172.21.15.157:6837/3628358732] up:boot 2023-08-25T09:16:48.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.157:6838/3963514040,v1:172.21.15.157:6839/3963514040] up:boot 2023-08-25T09:16:48.995 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: mds.? [v2:172.21.15.181:6836/4273543020,v1:172.21.15.181:6838/4273543020] up:boot 2023-08-25T09:16:48.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-25T09:16:48.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: fsmap 12 up:standby 2023-08-25T09:16:48.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T09:16:48.996 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T09:16:48.997 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T09:16:48.997 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T09:16:48.997 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:16:48.997 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T09:16:48.997 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:16:48.998 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:16:48.998 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:16:48.998 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3811792325' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:48.998 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:48 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:49.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-25T09:16:49.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-25T09:16:49.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: Cluster is now healthy 2023-08-25T09:16:49.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6834/1543732207,v1:172.21.15.157:6835/1543732207] up:boot 2023-08-25T09:16:49.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6834/355054271,v1:172.21.15.181:6835/355054271] up:boot 2023-08-25T09:16:49.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6837/3012800952,v1:172.21.15.181:6839/3012800952] up:boot 2023-08-25T09:16:49.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6836/3628358732,v1:172.21.15.157:6837/3628358732] up:boot 2023-08-25T09:16:49.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.157:6838/3963514040,v1:172.21.15.157:6839/3963514040] up:boot 2023-08-25T09:16:49.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: mds.? [v2:172.21.15.181:6836/4273543020,v1:172.21.15.181:6838/4273543020] up:boot 2023-08-25T09:16:49.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-25T09:16:49.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: fsmap 12 up:standby 2023-08-25T09:16:49.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T09:16:49.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T09:16:49.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T09:16:49.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T09:16:49.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T09:16:49.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T09:16:49.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T09:16:49.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T09:16:49.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='mgr.25202 172.21.15.99:0/3959719768' entity='mgr.x' 2023-08-25T09:16:49.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3811792325' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:49.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:48 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:49.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.431+0000 7fb60effd700 1 -- 172.21.15.99:0/3811792325 <== mon.2 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb614098b10 con 0x7fb618100870 2023-08-25T09:16:49.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb6040a0140 msgr2=0x7fb6040a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:49.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb6040a0140 0x7fb6040a2600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb61819a180 tx=0x7fb608009260 comp rx=0 tx=0).stop 2023-08-25T09:16:49.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb618100870 msgr2=0x7fb61819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:49.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb618100870 0x7fb61819e970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb61400e9f0 tx=0x7fb61400ed00 comp rx=0 tx=0).stop 2023-08-25T09:16:49.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 shutdown_connections 2023-08-25T09:16:49.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7fb6040a0140 0x7fb6040a2600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618106880 0x7fb618198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb618100870 0x7fb61819e970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 --2- 172.21.15.99:0/3811792325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb618101230 0x7fb61819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.435+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 >> 172.21.15.99:0/3811792325 conn(0x7fb6180fc3a0 msgr2=0x7fb61810bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:49.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.436+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 shutdown_connections 2023-08-25T09:16:49.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.436+0000 7fb61ffff700 1 -- 172.21.15.99:0/3811792325 wait complete. 2023-08-25T09:16:49.440 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:16:49.461 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:16:49.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 -- 172.21.15.99:0/692912805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744103480 msgr2=0x7f474410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:49.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 --2- 172.21.15.99:0/692912805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744103480 0x7f474410f990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f473c00b0a0 tx=0x7f473c00b3b0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 -- 172.21.15.99:0/692912805 shutdown_connections 2023-08-25T09:16:49.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 --2- 172.21.15.99:0/692912805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744103480 0x7f474410f990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 --2- 172.21.15.99:0/692912805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4744108ad0 0x7f4744108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 --2- 172.21.15.99:0/692912805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4744102ac0 0x7f4744102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 -- 172.21.15.99:0/692912805 >> 172.21.15.99:0/692912805 conn(0x7f47440fe570 msgr2=0x7f4744100990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:49.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 -- 172.21.15.99:0/692912805 shutdown_connections 2023-08-25T09:16:49.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.680+0000 7f474bbff700 1 -- 172.21.15.99:0/692912805 wait complete. 2023-08-25T09:16:49.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.681+0000 7f474bbff700 1 Processor -- start 2023-08-25T09:16:49.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.681+0000 7f474bbff700 1 -- start start 2023-08-25T09:16:49.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.681+0000 7f474bbff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744102ac0 0x7f474419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:49.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.681+0000 7f474bbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4744103480 0x7f474419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:49.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.681+0000 7f474bbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4744108ad0 0x7f4744198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:49.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474bbff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744198f80 con 0x7f4744102ac0 2023-08-25T09:16:49.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474bbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47441990f0 con 0x7f4744108ad0 2023-08-25T09:16:49.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474bbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47441992b0 con 0x7f4744103480 2023-08-25T09:16:49.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744102ac0 0x7f474419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:49.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744102ac0 0x7f474419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38072/0 (socket says 172.21.15.99:38072) 2023-08-25T09:16:49.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 -- 172.21.15.99:0/3630887382 learned_addr learned my addr 172.21.15.99:0/3630887382 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-08-25T09:16:49.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474919a700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4744103480 0x7f474419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:49.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474a19c700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4744108ad0 0x7f4744198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:49.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 -- 172.21.15.99:0/3630887382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4744103480 msgr2=0x7f474419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:49.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4744103480 0x7f474419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 -- 172.21.15.99:0/3630887382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4744108ad0 msgr2=0x7f4744198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:49.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4744108ad0 0x7f4744198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:49.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 -- 172.21.15.99:0/3630887382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c009d00 con 0x7f4744102ac0 2023-08-25T09:16:49.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.682+0000 7f474999b700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744102ac0 0x7f474419e8a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f474000e9a0 tx=0x7f474000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.683+0000 7f473affd700 1 -- 172.21.15.99:0/3630887382 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740009790 con 0x7f4744102ac0 2023-08-25T09:16:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.683+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4744199590 con 0x7f4744102ac0 2023-08-25T09:16:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.683+0000 7f473affd700 1 -- 172.21.15.99:0/3630887382 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4740010430 con 0x7f4744102ac0 2023-08-25T09:16:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.683+0000 7f473affd700 1 -- 172.21.15.99:0/3630887382 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740018680 con 0x7f4744102ac0 2023-08-25T09:16:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.683+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47441a5890 con 0x7f4744102ac0 2023-08-25T09:16:49.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.684+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47440676a0 con 0x7f4744102ac0 2023-08-25T09:16:49.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.686+0000 7f473affd700 1 -- 172.21.15.99:0/3630887382 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f47400105a0 con 0x7f4744102ac0 2023-08-25T09:16:49.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.687+0000 7f473affd700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f47300a00c0 0x7f47300a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:16:49.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.687+0000 7f473affd700 1 -- 172.21.15.99:0/3630887382 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8289+0+0 (secure 0 0 0) 0x7f4740014070 con 0x7f4744102ac0 2023-08-25T09:16:49.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.687+0000 7f474919a700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f47300a00c0 0x7f47300a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:16:49.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.688+0000 7f474919a700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f47300a00c0 0x7f47300a2580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4744108090 tx=0x7f4734009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:16:49.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.691+0000 7f473affd700 1 -- 172.21.15.99:0/3630887382 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4740051e00 con 0x7f4744102ac0 2023-08-25T09:16:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:49 smithi099 ceph-mon[97641]: pgmap v557: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 410 KiB/s wr, 38 op/s 2023-08-25T09:16:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:49 smithi099 ceph-mon[97641]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T09:16:49.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:49 smithi099 ceph-mon[97641]: osdmap e114: 12 total, 12 up, 12 in 2023-08-25T09:16:49.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:49 smithi157 ceph-mon[130879]: pgmap v557: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 410 KiB/s wr, 38 op/s 2023-08-25T09:16:49.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:49 smithi157 ceph-mon[130879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T09:16:49.880 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:49 smithi157 ceph-mon[130879]: osdmap e114: 12 total, 12 up, 12 in 2023-08-25T09:16:49.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:49.895+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4744199e20 con 0x7f4744102ac0 2023-08-25T09:16:49.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:49 smithi181 ceph-mon[131296]: pgmap v557: 97 pgs: 97 active+clean; 1.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 410 KiB/s wr, 38 op/s 2023-08-25T09:16:49.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:49 smithi181 ceph-mon[131296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T09:16:49.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:49 smithi181 ceph-mon[131296]: osdmap e114: 12 total, 12 up, 12 in 2023-08-25T09:16:50.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.614+0000 7f473affd700 1 -- 172.21.15.99:0/3630887382 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7f47400951d0 con 0x7f4744102ac0 2023-08-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f47300a00c0 msgr2=0x7f47300a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f47300a00c0 0x7f47300a2580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4744108090 tx=0x7f4734009260 comp rx=0 tx=0).stop 2023-08-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744102ac0 msgr2=0x7f474419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:16:50.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744102ac0 0x7f474419e8a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f474000e9a0 tx=0x7f474000ecb0 comp rx=0 tx=0).stop 2023-08-25T09:16:50.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 shutdown_connections 2023-08-25T09:16:50.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:6800/2884087217,v1:172.21.15.99:6801/2884087217] conn(0x7f47300a00c0 0x7f47300a2580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:50.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4744108ad0 0x7f4744198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:50.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4744102ac0 0x7f474419e8a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:50.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 --2- 172.21.15.99:0/3630887382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4744103480 0x7f474419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:16:50.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.619+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 >> 172.21.15.99:0/3630887382 conn(0x7f47440fe570 msgr2=0x7f47440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:16:50.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.620+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 shutdown_connections 2023-08-25T09:16:50.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-08-25T09:16:50.620+0000 7f474bbff700 1 -- 172.21.15.99:0/3630887382 wait complete. 2023-08-25T09:16:50.624 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-08-25T09:16:50.650 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T09:16:50.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T09:16:50.775 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T09:16:50.831 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-08-25T09:16:50.831 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-25T09:16:50.874 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-25T09:16:50.879 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:50 smithi157 ceph-mon[130879]: from='client.? 172.21.15.99:0/3630887382' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:50.891 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-25T09:16:50.908 INFO:tasks.cephadm:Stopping all daemons... 2023-08-25T09:16:50.909 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-08-25T09:16:50.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.a 2023-08-25T09:16:50.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:50 smithi181 ceph-mon[131296]: from='client.? 172.21.15.99:0/3630887382' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:51.017 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:50 smithi099 ceph-mon[97641]: from='client.? 172.21.15.99:0/3630887382' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T09:16:51.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:51 smithi099 systemd[1]: Stopping Ceph mon.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:16:52.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-a[97616]: 2023-08-25T09:16:51.594+0000 7fd32089b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:16:52.013 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-a[97616]: 2023-08-25T09:16:51.594+0000 7fd32089b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-08-25T09:16:52.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:Aug 25 09:16:52 smithi099 bash[156212]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-a 2023-08-25T09:16:52.572 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.a.service' 2023-08-25T09:16:52.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:16:52.675 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-08-25T09:16:52.675 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-08-25T09:16:52.675 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.b 2023-08-25T09:16:53.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:52 smithi157 systemd[1]: Stopping Ceph mon.b for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:16:53.313 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-b[130855]: 2023-08-25T09:16:53.022+0000 7fb041177700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:16:53.313 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-b[130855]: 2023-08-25T09:16:53.022+0000 7fb041177700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-08-25T09:16:53.857 INFO:journalctl@ceph.mon.b.smithi157.stdout:Aug 25 09:16:53 smithi157 bash[164240]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-b 2023-08-25T09:16:53.863 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.b.service' 2023-08-25T09:16:53.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:16:53.917 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-08-25T09:16:53.918 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-08-25T09:16:53.918 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.c 2023-08-25T09:16:54.243 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:53 smithi181 systemd[1]: Stopping Ceph mon.c for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:16:54.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:54 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-c[131272]: 2023-08-25T09:16:54.313+0000 7f44ce706700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:16:54.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:54 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-c[131272]: 2023-08-25T09:16:54.313+0000 7f44ce706700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-08-25T09:16:54.964 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.c.service' 2023-08-25T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:54 smithi181 bash[164832]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mon-c 2023-08-25T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:54 smithi181 systemd[1]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mon.c.service: Succeeded. 2023-08-25T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi181.stdout:Aug 25 09:16:54 smithi181 systemd[1]: Stopped Ceph mon.c for 3d4188ec-4324-11ee-9b3d-001a4aab830c. 2023-08-25T09:16:55.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:16:55.035 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-08-25T09:16:55.035 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-08-25T09:16:55.035 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.x 2023-08-25T09:16:55.563 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:16:55 smithi099 systemd[1]: Stopping Ceph mgr.x for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:16:55.886 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:16:55 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T09:16:55.593+0000 7feda75ae700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:16:55.887 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:16:55 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x[97888]: 2023-08-25T09:16:55.593+0000 7feda75ae700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-08-25T09:16:56.313 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Aug 25 09:16:55 smithi099 bash[156359]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-x 2023-08-25T09:16:56.370 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.x.service' 2023-08-25T09:16:56.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:16:56.460 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-08-25T09:16:56.460 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-08-25T09:16:56.461 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.y 2023-08-25T09:16:56.880 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 09:16:56 smithi157 systemd[1]: Stopping Ceph mgr.y for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:16:57.168 INFO:journalctl@ceph.mgr.y.smithi157.stdout:Aug 25 09:16:57 smithi157 bash[164384]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-y 2023-08-25T09:16:57.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.y.service' 2023-08-25T09:16:57.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:16:57.397 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-08-25T09:16:57.397 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-08-25T09:16:57.397 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.z 2023-08-25T09:16:57.743 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 09:16:57 smithi181 systemd[1]: Stopping Ceph mgr.z for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:16:58.103 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 09:16:58 smithi181 bash[164975]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mgr-z 2023-08-25T09:16:58.288 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Aug 25 09:16:58 smithi181 systemd[1]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-08-25T09:16:58.292 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mgr.z.service' 2023-08-25T09:16:58.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:16:58.328 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-08-25T09:16:58.329 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-08-25T09:16:58.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.0 2023-08-25T09:16:58.813 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 09:16:58 smithi099 systemd[1]: Stopping Ceph osd.0 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:16:59.313 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 09:16:58 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-0[110552]: 2023-08-25T09:16:58.827+0000 7f6cdaf66700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:16:59.313 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 09:16:58 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-0[110552]: 2023-08-25T09:16:58.827+0000 7f6cdaf66700 -1 osd.0 115 *** Got signal Terminated *** 2023-08-25T09:16:59.314 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 09:16:58 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-0[110552]: 2023-08-25T09:16:58.827+0000 7f6cdaf66700 -1 osd.0 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:17:05.063 INFO:journalctl@ceph.osd.0.smithi099.stdout:Aug 25 09:17:04 smithi099 bash[156508]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-0 2023-08-25T09:17:06.617 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.0.service' 2023-08-25T09:17:06.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:17:06.708 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-08-25T09:17:06.708 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-08-25T09:17:06.708 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.1 2023-08-25T09:17:07.072 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 09:17:06 smithi157 systemd[1]: Stopping Ceph osd.1 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:17:07.380 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 09:17:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-1[135921]: 2023-08-25T09:17:07.069+0000 7f68bce8e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:17:07.380 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 09:17:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-1[135921]: 2023-08-25T09:17:07.069+0000 7f68bce8e700 -1 osd.1 115 *** Got signal Terminated *** 2023-08-25T09:17:07.380 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 09:17:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-1[135921]: 2023-08-25T09:17:07.069+0000 7f68bce8e700 -1 osd.1 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:17:13.130 INFO:journalctl@ceph.osd.1.smithi157.stdout:Aug 25 09:17:12 smithi157 bash[164530]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-1 2023-08-25T09:17:14.280 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.1.service' 2023-08-25T09:17:14.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:17:14.321 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-08-25T09:17:14.321 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-08-25T09:17:14.321 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.2 2023-08-25T09:17:14.688 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 09:17:14 smithi181 systemd[1]: Stopping Ceph osd.2 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:17:14.993 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 09:17:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-2[137088]: 2023-08-25T09:17:14.685+0000 7f8a27735700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:17:14.994 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 09:17:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-2[137088]: 2023-08-25T09:17:14.685+0000 7f8a27735700 -1 osd.2 115 *** Got signal Terminated *** 2023-08-25T09:17:14.994 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 09:17:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-2[137088]: 2023-08-25T09:17:14.685+0000 7f8a27735700 -1 osd.2 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:17:20.743 INFO:journalctl@ceph.osd.2.smithi181.stdout:Aug 25 09:17:20 smithi181 bash[165134]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-2 2023-08-25T09:17:22.904 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.2.service' 2023-08-25T09:17:22.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:17:22.945 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-08-25T09:17:22.945 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-08-25T09:17:22.946 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.3 2023-08-25T09:17:23.313 INFO:journalctl@ceph.osd.3.smithi099.stdout:Aug 25 09:17:23 smithi099 systemd[1]: Stopping Ceph osd.3 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:17:23.813 INFO:journalctl@ceph.osd.3.smithi099.stdout:Aug 25 09:17:23 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-3[116007]: 2023-08-25T09:17:23.462+0000 7fde08758700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:17:23.813 INFO:journalctl@ceph.osd.3.smithi099.stdout:Aug 25 09:17:23 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-3[116007]: 2023-08-25T09:17:23.462+0000 7fde08758700 -1 osd.3 115 *** Got signal Terminated *** 2023-08-25T09:17:23.814 INFO:journalctl@ceph.osd.3.smithi099.stdout:Aug 25 09:17:23 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-3[116007]: 2023-08-25T09:17:23.462+0000 7fde08758700 -1 osd.3 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:17:24.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:24 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:24.575+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:25.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:25 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:25.611+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:26.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:26.644+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:27.653 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:27.372+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:27.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:27.651+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:28.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:27 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:27.897+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:28.700 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:28.324+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:28.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:28.698+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:29.219 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:28 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:28.921+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:29.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:28 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:28.882+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:29.493 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:29.353+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:29.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:29.186+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:29.563 INFO:journalctl@ceph.osd.3.smithi099.stdout:Aug 25 09:17:29 smithi099 bash[156796]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-3 2023-08-25T09:17:29.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:29.665+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:30.261 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:29 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:29.881+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:30.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:29 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:29.924+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:30.493 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:30.355+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:30.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:30.216+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:30.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:30.711+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:31.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:30 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:30.841+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:31.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:30 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:30.917+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:31.169 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.3.service' 2023-08-25T09:17:31.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:17:31.263 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-08-25T09:17:31.263 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-08-25T09:17:31.264 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.4 2023-08-25T09:17:31.293 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:30 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:30.973+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:31.626 INFO:journalctl@ceph.osd.4.smithi157.stdout:Aug 25 09:17:31 smithi157 systemd[1]: Stopping Ceph osd.4 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:17:31.743 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:31 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:31.380+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:31.744 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:31 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:31.260+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:31.880 INFO:journalctl@ceph.osd.4.smithi157.stdout:Aug 25 09:17:31 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-4[141296]: 2023-08-25T09:17:31.624+0000 7f47e10ec700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:17:31.880 INFO:journalctl@ceph.osd.4.smithi157.stdout:Aug 25 09:17:31 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-4[141296]: 2023-08-25T09:17:31.624+0000 7f47e10ec700 -1 osd.4 115 *** Got signal Terminated *** 2023-08-25T09:17:31.880 INFO:journalctl@ceph.osd.4.smithi157.stdout:Aug 25 09:17:31 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-4[141296]: 2023-08-25T09:17:31.624+0000 7f47e10ec700 -1 osd.4 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:17:32.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:31 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:31.752+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:32.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:31 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:31.821+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:32.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:31 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:31.904+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:32.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:31 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:31.978+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:32.743 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:32 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:32.418+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:32.744 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:32 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:32.274+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:33.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:32 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:32.777+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:33.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:32 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:32.907+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:33.239 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:32 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:32.758+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:33.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:32 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:32.958+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:33.493 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:33 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:33.390+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:33.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:33 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:33.238+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:33.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:33 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:33.723+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:33.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:33 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:33.723+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:34.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:33.744+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:34.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:33.744+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:34.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:33.894+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:34.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:33 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:33.944+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:34.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:34 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:34.226+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:34.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:34 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:34.406+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:34.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:34 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:34.406+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:34.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:34 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:34.717+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:34.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:34 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:34.717+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:35.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:34 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:34.707+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:35.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:34 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:34.707+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:35.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:34 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:34.921+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:35.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:34 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:34.921+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:35.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:34 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:34.895+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:35.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:35 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:35.234+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:35.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:35 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:35.410+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:35.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:35 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:35.410+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:35.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:35 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:35.708+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:35.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:35 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:35.708+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:36.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:35 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:35.714+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:36.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:35 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:35.714+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:36.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:35 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:35.949+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:36.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:35 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:35.949+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:36.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:35 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:35.835+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:36.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:35 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:35.863+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:36.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:35 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:35.863+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:36.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:36 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:36.212+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:36.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:36 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:36.362+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:36.495 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:36 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:36.362+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:36.979 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:36 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:36.669+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:36.979 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:36 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:36.669+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:36.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:36 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:36.671+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:36.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:36 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:36.671+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:37.119 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:36 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:36.850+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:37.120 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:36 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:36.837+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:37.120 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:36 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:36.837+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:37.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:36 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:36.978+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:37.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:36 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:36.978+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:37.380 INFO:journalctl@ceph.osd.4.smithi157.stdout:Aug 25 09:17:37 smithi157 bash[164798]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-4 2023-08-25T09:17:37.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:37 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:37.203+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:37.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:37 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:37.377+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:37.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:37 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:37.377+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:37.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:37 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:37.694+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:37.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:37 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:37.694+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:37.994 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:37 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:37.671+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:37.995 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:37 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:37.671+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:38.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:37 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:37.830+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:38.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:37 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:37.830+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:38.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:37 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:37.830+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:38.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:37 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:37.993+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:38.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:37 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:37.993+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:38.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:38 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:38.224+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:38.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:38 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:38.406+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:38.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:38 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:38.406+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:38.663 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.4.service' 2023-08-25T09:17:38.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:17:38.742 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-08-25T09:17:38.743 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-08-25T09:17:38.743 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.5 2023-08-25T09:17:38.994 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:38 smithi181 systemd[1]: Stopping Ceph osd.5 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:17:38.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:38 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:38.679+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:38.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:38 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:38.679+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:39.021 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:38 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:38.631+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:39.021 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:38 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:38.631+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:39.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:38 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:38.785+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:39.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:38 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:38.847+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:39.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:38 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:38.847+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:39.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:39 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:39.020+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:39.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:39 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:39.020+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:39.419 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:39 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:39.175+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:39.420 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:39 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:39.098+0000 7fb04eeac700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:17:39.420 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:39 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:39.098+0000 7fb04eeac700 -1 osd.5 115 *** Got signal Terminated *** 2023-08-25T09:17:39.420 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:39 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:39.098+0000 7fb04eeac700 -1 osd.5 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:17:39.743 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:39 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:39.418+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:39.744 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:39 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:39.418+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:39.745 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:39 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:39.644+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:39.745 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:39 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:39.644+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:40.053 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:39 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:39.608+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:40.053 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:39 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:39.608+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:40.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:39 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:39.787+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:40.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:39 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:39.889+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:40.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:39 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:39.889+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:40.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:40 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:40.052+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:40.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:40 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:40.052+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:40.392 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:40 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:40.137+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:40.743 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:40 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:40.391+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:40.744 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:40 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:40.391+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:40.744 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:40 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:40.624+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:40.744 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:40 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:40.624+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:41.026 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:40 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:40.650+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:41.026 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:40 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:40.650+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:41.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:40 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:40.832+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:41.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:40 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:40.867+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:41.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:40 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:40.867+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:41.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:41 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:41.025+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:41.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:41 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:41.025+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:41.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:41 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:41.178+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:41.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:41 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:41.358+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:41.495 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:41 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:41.358+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:41.495 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:41 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:41.358+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.223626+0000 front 2023-08-25T09:17:18.223658+0000 (oldest deadline 2023-08-25T09:17:40.523772+0000) 2023-08-25T09:17:41.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:41 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:41.627+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:41.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:41 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:41.627+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:42.044 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:41 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:41.675+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:42.044 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:41 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:41.675+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:42.045 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:41 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:41.675+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:42.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:41 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:41.877+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:42.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:41 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:41.866+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:42.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:41 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:41.866+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:42.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:41 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:41.866+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:42.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:42 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:42.043+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:42.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:42 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:42.043+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:42.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:42 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:42.043+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:42.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:42 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:42.137+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:42.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:42 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:42.310+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:42.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:42 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:42.310+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:42.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:42 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:42.310+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.223626+0000 front 2023-08-25T09:17:18.223658+0000 (oldest deadline 2023-08-25T09:17:40.523772+0000) 2023-08-25T09:17:42.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:42 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:42.672+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:42.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:42 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:42.672+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:43.011 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:42 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:42.692+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:43.012 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:42 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:42.692+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:43.012 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:42 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:42.692+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:43.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:43 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:43.010+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:43.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:43 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:43.010+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:43.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:43 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:43.010+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:43.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:42 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:42.895+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:43.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:42 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:42.896+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:43.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:42 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:42.896+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:43.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:42 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:42.896+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:43.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:43 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:43.156+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:43.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:43 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:43.289+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:01.220979+0000 front 2023-08-25T09:17:01.220955+0000 (oldest deadline 2023-08-25T09:17:26.520942+0000) 2023-08-25T09:17:43.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:43 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:43.289+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.422319+0000 front 2023-08-25T09:17:09.422341+0000 (oldest deadline 2023-08-25T09:17:34.122318+0000) 2023-08-25T09:17:43.494 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:43 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5[142525]: 2023-08-25T09:17:43.289+0000 7fb04a4a0700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.223626+0000 front 2023-08-25T09:17:18.223658+0000 (oldest deadline 2023-08-25T09:17:40.523772+0000) 2023-08-25T09:17:43.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:43 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:43.649+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:43.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:43 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:43.649+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:44.056 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:43 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:43.709+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:44.057 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:43 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:43.709+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:44.057 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:43 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:43.709+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:44.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:44 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:44.056+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:44.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:44 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:44.056+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:44.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:44 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:44.056+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:44.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:43 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:43.936+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:44.381 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:43 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:43.936+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:44.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:43 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:43.882+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:44.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:43 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:43.882+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:44.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:43 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:43.882+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:44.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:44 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:44.169+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:44.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:44 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:44.169+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:44.889 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:44 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:44.626+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:44.889 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:44 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:44.626+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:44.889 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:44 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:44.626+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:45.045 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:44 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:44.662+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:45.045 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:44 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:44.662+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:45.046 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:44 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:44.662+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:45.167 INFO:journalctl@ceph.osd.5.smithi181.stdout:Aug 25 09:17:44 smithi181 bash[165407]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-5 2023-08-25T09:17:45.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:45 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:45.043+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:45.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:45 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:45.043+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:45.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:45 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:45.043+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:45.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:44 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:44.904+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:45.381 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:44 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:44.904+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:45.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:44 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:44.902+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:45.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:44 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:44.902+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:45.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:44 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:44.902+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:45.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:45 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:45.165+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:45.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:45 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:45.165+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:45.917 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:45 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:45.627+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:45.917 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:45 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:45.627+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:45.918 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:45 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:45.627+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:46.027 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:45 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:45.691+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:46.027 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:45 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:45.691+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:46.027 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:45 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:45.691+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:46.234 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:46 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:46.163+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:46.235 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:46 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:46.163+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:46.241 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.5.service' 2023-08-25T09:17:46.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:17:46.282 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-08-25T09:17:46.282 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-08-25T09:17:46.282 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.6 2023-08-25T09:17:46.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:46.025+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:46.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:46.025+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:46.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:46.025+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:46.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:45 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:45.902+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:46.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:45 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:45.902+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:46.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:45 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:45.913+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:46.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:45 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:45.913+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:46.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:45 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:45.913+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:46.709 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:46 smithi099 systemd[1]: Stopping Ceph osd.6 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:17:46.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:46 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:46.616+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:46.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:46 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:46.616+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:46.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:46 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:46.616+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:47.029 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:46.790+0000 7fccc2e5a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:17:47.029 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:46.790+0000 7fccc2e5a700 -1 osd.6 115 *** Got signal Terminated *** 2023-08-25T09:17:47.030 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:46.790+0000 7fccc2e5a700 -1 osd.6 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:17:47.030 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:46.707+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:47.030 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:46.707+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:47.030 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:46 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:46.707+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:47.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:47.027+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:47.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:47.027+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:47.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:47.027+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:47.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:46 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:46.881+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:47.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:46 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:46.881+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:47.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:46 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:46.949+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:47.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:46 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:46.949+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:47.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:46 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:46.949+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:47.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:47.192+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:47.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:47.192+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:47.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:47.642+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:47.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:47.642+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:47.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:47 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:47.642+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:48.009 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:47.737+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:48.009 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:47.737+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:48.009 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:47 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:47.737+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:48.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:47.869+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:48.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:47.869+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:48.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:47.977+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:48.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:47.977+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:48.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:47 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:47.977+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:48.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:48.007+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:48.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:48.007+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:48.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:48.007+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:48.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:48 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:48.160+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:48.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:48 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:48.160+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:48.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:48 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:48.666+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:48.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:48 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:48.666+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:48.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:48 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:48.666+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:49.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:48.781+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:49.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:48.781+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:49.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:48.781+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:49.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:48.968+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:49.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:48.968+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:49.065 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:48.968+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:49.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:48 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:48.883+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:49.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:48 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:48.883+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:49.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:48 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:48.984+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:49.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:48 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:48.984+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:49.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:48 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:48.984+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:49.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:49 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:49.196+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:49.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:49 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:49.196+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:49.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:49 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:49.702+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:49.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:49 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:49.702+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:49.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:49 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:49.702+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:50.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:49 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:49.785+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:50.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:49 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:49.785+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:50.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:49 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:49.785+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:50.064 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:49 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:49.973+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:50.065 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:49 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:49.973+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:50.065 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:49 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:49.973+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:50.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:49 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:49.855+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:50.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:49 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:49.855+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:50.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:50 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:50.015+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:50.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:50 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:50.015+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:50.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:50 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:50.015+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:50.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:50 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:50.187+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:50.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:50 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:50.187+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:50.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:50 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:50.709+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:50.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:50 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:50.709+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:50.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:50 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:50.709+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:51.025 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:50 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:50.768+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:51.025 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:50 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:50.768+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:51.025 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:50 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:50.768+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:51.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:51.023+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:16:58.527047+0000 front 2023-08-25T09:16:58.527009+0000 (oldest deadline 2023-08-25T09:17:30.327843+0000) 2023-08-25T09:17:51.313 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:51.023+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.328595+0000 front 2023-08-25T09:17:10.328574+0000 (oldest deadline 2023-08-25T09:17:34.428669+0000) 2023-08-25T09:17:51.314 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6[121542]: 2023-08-25T09:17:51.023+0000 7fccbbc49700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.930448+0000 front 2023-08-25T09:17:17.930433+0000 (oldest deadline 2023-08-25T09:17:41.430519+0000) 2023-08-25T09:17:51.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:50 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:50.900+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:51.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:50 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:50.900+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:51.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:51 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:51.051+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:51.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:51 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:51.051+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:51.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:51 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:51.051+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:51.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:51 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:51.146+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:51.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:51 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:51.146+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:51.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:51 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:51.708+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:51.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:51 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:51.708+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:51.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:51 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:51.708+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:52.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:51.753+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:52.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:51.753+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:52.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:51 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:51.753+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:52.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:51 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:51.861+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:52.130 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:51 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:51.861+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:52.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:52 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:52.084+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:52.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:52 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:52.084+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:52.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:52 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:52.084+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:52.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:52 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:52.084+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:17:52.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:52 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:52.116+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:52.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:52 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:52.116+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:52.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:52 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:52.116+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:17:52.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:52 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:52.674+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:52.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:52 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:52.674+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:52.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:52 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:52.674+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:53.063 INFO:journalctl@ceph.osd.6.smithi099.stdout:Aug 25 09:17:52 smithi099 bash[157071]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-6 2023-08-25T09:17:53.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:52 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:52.763+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:53.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:52 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:52.763+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:53.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:52 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:52.763+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:53.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:53.066+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:53.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:53.066+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:53.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:53.066+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:53.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:53.066+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:17:53.382 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:52 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:52.879+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:53.383 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:52 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:52.879+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:53.383 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:52 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:52.879+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.199743+0000 front 2023-08-25T09:17:28.199725+0000 (oldest deadline 2023-08-25T09:17:52.299619+0000) 2023-08-25T09:17:53.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:53 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:53.158+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:53.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:53 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:53.158+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:53.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:53 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:53.158+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:17:53.993 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:53 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:53.710+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:53.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:53 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:53.710+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:53.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:53 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:53.710+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:54.061 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:53 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:53.788+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:54.062 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:53 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:53.788+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:54.062 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:53 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:53.788+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:54.062 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:53 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:53.788+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:17:54.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:54 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:54.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:54.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:54 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:54.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:54.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:54 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:54.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:54.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:54 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:54.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:17:54.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:53.841+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:54.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:53.841+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:54.132 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:53 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:53.841+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.199743+0000 front 2023-08-25T09:17:28.199725+0000 (oldest deadline 2023-08-25T09:17:52.299619+0000) 2023-08-25T09:17:54.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:54 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:54.152+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:54.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:54 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:54.152+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:54.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:54 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:54.152+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:17:54.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.6.service' 2023-08-25T09:17:54.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:17:54.647 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-08-25T09:17:54.647 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-08-25T09:17:54.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.7 2023-08-25T09:17:54.763 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:54 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:54.761+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:54.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:54 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:54.761+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:55.014 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:54 smithi157 systemd[1]: Stopping Ceph osd.7 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:17:55.014 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:54 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:54.840+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:55.014 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:54 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:54.840+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:55.015 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:54 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:54.840+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.199743+0000 front 2023-08-25T09:17:28.199725+0000 (oldest deadline 2023-08-25T09:17:52.299619+0000) 2023-08-25T09:17:55.015 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:55.011+0000 7f774176c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:17:55.015 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:55.011+0000 7f774176c700 -1 osd.7 115 *** Got signal Terminated *** 2023-08-25T09:17:55.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:54 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:54.761+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:55.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:54 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:54.761+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:17:55.153 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:54 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:54.746+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:55.153 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:54 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:54.746+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:55.153 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:54 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:54.746+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:55.380 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:55.011+0000 7f774176c700 -1 osd.7 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:17:55.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:55.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:55.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:55.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:55.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:55.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:55.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:55.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:17:55.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:55 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:55.151+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:55.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:55 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:55.151+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:55.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:55 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:55.151+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:17:56.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:55 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:55.801+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:56.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:55 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:55.801+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:56.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:55 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:55.801+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:56.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:55 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:55.801+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:17:56.085 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:55.796+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:56.086 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:55.796+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:56.087 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:55 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:55.796+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.199743+0000 front 2023-08-25T09:17:28.199725+0000 (oldest deadline 2023-08-25T09:17:52.299619+0000) 2023-08-25T09:17:56.147 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:55 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:55.777+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:56.148 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:55 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:55.777+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:56.148 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:55 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:55.777+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:56.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:56.082+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:56.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:56.082+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:56.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:56.082+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:56.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:56.082+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:17:56.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:56 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:56.146+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:56.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:56 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:56.146+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:56.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:56 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:56.146+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:17:57.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:56 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:56.777+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:57.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:56 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:56.777+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:57.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:56 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:56.777+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:57.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:56 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:56.777+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:17:57.102 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:56.798+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:57.103 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:56.798+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:57.103 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:56.798+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.199743+0000 front 2023-08-25T09:17:28.199725+0000 (oldest deadline 2023-08-25T09:17:52.299619+0000) 2023-08-25T09:17:57.151 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:56 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:56.806+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:57.152 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:56 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:56.806+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:57.152 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:56 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:56.806+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:57.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:57 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:57.101+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:57.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:57 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:57.101+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:57.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:57 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:57.101+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:57.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:57 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:57.101+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:17:57.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:57.150+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:57.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:57.150+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:57.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:57.150+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:17:58.063 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:57 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:57.806+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:58.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:57 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:57.806+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:58.064 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:57 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:57.806+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:58.065 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:57 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:57.806+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:17:58.073 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:57 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:57.791+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:58.073 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:57 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:57.791+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:58.073 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:57 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:57.791+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.199743+0000 front 2023-08-25T09:17:28.199725+0000 (oldest deadline 2023-08-25T09:17:52.299619+0000) 2023-08-25T09:17:58.074 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:57 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:57.791+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:32.299967+0000 front 2023-08-25T09:17:32.299990+0000 (oldest deadline 2023-08-25T09:17:57.600187+0000) 2023-08-25T09:17:58.152 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:57.843+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:58.153 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:57.843+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:58.153 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:57.843+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:58.154 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:57 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:57.843+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:17:58.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:58 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:58.072+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:58.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:58 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:58.072+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:58.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:58 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:58.072+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:58.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:58 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:58.072+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:17:58.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:58.150+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:58.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:58.150+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:58.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:58.150+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:17:59.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:59.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:17:59.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:59.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:17:59.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:59.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:17:59.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:17:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:17:59.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:17:59.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:58 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:58.818+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:17:59.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:58 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:58.818+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:17:59.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:58 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:58.818+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.199743+0000 front 2023-08-25T09:17:28.199725+0000 (oldest deadline 2023-08-25T09:17:52.299619+0000) 2023-08-25T09:17:59.132 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:58 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:58.818+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:32.299967+0000 front 2023-08-25T09:17:32.299990+0000 (oldest deadline 2023-08-25T09:17:57.600187+0000) 2023-08-25T09:17:59.132 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:58.811+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:17:59.133 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:58.811+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:17:59.133 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:58.811+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:17:59.133 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:58 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:58.811+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:17:59.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:58 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:58.848+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:17:59.462 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:58 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:58.848+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:17:59.462 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:58 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:58.848+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:17:59.463 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:58 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:58.848+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:17:59.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:59.128+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:17:59.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:59.128+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:17:59.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:17:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:17:59.128+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:00.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:00.069+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:00.130 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:00.069+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:00.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:00.069+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:00.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:00.069+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:00.131 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:00.069+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:00.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:59.836+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.795078+0000 front 2023-08-25T09:17:10.795107+0000 (oldest deadline 2023-08-25T09:17:34.895306+0000) 2023-08-25T09:18:00.132 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:59.836+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.297430+0000 front 2023-08-25T09:17:18.297300+0000 (oldest deadline 2023-08-25T09:17:42.997445+0000) 2023-08-25T09:18:00.132 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:59.836+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.199743+0000 front 2023-08-25T09:17:28.199725+0000 (oldest deadline 2023-08-25T09:17:52.299619+0000) 2023-08-25T09:18:00.132 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:17:59 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7[146671]: 2023-08-25T09:17:59.836+0000 7f773c55f700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:32.299967+0000 front 2023-08-25T09:17:32.299990+0000 (oldest deadline 2023-08-25T09:17:57.600187+0000) 2023-08-25T09:18:00.155 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:59.768+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:00.155 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:59.768+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:00.155 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:59.768+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:00.156 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:17:59 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:17:59.768+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:00.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:59 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:59.882+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:00.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:59 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:59.882+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:00.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:59 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:59.882+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:00.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:17:59 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:17:59.882+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:00.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:00.153+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:18:00.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:00.153+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:18:00.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:00.153+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:01.023 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:01.020+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:01.024 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:01.020+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:01.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Aug 25 09:18:00 smithi157 bash[165071]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-7 2023-08-25T09:18:01.131 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:00.748+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:01.132 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:00.748+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:01.132 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:00.748+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:01.132 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:00 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:00.748+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:01.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:00.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:01.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:00.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:01.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:00.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:01.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:00 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:00.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:01.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:01.020+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:01.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:01.020+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:01.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:01 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:01.020+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:01.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:01.129+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:18:01.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:01.129+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:18:01.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:01.129+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:01.495 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:01.129+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:35.201504+0000 front 2023-08-25T09:17:35.201453+0000 (oldest deadline 2023-08-25T09:18:00.501450+0000) 2023-08-25T09:18:02.097 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:01.796+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:02.097 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:01.796+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:02.097 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:01.796+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:02.098 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:01 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:01.796+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:02.134 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:02.047+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:02.134 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:02.047+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:02.134 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:02.047+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:02.134 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:02.047+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:02.135 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:02 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:02.047+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:02.192 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.7.service' 2023-08-25T09:18:02.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:02.232 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-08-25T09:18:02.233 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-08-25T09:18:02.233 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.8 2023-08-25T09:18:02.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:01.876+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:02.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:01.876+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:02.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:01.876+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:02.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:01 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:01.876+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:02.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:02.095+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:18:02.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:02.095+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:18:02.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:02.095+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:02.495 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:02.095+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:35.201504+0000 front 2023-08-25T09:17:35.201453+0000 (oldest deadline 2023-08-25T09:18:00.501450+0000) 2023-08-25T09:18:02.495 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:02 smithi181 systemd[1]: Stopping Ceph osd.8 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:02.993 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:02.585+0000 7f0312255700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:02.994 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:02.585+0000 7f0312255700 -1 osd.8 115 *** Got signal Terminated *** 2023-08-25T09:18:02.994 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:02.585+0000 7f0312255700 -1 osd.8 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:18:02.994 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:02.823+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:02.995 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:02.823+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:02.995 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:02.823+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:02.995 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:02 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:02.823+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:03.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:02.867+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:03.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:02.867+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:03.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:02.867+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:03.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:02 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:02.867+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:03.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:03.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:03.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:03.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:03.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:03.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:03.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:03.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:03.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:03 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:03.056+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:03.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:03.085+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:18:03.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:03.085+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:18:03.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:03.085+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:03.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:03.085+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:35.201504+0000 front 2023-08-25T09:17:35.201453+0000 (oldest deadline 2023-08-25T09:18:00.501450+0000) 2023-08-25T09:18:04.054 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:03.797+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:04.055 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:03.797+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:04.055 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:03.797+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:04.055 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:03 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:03.797+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:04.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:03.863+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:04.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:03.863+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:04.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:03.863+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:04.315 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:03 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:03.863+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:04.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:04.075+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:04.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:04.075+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:04.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:04.075+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:04.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:04.075+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:04.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:04.075+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:04.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:04.053+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:18:04.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:04.053+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:18:04.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:04.053+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:04.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:04.053+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:35.201504+0000 front 2023-08-25T09:17:35.201453+0000 (oldest deadline 2023-08-25T09:18:00.501450+0000) 2023-08-25T09:18:05.059 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:04.805+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:05.060 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:04.805+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:05.060 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:04.805+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:05.061 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:04 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:04.805+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:05.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:04 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:04.889+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:05.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:04 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:04.889+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:05.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:04 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:04.889+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:05.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:04 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:04.889+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:05.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:05.027+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:05.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:05.027+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:05.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:05.027+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:05.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:05.027+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:05.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:05 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:05.027+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:05.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:05.057+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:18:05.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:05.057+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:18:05.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:05.057+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:05.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:05.057+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:35.201504+0000 front 2023-08-25T09:17:35.201453+0000 (oldest deadline 2023-08-25T09:18:00.501450+0000) 2023-08-25T09:18:06.027 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:05.758+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:06.028 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:05.758+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:06.028 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:05.759+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:06.028 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:05.759+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:06.029 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:05 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:05.759+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:06.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:05.913+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:06.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:05.913+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:06.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:05.913+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:06.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:05 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:05.913+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:06.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:06 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:06.031+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:06.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:06 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:06.031+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:06.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:06 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:06.031+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:06.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:06 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:06.031+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:06.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:06 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:06.031+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:06.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:06.026+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:18:06.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:06.026+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:18:06.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:06.026+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:06.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:06.026+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:35.201504+0000 front 2023-08-25T09:17:35.201453+0000 (oldest deadline 2023-08-25T09:18:00.501450+0000) 2023-08-25T09:18:06.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:06.026+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:40.501878+0000 front 2023-08-25T09:17:40.501900+0000 (oldest deadline 2023-08-25T09:18:05.102603+0000) 2023-08-25T09:18:07.025 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:06.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:07.025 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:06.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:07.025 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:06.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:07.026 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:06.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:07.026 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:06 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:06.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:07.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:06.959+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:07.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:06.959+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:07.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:06.959+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:07.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:06 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:06.959+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:07.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:07.008+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:07.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:07.008+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:07.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:07.008+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:07.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:07.008+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:07.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:07.008+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:07.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:07 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:07.008+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:07.493 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:07.023+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.195182+0000 front 2023-08-25T09:17:03.195230+0000 (oldest deadline 2023-08-25T09:17:28.495228+0000) 2023-08-25T09:18:07.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:07.023+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:14.397846+0000 front 2023-08-25T09:17:14.397858+0000 (oldest deadline 2023-08-25T09:17:43.796800+0000) 2023-08-25T09:18:07.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:07.023+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:28.299671+0000 front 2023-08-25T09:17:28.299645+0000 (oldest deadline 2023-08-25T09:17:51.799729+0000) 2023-08-25T09:18:07.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:07.023+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:35.201504+0000 front 2023-08-25T09:17:35.201453+0000 (oldest deadline 2023-08-25T09:18:00.501450+0000) 2023-08-25T09:18:07.494 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8[148040]: 2023-08-25T09:18:07.023+0000 7f030d048700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:40.501878+0000 front 2023-08-25T09:17:40.501900+0000 (oldest deadline 2023-08-25T09:18:05.102603+0000) 2023-08-25T09:18:08.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:07.795+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:08.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:07.795+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:08.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:07.795+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:08.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:07.795+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:08.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:07 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:07.795+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:08.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:07 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:07.934+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:08.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:07 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:07.934+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:08.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:07 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:07.934+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:08.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:07 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:07.934+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:08.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:07 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:07.934+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:38.845337+0000 front 2023-08-25T09:17:38.845389+0000 (oldest deadline 2023-08-25T09:18:07.646366+0000) 2023-08-25T09:18:08.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:08 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:08.023+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:08.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:08 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:08.023+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:08.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:08 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:08.023+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:08.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:08 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:08.023+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:08.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:08 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:08.023+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:08.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:08 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:08.023+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:08.744 INFO:journalctl@ceph.osd.8.smithi181.stdout:Aug 25 09:18:08 smithi181 bash[165673]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-8 2023-08-25T09:18:09.048 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:08 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:08.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:09.049 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:08 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:08.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:09.049 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:08 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:08.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:09.049 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:08 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:08.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:09.049 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:08 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:08.747+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:09.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:08.979+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:09.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:08.979+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:09.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:08.979+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:09.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:08.979+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:09.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:08 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:08.979+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:38.845337+0000 front 2023-08-25T09:17:38.845389+0000 (oldest deadline 2023-08-25T09:18:07.646366+0000) 2023-08-25T09:18:09.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:09 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:09.058+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:09.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:09 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:09.058+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:09.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:09 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:09.058+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:09.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:09 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:09.058+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:09.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:09 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:09.058+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:09.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:09 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:09.058+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:09.769 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.8.service' 2023-08-25T09:18:09.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:09.810 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-08-25T09:18:09.810 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-08-25T09:18:09.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.9 2023-08-25T09:18:10.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:09 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:09.763+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:10.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:09 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:09.763+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:10.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:09 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:09.763+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:10.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:09 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:09.763+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:10.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:09 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:09.763+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:10.311 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:09 smithi099 systemd[1]: Stopping Ceph osd.9 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:10.312 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:09 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:09.929+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:10.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:09 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:09.929+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:10.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:09 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:09.929+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:10.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:09 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:09.929+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:10.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:09 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:09.929+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:38.845337+0000 front 2023-08-25T09:17:38.845389+0000 (oldest deadline 2023-08-25T09:18:07.646366+0000) 2023-08-25T09:18:10.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:10 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:10.078+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:10.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:10 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:10.078+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:10.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:10 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:10.078+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:10.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:10 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:10.078+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:10.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:10 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:10.078+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:10.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:10 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:10.078+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:10.564 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:10.309+0000 7f55223b2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:10.565 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:10.309+0000 7f55223b2700 -1 osd.9 115 *** Got signal Terminated *** 2023-08-25T09:18:10.565 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:10.309+0000 7f55223b2700 -1 osd.9 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:18:11.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:10 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:10.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:11.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:10 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:10.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:11.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:10 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:10.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:11.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:10 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:10.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:11.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:10 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:10.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:11.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:10.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:11.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:10.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:11.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:10.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:11.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:10.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:11.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:10 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:10.912+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:38.845337+0000 front 2023-08-25T09:17:38.845389+0000 (oldest deadline 2023-08-25T09:18:07.646366+0000) 2023-08-25T09:18:11.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:11 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:11.041+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:11.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:11 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:11.041+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:11.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:11 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:11.041+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:11.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:11 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:11.041+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:11.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:11 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:11.041+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:11.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:11 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:11.041+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:12.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:11 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:11.823+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:12.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:11 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:11.824+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:12.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:11 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:11.824+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:12.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:11 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:11.824+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:12.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:11 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:11.824+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:12.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:11 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:11.872+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:12.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:11 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:11.872+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:12.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:11 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:11.872+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:12.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:11 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:11.872+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:12.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:11 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:11.872+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:38.845337+0000 front 2023-08-25T09:17:38.845389+0000 (oldest deadline 2023-08-25T09:18:07.646366+0000) 2023-08-25T09:18:12.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:12 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:12.016+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:12.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:12 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:12.016+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:12.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:12 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:12.016+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:12.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:12 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:12.016+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:12.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:12 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:12.016+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:12.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:12 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:12.016+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:13.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:12 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:12.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:13.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:12 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:12.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:13.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:12 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:12.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:13.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:12 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:12.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:13.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:12 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:12.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:13.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:12 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:12.921+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:13.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:12 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:12.921+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:13.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:12 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:12.921+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:13.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:12 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:12.921+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:13.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:12 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:12.921+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:38.845337+0000 front 2023-08-25T09:17:38.845389+0000 (oldest deadline 2023-08-25T09:18:07.646366+0000) 2023-08-25T09:18:13.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:13 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:13.048+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:13.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:13 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:13.048+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:13.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:13 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:13.048+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:13.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:13 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:13.048+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:13.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:13 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:13.048+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:13.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:13 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:13.048+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:14.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:13 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:13.846+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:14.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:13 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:13.846+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:14.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:13 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:13.846+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:14.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:13 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:13.846+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:14.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:13 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:13.846+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:14.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:13 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:13.955+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:14.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:13 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:13.955+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:14.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:13 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:13.955+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:14.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:13 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:13.955+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:14.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:13 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:13.955+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:38.845337+0000 front 2023-08-25T09:17:38.845389+0000 (oldest deadline 2023-08-25T09:18:07.646366+0000) 2023-08-25T09:18:14.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:14 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:14.071+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:14.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:14 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:14.071+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:14.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:14 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:14.071+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:14.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:14 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:14.071+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:14.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:14 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:14.071+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:14.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:14 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:14.071+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:15.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:14.858+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:15.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:14.858+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:15.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:14.858+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:15.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:14.858+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:15.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:14 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:14.858+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:15.313 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:14 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:14.994+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:03.534242+0000 front 2023-08-25T09:17:03.534448+0000 (oldest deadline 2023-08-25T09:17:27.634505+0000) 2023-08-25T09:18:15.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:14 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:14.994+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.937297+0000 front 2023-08-25T09:17:10.937327+0000 (oldest deadline 2023-08-25T09:17:33.237573+0000) 2023-08-25T09:18:15.314 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:14 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:14.994+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:17.240338+0000 front 2023-08-25T09:17:17.240452+0000 (oldest deadline 2023-08-25T09:17:40.740331+0000) 2023-08-25T09:18:15.315 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:14 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:14.994+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.643322+0000 front 2023-08-25T09:17:27.643297+0000 (oldest deadline 2023-08-25T09:17:53.543570+0000) 2023-08-25T09:18:15.315 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:14 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9[127114]: 2023-08-25T09:18:14.994+0000 7f551a9a0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:38.845337+0000 front 2023-08-25T09:17:38.845389+0000 (oldest deadline 2023-08-25T09:18:07.646366+0000) 2023-08-25T09:18:15.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:15 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:15.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:15.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:15 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:15.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:15.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:15 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:15.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:15.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:15 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:15.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:15.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:15 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:15.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:15.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:15 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:15.060+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:16.247 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:15 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:15.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:16.248 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:15 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:15.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:16.248 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:15 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:15.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:16.249 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:15 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:15.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:16.249 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:15 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:15.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:16.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:16 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:16.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:16.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:16 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:16.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:16.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:16 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:16.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:16.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:16 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:16.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:16.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:16 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:16.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:16.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:16 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:16.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:16.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:16 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:16.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.256422+0000 front 2023-08-25T09:17:50.256497+0000 (oldest deadline 2023-08-25T09:18:15.556236+0000) 2023-08-25T09:18:16.563 INFO:journalctl@ceph.osd.9.smithi099.stdout:Aug 25 09:18:16 smithi099 bash[157342]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-9 2023-08-25T09:18:17.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:16 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:16.862+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:17.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:16 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:16.862+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:17.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:16 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:16.862+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:17.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:16 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:16.862+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:17.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:16 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:16.862+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:17.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:16 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:16.862+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:17.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:17 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:17.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:17.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:17 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:17.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:17.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:17 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:17.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:17.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:17 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:17.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:17.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:17 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:17.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:17.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:17 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:17.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:17.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:17 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:17.021+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.256422+0000 front 2023-08-25T09:17:50.256497+0000 (oldest deadline 2023-08-25T09:18:15.556236+0000) 2023-08-25T09:18:18.102 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.9.service' 2023-08-25T09:18:18.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:18.193 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-08-25T09:18:18.194 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-08-25T09:18:18.194 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.10 2023-08-25T09:18:18.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:17 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:17.830+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:18.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:17 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:17.830+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:18.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:17 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:17.830+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:18.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:17 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:17.830+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:18.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:17 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:17.830+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:18.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:17 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:17.830+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:18.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.029+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:18.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.029+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:18.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.029+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:18.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.029+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:18.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.029+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:18.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.029+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:18.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.029+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.256422+0000 front 2023-08-25T09:17:50.256497+0000 (oldest deadline 2023-08-25T09:18:15.556236+0000) 2023-08-25T09:18:18.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 systemd[1]: Stopping Ceph osd.10 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:18.880 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.549+0000 7f9a885ed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:18.880 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.549+0000 7f9a885ed700 -1 osd.10 115 *** Got signal Terminated *** 2023-08-25T09:18:18.881 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.549+0000 7f9a885ed700 -1 osd.10 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:18:19.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:18 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:18.827+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:19.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:18 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:18.827+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:19.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:18 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:18.827+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:19.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:18 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:18.827+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:19.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:18 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:18.827+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:19.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:18 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:18.827+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:19.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.981+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:19.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.981+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:19.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.981+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:19.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.981+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:19.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.981+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:19.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.981+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:19.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:18 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:18.981+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.256422+0000 front 2023-08-25T09:17:50.256497+0000 (oldest deadline 2023-08-25T09:18:15.556236+0000) 2023-08-25T09:18:20.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:19 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:19.852+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:20.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:19 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:19.852+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:20.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:19 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:19.852+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:20.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:19 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:19.852+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:20.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:19 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:19.852+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:20.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:19 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:19.852+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:20.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:19 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:19.996+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:20.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:19 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:19.996+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:20.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:19 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:19.996+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:20.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:19 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:19.996+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:20.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:19 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:19.996+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:20.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:19 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:19.996+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:20.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:19 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:19.996+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.256422+0000 front 2023-08-25T09:17:50.256497+0000 (oldest deadline 2023-08-25T09:18:15.556236+0000) 2023-08-25T09:18:21.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:20 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:20.854+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:21.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:20 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:20.854+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:21.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:20 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:20.854+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:21.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:20 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:20.854+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:21.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:20 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:20.854+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:21.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:20 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:20.854+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:21.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.044+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:21.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.044+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:21.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.044+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:21.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.044+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:21.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.044+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:21.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.044+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:21.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.044+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.256422+0000 front 2023-08-25T09:17:50.256497+0000 (oldest deadline 2023-08-25T09:18:15.556236+0000) 2023-08-25T09:18:22.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:21 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:21.828+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:22.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:21 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:21.828+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:22.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:21 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:21.828+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:22.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:21 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:21.828+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:22.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:21 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:21.828+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:22.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:21 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:21.828+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:22.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:21 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:21.828+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:22.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.995+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:22.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.995+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:22.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.995+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:22.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.995+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:22.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.995+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:22.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.995+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:22.382 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:21 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:21.995+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.256422+0000 front 2023-08-25T09:17:50.256497+0000 (oldest deadline 2023-08-25T09:18:15.556236+0000) 2023-08-25T09:18:23.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:22 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:22.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:23.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:22 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:22.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:23.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:22 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:22.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:23.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:22 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:22.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:23.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:22 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:22.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:23.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:22 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:22.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:23.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:22 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:22.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:23.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:22 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:22.988+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.948450+0000 front 2023-08-25T09:17:02.948512+0000 (oldest deadline 2023-08-25T09:17:28.248204+0000) 2023-08-25T09:18:23.380 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:22 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:22.988+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:09.949466+0000 front 2023-08-25T09:17:09.949675+0000 (oldest deadline 2023-08-25T09:17:35.849638+0000) 2023-08-25T09:18:23.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:22 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:22.988+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:15.850153+0000 front 2023-08-25T09:17:15.850247+0000 (oldest deadline 2023-08-25T09:17:41.150270+0000) 2023-08-25T09:18:23.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:22 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:22.988+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2023-08-25T09:17:27.552084+0000 front 2023-08-25T09:17:27.552215+0000 (oldest deadline 2023-08-25T09:17:51.651910+0000) 2023-08-25T09:18:23.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:22 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:22.988+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.253718+0000 front 2023-08-25T09:17:36.253767+0000 (oldest deadline 2023-08-25T09:17:59.154050+0000) 2023-08-25T09:18:23.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:22 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:22.988+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:43.255357+0000 front 2023-08-25T09:17:43.255458+0000 (oldest deadline 2023-08-25T09:18:06.155148+0000) 2023-08-25T09:18:23.381 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:22 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10[152220]: 2023-08-25T09:18:22.988+0000 7f9a83be1700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.256422+0000 front 2023-08-25T09:17:50.256497+0000 (oldest deadline 2023-08-25T09:18:15.556236+0000) 2023-08-25T09:18:24.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:23.865+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:24.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:23.865+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:24.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:23.865+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:24.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:23.865+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:24.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:23.865+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:24.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:23.865+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:24.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:23 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:23.865+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:24.533 INFO:journalctl@ceph.osd.10.smithi157.stdout:Aug 25 09:18:24 smithi157 bash[165339]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-10 2023-08-25T09:18:25.243 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:24 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:24.829+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:25.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:24 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:24.829+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:25.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:24 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:24.829+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:25.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:24 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:24.829+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:25.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:24 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:24.829+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:25.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:24 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:24.829+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:25.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:24 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:24.829+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:25.900 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.10.service' 2023-08-25T09:18:25.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:25.942 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-08-25T09:18:25.943 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-08-25T09:18:25.943 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.11 2023-08-25T09:18:26.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:25 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:25.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:26.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:25 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:25.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:26.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:25 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:25.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:26.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:25 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:25.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:26.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:25 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:25.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:26.247 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:25 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:25.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:26.247 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:25 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:25.783+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:26.247 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:25 smithi181 systemd[1]: Stopping Ceph osd.11 for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:26.744 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.299+0000 7f6fb6a9e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:26.744 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.299+0000 7f6fb6a9e700 -1 osd.11 115 *** Got signal Terminated *** 2023-08-25T09:18:26.744 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.299+0000 7f6fb6a9e700 -1 osd.11 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T09:18:27.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.814+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:27.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.814+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:27.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.814+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:27.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.814+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:27.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.814+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:27.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.814+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:27.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:26 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:26.814+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:28.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:27.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:28.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:27.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:28.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:27.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:28.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:27.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:28.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:27.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:28.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:27.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:28.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:27 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:27.845+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:29.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:28.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:29.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:28.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:29.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:28.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:29.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:28.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:29.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:28.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:29.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:28.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:29.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:28 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:28.834+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:30.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:29.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:30.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:29.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:30.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:29.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:30.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:29.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:30.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:29.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:30.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:29.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:30.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:29 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:29.842+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:31.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:30.808+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2023-08-25T09:17:02.265948+0000 front 2023-08-25T09:17:02.266147+0000 (oldest deadline 2023-08-25T09:17:24.566126+0000) 2023-08-25T09:18:31.244 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:30.808+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6804 osd.1 since back 2023-08-25T09:17:10.368187+0000 front 2023-08-25T09:17:10.367914+0000 (oldest deadline 2023-08-25T09:17:33.268073+0000) 2023-08-25T09:18:31.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:30.808+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-08-25T09:17:18.569449+0000 front 2023-08-25T09:17:18.569352+0000 (oldest deadline 2023-08-25T09:17:44.469535+0000) 2023-08-25T09:18:31.245 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:30.808+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6812 osd.4 since back 2023-08-25T09:17:36.172255+0000 front 2023-08-25T09:17:36.172071+0000 (oldest deadline 2023-08-25T09:17:57.272102+0000) 2023-08-25T09:18:31.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:30.808+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-08-25T09:17:39.573075+0000 front 2023-08-25T09:17:39.573126+0000 (oldest deadline 2023-08-25T09:18:04.873243+0000) 2023-08-25T09:18:31.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:30.808+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2023-08-25T09:17:50.575710+0000 front 2023-08-25T09:17:50.575755+0000 (oldest deadline 2023-08-25T09:18:16.475557+0000) 2023-08-25T09:18:31.246 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:30 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11[153575]: 2023-08-25T09:18:30.808+0000 7f6faf88d700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.157:6820 osd.7 since back 2023-08-25T09:17:58.677551+0000 front 2023-08-25T09:17:58.677661+0000 (oldest deadline 2023-08-25T09:18:20.977452+0000) 2023-08-25T09:18:32.183 INFO:journalctl@ceph.osd.11.smithi181.stdout:Aug 25 09:18:32 smithi181 bash[165951]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-osd-11 2023-08-25T09:18:33.449 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@osd.11.service' 2023-08-25T09:18:33.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:33.489 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-08-25T09:18:33.490 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-08-25T09:18:33.490 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.a 2023-08-25T09:18:33.813 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 09:18:33 smithi099 systemd[1]: Stopping Ceph mds.a for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:34.313 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 09:18:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-a[132437]: 2023-08-25T09:18:33.987+0000 7f8d98f9b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:34.313 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 09:18:33 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-a[132437]: 2023-08-25T09:18:33.987+0000 7f8d98f9b700 -1 mds.a *** got signal Terminated *** 2023-08-25T09:18:38.563 INFO:journalctl@ceph.mds.a.smithi099.stdout:Aug 25 09:18:38 smithi099 bash[157616]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-a 2023-08-25T09:18:38.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.a.service' 2023-08-25T09:18:38.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:38.791 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-08-25T09:18:38.792 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-08-25T09:18:38.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.d 2023-08-25T09:18:39.063 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 09:18:38 smithi099 systemd[1]: Stopping Ceph mds.d for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:39.563 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 09:18:39 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-d[131436]: 2023-08-25T09:18:39.277+0000 7fdf7b078700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:39.564 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 09:18:39 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-d[131436]: 2023-08-25T09:18:39.277+0000 7fdf7b078700 -1 mds.d *** got signal Terminated *** 2023-08-25T09:18:42.063 INFO:journalctl@ceph.mds.d.smithi099.stdout:Aug 25 09:18:41 smithi099 bash[157761]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-d 2023-08-25T09:18:42.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.d.service' 2023-08-25T09:18:42.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:42.301 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-08-25T09:18:42.301 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-08-25T09:18:42.302 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.g 2023-08-25T09:18:42.563 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 09:18:42 smithi099 systemd[1]: Stopping Ceph mds.g for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:43.063 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 09:18:42 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-g[132054]: 2023-08-25T09:18:42.785+0000 7fca383fa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:43.063 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 09:18:42 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-g[132054]: 2023-08-25T09:18:42.785+0000 7fca383fa700 -1 mds.g *** got signal Terminated *** 2023-08-25T09:18:47.313 INFO:journalctl@ceph.mds.g.smithi099.stdout:Aug 25 09:18:46 smithi099 bash[157910]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-g 2023-08-25T09:18:47.437 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.g.service' 2023-08-25T09:18:47.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:47.518 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-08-25T09:18:47.518 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-08-25T09:18:47.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.j 2023-08-25T09:18:47.813 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 09:18:47 smithi099 systemd[1]: Stopping Ceph mds.j for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:48.313 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 09:18:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-j[131050]: 2023-08-25T09:18:48.009+0000 7f416477b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:48.313 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 09:18:48 smithi099 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-j[131050]: 2023-08-25T09:18:48.009+0000 7f416477b700 -1 mds.j *** got signal Terminated *** 2023-08-25T09:18:50.581 INFO:journalctl@ceph.mds.j.smithi099.stdout:Aug 25 09:18:50 smithi099 bash[158049]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-j 2023-08-25T09:18:50.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.j.service' 2023-08-25T09:18:50.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:50.966 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-08-25T09:18:50.967 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-08-25T09:18:50.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.b 2023-08-25T09:18:51.338 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 09:18:51 smithi157 systemd[1]: Stopping Ceph mds.b for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:51.630 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 09:18:51 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-b[155907]: 2023-08-25T09:18:51.335+0000 7f91c7e5c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:51.630 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 09:18:51 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-b[155907]: 2023-08-25T09:18:51.335+0000 7f91c7e5c700 -1 mds.b *** got signal Terminated *** 2023-08-25T09:18:55.630 INFO:journalctl@ceph.mds.b.smithi157.stdout:Aug 25 09:18:55 smithi157 bash[165613]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-b 2023-08-25T09:18:55.714 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.b.service' 2023-08-25T09:18:55.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:55.752 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-08-25T09:18:55.752 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-08-25T09:18:55.753 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.e 2023-08-25T09:18:56.130 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 09:18:55 smithi157 systemd[1]: Stopping Ceph mds.e for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:18:56.630 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 09:18:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-e[155225]: 2023-08-25T09:18:56.134+0000 7f0a1816d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:18:56.630 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 09:18:56 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-e[155225]: 2023-08-25T09:18:56.134+0000 7f0a1816d700 -1 mds.e *** got signal Terminated *** 2023-08-25T09:18:59.630 INFO:journalctl@ceph.mds.e.smithi157.stdout:Aug 25 09:18:59 smithi157 bash[165760]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-e 2023-08-25T09:18:59.701 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.e.service' 2023-08-25T09:18:59.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:18:59.734 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-08-25T09:18:59.735 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-08-25T09:18:59.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.h 2023-08-25T09:19:00.127 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:18:59 smithi157 systemd[1]: Stopping Ceph mds.h for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:19:00.380 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:19:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[162818]: 2023-08-25T09:19:00.125+0000 7fb192ec5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:19:00.380 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:19:00 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h[162818]: 2023-08-25T09:19:00.125+0000 7fb192ec5700 -1 mds.h *** got signal Terminated *** 2023-08-25T09:19:03.630 INFO:journalctl@ceph.mds.h.smithi157.stdout:Aug 25 09:19:03 smithi157 bash[165905]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-h 2023-08-25T09:19:03.728 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.h.service' 2023-08-25T09:19:03.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:19:03.764 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-08-25T09:19:03.764 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-08-25T09:19:03.765 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.k 2023-08-25T09:19:04.130 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 09:19:03 smithi157 systemd[1]: Stopping Ceph mds.k for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:19:04.630 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 09:19:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-k[154885]: 2023-08-25T09:19:04.165+0000 7fb5e7559700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:19:04.630 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 09:19:04 smithi157 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-k[154885]: 2023-08-25T09:19:04.165+0000 7fb5e7559700 -1 mds.k *** got signal Terminated *** 2023-08-25T09:19:07.408 INFO:journalctl@ceph.mds.k.smithi157.stdout:Aug 25 09:19:07 smithi157 bash[166052]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-k 2023-08-25T09:19:07.611 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.k.service' 2023-08-25T09:19:07.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:19:07.646 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-08-25T09:19:07.647 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-08-25T09:19:07.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.c 2023-08-25T09:19:07.994 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 09:19:07 smithi181 systemd[1]: Stopping Ceph mds.c for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:19:08.493 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 09:19:08 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-c[157078]: 2023-08-25T09:19:08.017+0000 7fb746155700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:19:08.494 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 09:19:08 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-c[157078]: 2023-08-25T09:19:08.017+0000 7fb746155700 -1 mds.c *** got signal Terminated *** 2023-08-25T09:19:11.646 INFO:journalctl@ceph.mds.c.smithi181.stdout:Aug 25 09:19:11 smithi181 bash[166209]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-c 2023-08-25T09:19:11.654 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.c.service' 2023-08-25T09:19:11.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:19:11.694 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-08-25T09:19:11.695 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-08-25T09:19:11.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.f 2023-08-25T09:19:11.994 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 09:19:11 smithi181 systemd[1]: Stopping Ceph mds.f for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:19:12.493 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 09:19:12 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-f[156735]: 2023-08-25T09:19:12.108+0000 7f3ae05f3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:19:12.494 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 09:19:12 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-f[156735]: 2023-08-25T09:19:12.108+0000 7f3ae05f3700 -1 mds.f *** got signal Terminated *** 2023-08-25T09:19:15.493 INFO:journalctl@ceph.mds.f.smithi181.stdout:Aug 25 09:19:15 smithi181 bash[166370]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-f 2023-08-25T09:19:15.634 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.f.service' 2023-08-25T09:19:15.669 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:19:15.669 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-08-25T09:19:15.670 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-08-25T09:19:15.670 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.i 2023-08-25T09:19:15.993 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:19:15 smithi181 systemd[1]: Stopping Ceph mds.i for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:19:16.493 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:19:16 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-i[157419]: 2023-08-25T09:19:16.049+0000 7f3fc0a57700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:19:16.494 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:19:16 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-i[157419]: 2023-08-25T09:19:16.049+0000 7f3fc0a57700 -1 mds.i *** got signal Terminated *** 2023-08-25T09:19:19.417 INFO:journalctl@ceph.mds.i.smithi181.stdout:Aug 25 09:19:19 smithi181 bash[166510]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-i 2023-08-25T09:19:19.600 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.i.service' 2023-08-25T09:19:19.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:19:19.635 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-08-25T09:19:19.635 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-08-25T09:19:19.636 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.l 2023-08-25T09:19:19.993 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 09:19:19 smithi181 systemd[1]: Stopping Ceph mds.l for 3d4188ec-4324-11ee-9b3d-001a4aab830c... 2023-08-25T09:19:20.493 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 09:19:20 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-l[156391]: 2023-08-25T09:19:20.014+0000 7f1d847bb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T09:19:20.494 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 09:19:20 smithi181 ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-l[156391]: 2023-08-25T09:19:20.014+0000 7f1d847bb700 -1 mds.l *** got signal Terminated *** 2023-08-25T09:19:24.382 INFO:journalctl@ceph.mds.l.smithi181.stdout:Aug 25 09:19:24 smithi181 bash[166649]: ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c-mds-l 2023-08-25T09:19:24.586 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d4188ec-4324-11ee-9b3d-001a4aab830c@mds.l.service' 2023-08-25T09:19:24.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T09:19:24.621 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-08-25T09:19:24.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c --force --keep-logs 2023-08-25T09:19:25.015 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T09:19:28.129 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c --force --keep-logs 2023-08-25T09:19:28.427 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T09:19:30.010 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c --force --keep-logs 2023-08-25T09:19:30.311 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T09:19:31.908 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T09:19:31.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T09:19:32.023 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T09:19:32.054 INFO:tasks.cephadm:Archiving crash dumps... 2023-08-25T09:19:32.057 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi099/crash 2023-08-25T09:19:32.058 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash -- . 2023-08-25T09:19:32.138 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-25T09:19:32.139 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2023-08-25T09:19:32.141 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi157/crash 2023-08-25T09:19:32.143 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash -- . 2023-08-25T09:19:32.188 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-25T09:19:32.188 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2023-08-25T09:19:32.196 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi181/crash 2023-08-25T09:19:32.197 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash -- . 2023-08-25T09:19:32.244 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-25T09:19:32.245 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2023-08-25T09:19:32.247 INFO:tasks.cephadm:Checking cluster log for badness... 2023-08-25T09:19:32.247 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-08-25T09:19:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:grep: /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph.log: No such file or directory 2023-08-25T09:19:32.329 INFO:tasks.cephadm:Compressing logs... 2023-08-25T09:19:32.329 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T09:19:32.334 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T09:19:32.337 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T09:19:32.376 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-08-25T09:19:32.377 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-25T09:19:32.378 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-08-25T09:19:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-25T09:19:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.5.log 2023-08-25T09:19:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.2.log 2023-08-25T09:19:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.11.log 2023-08-25T09:19:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.z.log 2023-08-25T09:19:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.8.log 2023-08-25T09:19:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.f.log 2023-08-25T09:19:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.c.log 2023-08-25T09:19:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.i.log 2023-08-25T09:19:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.c.log 2023-08-25T09:19:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log 2023-08-25T09:19:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.l.log 2023-08-25T09:19:32.384 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.10.log 2023-08-25T09:19:32.384 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.h.log 2023-08-25T09:19:32.385 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.e.log 2023-08-25T09:19:32.385 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.4.log 2023-08-25T09:19:32.385 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.1.log 2023-08-25T09:19:32.385 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.k.log 2023-08-25T09:19:32.386 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log 2023-08-25T09:19:32.386 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.c.log: /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log: /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.l.log.gz 2023-08-25T09:19:32.387 INFO:teuthology.orchestra.run.smithi181.stderr: 92.2% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.z.log.gz 2023-08-25T09:19:32.388 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.7.log 2023-08-25T09:19:32.388 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.b.log 2023-08-25T09:19:32.388 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.b.log 2023-08-25T09:19:32.388 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.y.log 2023-08-25T09:19:32.389 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.b.log: /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.y.log: 94.4% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.k.log.gz 2023-08-25T09:19:32.393 INFO:teuthology.orchestra.run.smithi157.stderr: 92.2% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.y.log.gz 2023-08-25T09:19:32.396 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-25T09:19:32.397 INFO:teuthology.orchestra.run.smithi099.stderr:: No such file or directory 2023-08-25T09:19:32.398 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.0.log 2023-08-25T09:19:32.398 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.g.log 2023-08-25T09:19:32.399 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.a.log 2023-08-25T09:19:32.400 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.6.log 2023-08-25T09:19:32.402 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.3.log 2023-08-25T09:19:32.404 INFO:teuthology.orchestra.run.smithi157.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-25T09:19:32.404 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.d.log 2023-08-25T09:19:32.405 INFO:teuthology.orchestra.run.smithi181.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-25T09:19:32.406 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.3.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-25T09:19:32.406 INFO:teuthology.orchestra.run.smithi099.stderr: 94.4% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.g.log.gz 2023-08-25T09:19:32.407 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.x.log 2023-08-25T09:19:32.407 INFO:teuthology.orchestra.run.smithi099.stderr: 94.5% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.a.log.gz 2023-08-25T09:19:32.408 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.9.log 2023-08-25T09:19:32.409 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log 2023-08-25T09:19:32.411 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.j.log 2023-08-25T09:19:32.412 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log: 94.3% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.d.log.gz 2023-08-25T09:19:32.414 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.a.log 2023-08-25T09:19:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.a.log: 91.3% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log.gz 2023-08-25T09:19:32.419 INFO:teuthology.orchestra.run.smithi099.stderr: 94.6% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.j.log.gz 2023-08-25T09:19:32.819 INFO:teuthology.orchestra.run.smithi099.stderr: 89.7% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mgr.x.log.gz 2023-08-25T09:19:32.836 INFO:teuthology.orchestra.run.smithi181.stderr: 91.2% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log.gz 2023-08-25T09:19:33.460 INFO:teuthology.orchestra.run.smithi099.stderr: 90.3% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.a.log.gz 2023-08-25T09:19:33.495 INFO:teuthology.orchestra.run.smithi157.stderr: 91.9% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.b.log.gz 2023-08-25T09:19:33.768 INFO:teuthology.orchestra.run.smithi157.stderr: 91.2% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-volume.log.gz 2023-08-25T09:19:36.388 INFO:teuthology.orchestra.run.smithi181.stderr: 93.4% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mon.c.log.gz 2023-08-25T09:19:41.652 INFO:teuthology.orchestra.run.smithi157.stderr: 95.1% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.e.log.gz 2023-08-25T09:19:42.539 INFO:teuthology.orchestra.run.smithi157.stderr: 95.1% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.h.log.gz 2023-08-25T09:19:47.948 INFO:teuthology.orchestra.run.smithi181.stderr: 94.7% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.c.log.gz 2023-08-25T09:19:53.352 INFO:teuthology.orchestra.run.smithi099.stderr: 91.1% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.9.log.gz 2023-08-25T09:19:54.919 INFO:teuthology.orchestra.run.smithi099.stderr: 91.5% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.0.log.gz 2023-08-25T09:19:56.425 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.6.log.gz 2023-08-25T09:19:56.568 INFO:teuthology.orchestra.run.smithi157.stderr: 91.5% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.1.log.gz 2023-08-25T09:19:59.719 INFO:teuthology.orchestra.run.smithi099.stderr: 91.2% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.3.log.gz 2023-08-25T09:19:59.722 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-08-25T09:19:59.723 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m27.352s 2023-08-25T09:19:59.723 INFO:teuthology.orchestra.run.smithi099.stderr:user 1m29.340s 2023-08-25T09:19:59.723 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m7.021s 2023-08-25T09:20:02.396 INFO:teuthology.orchestra.run.smithi157.stderr: 91.3% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.4.log.gz 2023-08-25T09:20:05.340 INFO:teuthology.orchestra.run.smithi157.stderr: 91.2% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.7.log.gz 2023-08-25T09:20:07.559 INFO:teuthology.orchestra.run.smithi157.stderr: 91.1% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.10.log.gz 2023-08-25T09:20:16.803 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.11.log.gz 2023-08-25T09:20:21.870 INFO:teuthology.orchestra.run.smithi181.stderr: 91.2% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.2.log.gz 2023-08-25T09:20:22.223 INFO:teuthology.orchestra.run.smithi181.stderr: 90.9% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.5.log.gz 2023-08-25T09:20:25.414 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-osd.8.log.gz 2023-08-25T09:21:53.431 INFO:teuthology.orchestra.run.smithi157.stderr: 93.3% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.b.log.gz 2023-08-25T09:21:53.433 INFO:teuthology.orchestra.run.smithi157.stderr: 2023-08-25T09:21:53.433 INFO:teuthology.orchestra.run.smithi157.stderr:real 2m21.079s 2023-08-25T09:21:53.434 INFO:teuthology.orchestra.run.smithi157.stderr:user 4m3.309s 2023-08-25T09:21:53.434 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m8.343s 2023-08-25T09:22:11.905 INFO:teuthology.orchestra.run.smithi181.stderr: 92.9% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.i.log.gz 2023-08-25T09:22:14.724 INFO:teuthology.orchestra.run.smithi181.stderr: 93.1% -- replaced with /var/log/ceph/3d4188ec-4324-11ee-9b3d-001a4aab830c/ceph-mds.f.log.gz 2023-08-25T09:22:14.744 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-08-25T09:22:14.745 INFO:teuthology.orchestra.run.smithi181.stderr:real 2m42.390s 2023-08-25T09:22:14.745 INFO:teuthology.orchestra.run.smithi181.stderr:user 5m21.570s 2023-08-25T09:22:14.745 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m11.809s 2023-08-25T09:22:14.746 INFO:tasks.cephadm:Archiving logs... 2023-08-25T09:22:14.747 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi099/log 2023-08-25T09:22:14.756 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T09:22:27.514 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi157/log 2023-08-25T09:22:27.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T09:22:57.512 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi181/log 2023-08-25T09:22:57.514 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T09:23:37.523 INFO:tasks.cephadm:Removing cluster... 2023-08-25T09:23:37.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c --force 2023-08-25T09:23:37.914 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T09:23:38.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c --force 2023-08-25T09:23:39.289 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T09:23:40.065 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d4188ec-4324-11ee-9b3d-001a4aab830c --force 2023-08-25T09:23:40.820 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 3d4188ec-4324-11ee-9b3d-001a4aab830c 2023-08-25T09:23:41.691 INFO:tasks.cephadm:Removing cephadm ... 2023-08-25T09:23:41.691 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-25T09:23:41.740 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-25T09:23:41.771 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-25T09:23:41.799 INFO:tasks.cephadm:Teardown complete 2023-08-25T09:23:41.800 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-25T09:23:41.811 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-25T09:23:41.812 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T09:23:41.821 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T09:23:41.823 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T09:23:41.953 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-25T09:23:41.965 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-25T09:23:41.966 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T09:23:41.971 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T09:23:41.974 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T09:23:41.990 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-08-25T09:23:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-08-25T09:23:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-08-25T09:23:42.009 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T09:23:42.010 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-08-25T09:23:42.010 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 +2008us[+2008us] +/- 87ms 2023-08-25T09:23:42.010 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -1648us[-1646us] +/- 71ms 2023-08-25T09:23:42.010 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +185us[ +185us] +/- 85ms 2023-08-25T09:23:42.010 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:23:42.016 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T09:23:42.016 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-08-25T09:23:42.017 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +1991us[+1991us] +/- 87ms 2023-08-25T09:23:42.017 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -1678us[-1676us] +/- 71ms 2023-08-25T09:23:42.018 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +146us[ +146us] +/- 85ms 2023-08-25T09:23:42.018 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:23:42.019 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T09:23:42.019 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-08-25T09:23:42.020 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +1959us[+1979us] +/- 87ms 2023-08-25T09:23:42.020 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -1692us[-1672us] +/- 71ms 2023-08-25T09:23:42.020 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +139us[ +158us] +/- 85ms 2023-08-25T09:23:42.021 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:23:42.022 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-25T09:23:42.032 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-25T09:23:42.033 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-25T09:23:42.050 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T09:23:42.230 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T09:23:42.346 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T09:23:42.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T09:23:42.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:23:42.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T09:23:42.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:23:42.604 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T09:23:42.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:23:42.637 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-25T09:23:42.651 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-25T09:23:42.668 INFO:teuthology.task.internal:Duration was 3021.084269 seconds 2023-08-25T09:23:42.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-25T09:23:42.683 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-25T09:23:42.684 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T09:23:42.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T09:23:42.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T09:23:42.768 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T09:23:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T09:23:42.800 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T09:23:43.018 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-25T09:23:43.019 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2023-08-25T09:23:43.019 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T09:23:43.078 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2023-08-25T09:23:43.078 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T09:23:43.110 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2023-08-25T09:23:43.110 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T09:23:43.145 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-25T09:23:43.146 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T09:23:43.150 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T09:23:43.153 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T09:23:43.243 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-25T09:23:43.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T09:23:43.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T09:23:43.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T09:23:43.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-25T09:23:43.418 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-25T09:23:43.418 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T09:23:43.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T09:23:43.528 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T09:23:43.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-25T09:23:43.589 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:23:43.594 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:23:43.596 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:23:43.628 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2023-08-25T09:23:43.641 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2023-08-25T09:23:43.657 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2023-08-25T09:23:43.692 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:23:43.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:23:43.734 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:23:43.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:23:43.751 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:23:43.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:23:43.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-25T09:23:43.780 INFO:teuthology.task.internal:Transferring archived files... 2023-08-25T09:23:43.781 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi099 2023-08-25T09:23:43.782 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T09:23:43.862 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi157 2023-08-25T09:23:43.862 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T09:23:43.900 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270/remote/smithi181 2023-08-25T09:23:43.900 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T09:23:43.942 INFO:teuthology.task.internal:Removing archive directory... 2023-08-25T09:23:43.942 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T09:23:43.947 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T09:23:43.949 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T09:23:43.992 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-25T09:23:44.006 INFO:teuthology.task.internal:Not uploading archives. 2023-08-25T09:23:44.006 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-25T09:23:44.025 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-25T09:23:44.025 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T09:23:44.030 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T09:23:44.032 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T09:23:44.046 INFO:teuthology.orchestra.run.smithi157.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 09:23 /home/ubuntu/cephtest 2023-08-25T09:23:44.048 INFO:teuthology.orchestra.run.smithi181.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 09:23 /home/ubuntu/cephtest 2023-08-25T09:23:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 09:23 /home/ubuntu/cephtest 2023-08-25T09:23:44.071 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-25T09:23:44.084 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-25T09:23:44.111 INFO:teuthology.nuke:Checking targets against current locks 2023-08-25T09:23:44.137 DEBUG:teuthology.nuke:shortname: smithi099 2023-08-25T09:23:44.138 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T09:23:44.160 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 08:24:41.311634', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-25T09:23:44.164 DEBUG:teuthology.nuke:shortname: smithi157 2023-08-25T09:23:44.164 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T09:23:44.184 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 08:24:41.312701', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-25T09:23:44.192 DEBUG:teuthology.nuke:shortname: smithi181 2023-08-25T09:23:44.192 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T09:23:44.220 INFO:teuthology.orchestra.console.smithi099:Power off 2023-08-25T09:23:44.222 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T09:23:44.242 INFO:teuthology.orchestra.console.smithi157:Power off 2023-08-25T09:23:44.242 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T09:23:44.253 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 08:24:41.309983', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-25T09:23:44.257 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2023-08-25T09:23:44.258 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:44.269 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-08-25T09:23:44.270 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:44.285 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-08-25T09:23:44.294 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-08-25T09:23:44.313 INFO:teuthology.orchestra.console.smithi181:Power off 2023-08-25T09:23:44.314 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T09:23:44.337 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-08-25T09:23:44.337 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:44.359 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-08-25T09:23:48.286 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:48.298 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:48.360 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:48.401 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-08-25T09:23:48.411 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-08-25T09:23:48.472 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-08-25T09:23:52.404 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:52.416 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:52.473 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:52.518 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-08-25T09:23:52.528 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-08-25T09:23:52.585 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-08-25T09:23:52.629 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-08-25T09:23:56.520 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:56.586 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:23:56.632 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-08-25T09:23:56.698 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-08-25T09:23:56.734 INFO:teuthology.orchestra.console.smithi099:Power off completed 2023-08-25T09:23:56.799 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-08-25T09:23:56.906 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3021.084269285202 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi099 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4b369cf18ed1391a426ab4ae86da834e9c074f81 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a701c7b72d714a6f8768ccd22ebc9244 status: fail success: false 2023-08-25T09:23:56.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-25T09:23:57.000 INFO:teuthology.run:FAIL